There definitely comes a time when you want to add some variety to your basic layout, ie. switches, sidings, reverse loops, etc.
Theoretically, with perfect track and perfect connections, with no voltage losses no matter how small, the wallpack could power a layout of unlimited physical size. Alas, we do not live in a perfect world, so there are some losses that add up to limit the physical size of a layout that any one transformer could properly power.
Losses occur mostly at the connections between tracks, with either loose or dirty/corroded connections. Keeping the connections clean is of utmost importance. With standard transformers using standard terminal tracks, multiple terminal tracks are spread out around the layout; perhaps 4 or 6 could be used. These terminal tracks are all interconnected together with wires, and this helps distribute the power evenly around the layout. This procedure is done as a matter of good electrical practice on anything more than a small, basic layout.
If there is a power loss at a certain spot, or on the whole layout for some reason, the train would just run slower.
Larry