When I wrote the Doom 3 polysoup maker, one thing that surprised me was how much geometry was actually outside the worldspawn and had to be translated back into place. It just didn't make any sense to me.
While showering this morning, I figured it out, and all I could think was, "Carmack, you are a clever bastard..."
Doom 3 uses a combination portal/PVS engine to determine what to render. Lots of information gets precalculated, but all of it can be regenerated from the .map file.
Doom 3 seems to use a BSP to speed generation of the PVS. The more geometry that is located inside the worldspawn, the longer it takes to generate the BSP and the more PVS nodes that will end up being created. Once the PVS has been determined, the rest of the geometry seems to get shifted back and associated with the PVS nodes.
So it seems that for optimal map generation speed and PVS generation speed, the worldspawn should only contain the base geometry for the world, any static brushes that are going to be used to block vis, and the portals.
As for brushes that aren't going to move that are going to have non-moving lights strike them and cast shadows, I'm not sure if they should be in the worldspawn or not. That's going to require some additional testing to determine...but still, Carmack, you are a clever bastard.