The small harbour itself is not a particularly good natural harbour and is protected from swells by a breakwater that was built with thousands of huge blocks of sandstone quarried out of the face of the mountain above. Simon's Town is now in effect a suburb of greater Cape Town. It is the terminus of a railway line that runs south from the central business district of Cape Town. In places the railway line hugs the steep eastern shore of False Bay quite spectacularly and in bad weather foam from some heavy swells will fly up and wet the trains.

Boulders Beach is located a few kilometres to the south of Simon's Town. Here small coves with white sandy beaches and calm shallow water are interspersed between boulders of Cape granite. There has been a colony of African penguins at Boulders Beach since 1985. There is no record of the birds having lived here prior to that date, so their decision to settle in an area already well-utilized by humans is remarkable. There are only three penguin populations on the mainland in southern Africa; the others are close to Hermanus at Stoney Point and Betty's Bay.