In this topical review we discuss the nature of the low-temperature phase in both
infinite-ranged and short-ranged spin glasses. We analyse the meaning of pure
states in spin glasses, and distinguish between physical, or 'observable', states and
(probably) unphysical, 'invisible' states. We review replica symmetry breaking
(RSB), and describe what it would mean in short-ranged spin glasses.
We introduce the notion of thermodynamic chaos, which leads to the
metastate construct. We apply these tools to short-ranged spin glasses, and
conclude that RSB, in any form, cannot describe the low-temperature
spin glass phase in any finite dimension. We then discuss the remaining
possible scenarios that could describe the low-temperature phase, and the
differences they exhibit in some of their physical properties—in particular, the
interfaces that separate them. We also present rigorous results on metastable
states and discuss their connection to thermodynamic states. Finally,
we discuss some of the differences between the statistical mechanics of
homogeneous systems and those with quenched disorder and frustration.