Although oxide glasses have many unique properties, their range of applications remains limited by their brittleness. By mimicking the microstructure of composite materials, the presence of controlled nanoscale phase separation in glass could overcome this limitation. However, the nature of the toughening mechanism induced by such nanostructuring remains poorly understood. Here, based on peridynamic simulations, we investigate the effect of nanoscale phase separation on the crack propagation mechanism. We show that phase separation can significantly increase glass’s toughness (with up to a 90% increase in the fracture energy for the range of conditions investigated herein). The extent of toughening is found to arise from a balance between the overall cohesion of the phase-separated glass and the propensity for crack deflection. This suggests that controlled nanoscale phase separation is a promising route toward the development of tough, yet optically transparent glasses.