The theoretically-derived Makishima-Mackenzie (MM) model expresses Young’s modulus of glass in terms of two determining factors, namely, the inter-atomic bonding strength (dissociation energy) and the ways in which atoms are packed (atomic packing fraction). This simple model offers a clear physical picture to understand the compositional dependence of the stiffness of glasses, but it generally underestimates the actual Young’s modulus for many glasses, especially in the high-value range. In this study, we argue that the inadequacy of the MM model mainly arises from its definition of the atomic packing fraction—which is defined as the ratio between the volumes of the atoms and the actual macroscopic volume of the glass. Such a definition results in a considerable amount of spacing within the basic building units being counted as free volume, which eventually leads to low packing fractions and, consequently, to low Young’s modulus values. Here, we propose a more suitable packing metric, the Rigid Unit Packing Fraction (RUPF), which defines the basic building units as fully-filled, whole polyhedra made of “touching” oxygen atoms with no interstitial free volume. Young’s moduli of 155 oxide glasses predicted from our revised MM model show a significantly improved level of agreement with respect to experimental data as compared to the original MM model. This study not only improves the ability of the physics-based MM model to yield accurate predictions of Young’s modulus but also supports the relevance of rigid -unit theory, which could be applied as a basis to decipher other property-structure correlations.