Understanding the composition dependence of the hardness in materials is of primary importance for infrastructures and handled devices. Stimulated by the need for stronger protective screens, topological constraint theory has recently been used to predict the hardness in glasses. Herein, we report that the concept of rigidity transition can be extended to a broader range of materials than just glass. We show that hardness depends linearly on the number of angular constraints, which, compared to radial interactions, constitute the weaker ones acting between the atoms. This leads to a predictive model for hardness, generally applicable to any crystalline or glassy material.