People's collectively shared beliefs can have significant social implications, including on democratic processes and policies. Unfortunately, as people interact with peers to form and update their beliefs, various cognitive and social biases can hinder their collective wisdom. In this paper, we probe whether and how the psychological construct of intellectual humility can modulate collective wisdom in a networked interaction setting. Through agent-based modeling and data-calibrated simulations, we provide a proof of concept demonstrating that intellectual humility can foster more accurate estimations while mitigating polarization in social networks. We investigate the mechanisms behind the performance improvements and confirm robustness across task settings and network structures. Our work can guide intervention designs to capitalize on the promises of intellectual humility in boosting collective wisdom in social networks.