The shift to virtual meetings, online classes, and remote work has established a new norm, leading to a surge in the use of virtual communication platforms such as Zoom and Microsoft Teams. This shift has increased the demand for high-quality headsets and speakerphones, emphasizing the need for clear, superior audio quality. The process of calibrating material properties typically relies on repetitive simulations guided by experts' intuition, presenting challenges in establishing new Finite Element Models (FEMs) of loudspeakers, as it requires the repeated identification of material property values. We present a systematic framework for calibrating the mechanical material properties of loudspeaker drivers, a crucial prerequisite for developing accurate FEMs of loudspeakers. Specifically, we propose a statistically-driven approach to replace the conventional manual calibration process, which typically relies on multiple simulations guided by expert intuition. Efficient Global Optimization (EGO) is applied to address the expensive optimization problems of loudspeaker simulation. To tackle the curse of dimensionality, the objective function is decomposed into several functions based on effective parameter groups using Global Sensitivity Analysis (GSA) results. The parameters of the FEM are then calibrated to the reference data from the Lumped Parameter Model (LPM) using the decomposed-reduced objective function, providing the calibrated parameters for the loudspeaker simulation. By implementing this novel approach, even individuals without prior knowledge or experience in loudspeaker material properties can effectively and reliably obtain the necessary data for finite element modeling.