Even though many efforts have been devoted to effective strategies to build accurate surrogate models, surrogate model uncertainty is inevitable due to a limited number of available simulation samples. Therefore, the surrogate model uncertainty, one of the epistemic uncertainties in reliability-based design optimization (RBDO), has to be considered during the design process to prevent unexpected failure of a system that stems from an inaccurate surrogate model. However, there have been limited attempts to obtain a reliable optimum taking into account the surrogate model uncertainty due to its complexity and computational burden. Thus, this paper proposes a confidence-based design optimization (CBDO) under surrogate model uncertainty to find a conservative optimum despite an insufficient number of simulation samples. To compensate the surrogate model uncertainty in reliability analysis, the confidence of reliability is brought to describe the uncertainty of reliability. The proposed method employs the Gaussian process modeling to explicitly quantify the uncertainty of a surrogate model. Thus, metamodel-based importance sampling and expansion optimal linear estimation are exploited to reduce the computational burden on confidence estimation. In addition, stochastic sensitivity analysis of the confidence is developed for CBDO, which is formulated to find a conservative optimum than an RBDO optimum at a specific confidence level. Numerical examples using mathematical functions and finite element analysis show that the proposed confidence analysis and CBDO can prevent overestimation of reliability caused by an inaccurate surrogate model.