This paper proposes a methodology to select a site for a heliostat field to further improve its optical efficiency. Site selection implies to select a location with advantageous geographical features for optical efficiency improvement. Therefore, analysis on the site slope effect and guideline to select the most optically efficient site type are the main concerns of this study. The paper consists of two parts: (1) analysis of the site slope effect and determination of the most efficient site type at each latitude, and (2) optimization of heliostat layout on the optimal site for further improvement. Latitude is treated as a control variable since it changes all the solar characteristics, so the site effects at different latitudes are examined independently in this study. Surrogate modeling and genetic algorithm are utilized for efficient and accurate optimization of the heliostat field layout. This study reveals that the optimal site type varies according to the latitude, and utilization of the most efficient heliostat field site improves the optical efficiency by up to 1.65% point that can hardly be achieved by field layout optimization only. Furthermore, this study suggests the applicability of central receiver system (CRS) at high latitude with appropriate site selection and layout optimization.