Purpose: Commercial and residential buildings are primary building types covering the majority of our built environment. Though, energy consumption pattern identification of those building types based on real-world data analysis has not been aggressively explored. Identification of primary factors influencing energy consumption patterns of commercial and residential buildings is the goal of this study. Method: CBECS and RECS data sets in the United States were used for commercial and residential building-energy-consumption pattern analysis. Multi-linear and seven machine learning algorithms are utilized to analyze building characteristics and end-use energy consumption patterns, e.g., cooling, heating, and water-heating. The SHAP value is utilized to describe influential factors in each energy consumption analysis model. Result: Ensemble algorithm yielded the lowest error rates compared with other algorithms. The calculated error rates also showed a lower level than the precedent studies performed on the CBECS and the RECS. Commercial building's cooling and heating energy consumption is more likely influenced by occupancy, while residential building's energy consumption is affected by equipment and climatic conditions. In the meantime, water-heating energy consumption shows noticeable dependency over the occupancy and climatic conditions for commercial and residential buildings. As a critical passive design element in the building, window has a more significant influence than overall insulation or roof finishing in identified residential buildings' energy consumption patterns.