Building identification refers to the recognition of the identity of a building when the building is sighted. With proper identification, additional useful information can be gained, such as the geographical information of the building and facility information within the building based on the recognized identity. However, existing studies related to building identification require additional information such as building images in advance, or have constraints on the types of possible input images. Therefore, we propose an approach that undertakes building identification using a building boundary map and smartphone sensors. Our approach measures the position of the user and the orientation of the user's view using sensors embedded in a smartphone. We find the building sighted by the user by reducing the area of buildings that can exist in the user's orientation in a step-by-step manner. During the validation of our approach, it identified the buildings with accuracy of up to 83.3% despite the inaccuracy of the building boundary map used and the error inherent in the user's position and orientation based on the smartphone.