A GPS is usually installed at the center of a vehicle because with this configuration its data become the position of the vehicle without complex calibration and transformation. In this paper, however, we verify that the off-center arrangement of the GPS improves the position and orientation accuracy theoretically and experimentally. The extended Kalman filter (EKF) formulation and its observability and uncertainty analysis present that the off-centered GPS is better observable and less uncertain. Moreover, our experiments with synthetic data and real sensor dataset confirm the improvement of localization accuracy. The synthetic data generated in various situations reveal the important properties of localization with the off-centered GPS. The public outdoor dataset with real GPS data also supports the advantage of the off-centered GPS.