In this paper, a method of shape design sensitivity analysis of point stress constraints on the boundary of linear elastic systems is presented. Constraints on point stress at the boundary are first transformed to constraints on an unsmoothed stress functional defined on the domains of finite elements that include the point, using the local smoothing met for stress, which is widely adopted for post-processing analysis results in the finite element method. The material derivative idea and an adjoint variable technique are then employed for shape design sensitivity analysis of the constraints. Validity of the method is tested through a numerical example. It is shown that the method is stable, with accurate shape design sensitivity results, and readily applicable to practical structural shape optimization for linear elastic systems.