An algorithm using spatio-temporal thresholding for object detection with spatio-temporal distance metric in image sequences is proposed. The distance metric consists of the feature which uses the intensity and gradient at the same time in feature level instead of in decision level. In the model update process truncated variable adaptation rate is used, which can control adaptation rate up to its statistics, so it is able to maintain its statistics properly through the whole sequence. Some experimental results in various environments show that the averaged performance of the proposed algorithm is good.