In this paper, we present a new real-time visual servoing unit for laparoscopic surgery. This unit can automatically control a laparoscope manipulator through visual tracking of the laparoscopic surgical tool. For the tracking, we present a two-stage adaptive CONDENSATION (conditional density propagation) algorithm to detect the accurate position of the surgical tool tip from a surgical image sequence in real-time. This algorithm can be adaptable to abrupt changes of illumination. The experimental results show that the proposed visual tracking algorithm is highly robust.