A haptic mouse interface system includes a mouse, a first force feedback unit for providing a first force feedback in a first direction of a virtual object to the user's hand and arm, a second force feedback unit for providing a second force feedback in second and third directions to the user's fingers, and a tactile feedback unit for providing tactile sensations to the user's fingers. The first force feedback unit has first and second encoders to receive signals relating to a first direction of the virtual object, first and second motors driven by the first and second encoders, and a linkage, which is connected between shafts of the first and second motors and the mouse. The second force feedback unit is provided in the mouse, and has a third encoder to receive a signal relating to the second and third directions of the virtual object, a third motor driven by the third encoder and having a shaft, a pair of finger pads provided at both sides of the mouse and linearly moved by the third motor. The tactile feedback unit is provided in the mouse, and has actuators attached to the mouse, and a plurality of pins coupled to the actuators to come into contact with the user's fingers, to provide pressure or vibration exhibiting the surface properties of the virtual object to the user's fingers.