To reduce handoff overhead such as signaling overhead and latency, optimal handoff algorithm is needed for Mobile Multihop Relay (MMR) systems. This thesis proposes a novel handoff method that reduces intercell handoff but increases intracell handoff events, because intercell handoff has more overhead in view of signaling and gives more latency time to users rather compared to intracell handoff. Therefore, in the proposed scheme, intracell handoff is encouraged instead of intercell handoff using an adaptive hysteresis level. Intracell handoff performs based on lower handoff hysteresis than intercell handoff hysteresis. This thesis also presents hierarchical base station and relay station ID (BS/RS ID) concept and new TLV encoding of MOB_NBR-ADV messages based on the IEEE 802.16e standard as a method for acquiring a cell information. The MOB_NBR-ADV message is proposed in case that hierarchical ID concept could not be used in a system. This proposed algorithm can decrease handoff signaling overhead, latency, and unnecessary handoffs called ping-pong effect as well. Also, the proposed algorithm is backward compatible with mobile station (MS). Performance analysis and simulation results validate the efficiency of the proposed algorithm.