Hysteresis causes difficulties in precisely controlling motion of flexible surgery robots and degrades the surgical performance. In order to reduce hysteresis, model-based feed-forward and feedback-based methods using endoscopic cameras have been suggested. However, model-based methods show limited performance when the sheath configuration is deformed. Although feedback-based methods maintain their performance regardless of the changing sheath configuration, these methods are limited in practical situations where the surgical instruments are obscured by surgical debris, such as blood and tissues. In this letter, a hysteresis compensation method using learning-based hybrid joint angle estimation (LBHJAE) is proposed to address both of these situations. This hybrid method combines image-based joint angle estimation (IBJAE) and kinematic-based joint angle estimation (KBJAE) using a Kalman filter. The proposed method can estimate an actual joint angle of a surgical instrument as well as reduce its hysteresis both in the face of partial obscuration and in different sheath configurations. We use a flexible surgery robot, K-FLEX, to evaluate our approach. The results indicate that the proposed method has effective performance in reducing hysteresis.