Users of Social Networking Sites (SNS) often engage in social comparison, an experience that can elicit various affective responses. While previous studies have established important links between social comparison and emotion, the granularity and ecological validity of this work remains limited. To address this issue, we conducted a field study with 40 Instagram users. We unobtrusively recorded their Instagram usage patterns (e.g., time on site and the type of content consumed, such as feed, story, or message). Additionally, we used experience sampling methods and questionnaires to capture their affective experiences, social comparisons, and depression severity. Our results elucidate the interplay between these emotions, traits, and tendencies during Instagram use. Notably, we find that while passive or prolonged Instagram use affects users’ emotions, the magnitude of this effect depends on the frequency of users’ social comparisons. These results can inform the design of systems to improve mental well-being on SNS.