A subgoal is a unit that groups a set of steps by their functions in a problem-solving procedure, such as cooking, how-to's and programming. Studies showed that learning hierarchical subgoal structures of worked examples can aid transfer in learning. To support subgoal learning at scale, we need to generate subgoal hierarchies that consist of both the goal structures and labels. While prior work [3, 8] has focused on using learnersourcing to generate high quality subgoal labels at scale, generation of hierarchical subgoal structures had little attention and has been done manually by domain experts. Generation of hierarchical subgoal structures is especially challenging for both AIs and crowdworkers because it requires comprehensive understanding of the entire problem-solving procedure. In order to enable subgoal hierarchy generation at scale without expert interventions, we propose a novel learnersourcing workflow that combines learners' local understanding of subgoal structures into multi-granular subgoal hierarchies.