Mining sequential knowledge for interaction design

Abstract

From blogs to how-to videos to social media posts, it has become increasingly easy to share knowledge online. As a result, even for a single task seemingly as simple as baking chocolate chip cookies, we have access to hundreds of thousands to millions of tutorials in less than a second through a simple web search. Each tutorial shared online is not only a set of task instructions, but also a display of the authors' unique set of skills, constraints, and strategies. Consequently, the resulting diversity and scale of available online tutorials present users with both opportunities and challenges. Specifically, it remains difficult 1) to compare and analyze which tutorial to follow and invest the time to learn, and 2) to apply the picked tutorial to the current user's expertise, available tools, and other workflow contexts. However, the typical form factor for information retrieval systems like search engines and recommendation systems force users to go back and forth between the search results and individual tutorials to compare and analyze how they are different or similar. Also, users have a difficult time conceptualizing how different methods agree and disagree with each other and understand what makes one tutorial work and the other not work. However, whether people accomplish to find applicable tutorials depends on their ability to access not ``the single right'' tutorial, but on their ability to access and explore the ``right set'' of tutorials. This thesis presents systems that support users in forming hypotheses, comparing, and analyzing tutorials in different settings. I demonstrate techniques for extracting the nonlinear compositions of semantic substrates in large scale tutorials and demonstrations, such as task-specific domain knowledge, constraints, and users' strategies. Then I present abstract representations reconstructed with these semantic substrates and novel interface designs that strengthen people's ability to learn from tutorials.