Understanding mobile document capture and correcting orientation errors

Abstract

Smartphone cameras are increasingly used for document capture in daily life. To understand user behaviors, we performed two studies: (1) an online survey (n=106) to understand general smartphone camera usage behaviors related to information capture, as well as participants' experiences of orientation errors, and (2) a controlled lab study (n=16) to understand detailed document capture behaviors and to identify patterns in orientation errors. According to our online survey, 79.30% of the respondents reported experiencing orientation errors during document capture. In addition, our lab study showed that more than 90% of landscape capture tasks result in incorrect orientation. To solve this problem, we systematically analyzed the user behavior during document capture (e.g., video sequences and photographs taken or hand grip used) and propose a novel solution called ScanShot, which detects document capture time to help users correct orientation errors. ScanShot tracks the direction of gravity during document capture and monitors the users rotational or tilting movements of to update changes in orientation automatically. Our results confirm that document capture with 93.44% accuracy; in addition, our orientation update mechanism can reduce orientation errors by 92.85% using a gyroscope (for rotation) and 81.60% using an accelerometer (for micro-tilts).