Proposing, validating, and expanding IQ assessment framework for presentation slides

Abstract

Computerized presentation slides have become essential for many occasions such as business meetings, classroom discussions, and public events. Given the tremendous increase in online resources and materials, locating high-quality slides relevant to a given task is a daunting challenge, particularly when users look for superior quality slides out of the massive amount of slides. This study proposes a new, comprehensive framework for Information Quality (IQ) developed specifically for computerized presentation slides, explores the possibility of automatically detecting the IQ of slides, and expands the IQ framework by including difficulty assessment criteria and technique. To determine slide-specific IQ criteria as well as their relative importance, we conducted a user study involving 60 participants from two universities and utilizing extensive coding analysis. The coding analysis summarized users comments and provided 216 IQ criteria for presentation slides. The suggested IQ criteria was assigned into proper IQ dimensions by card sorting method. Finally, we built a IQ framework comprising 216 criteria across 13 dimensions. Further, we subsequently conducted a series of multiple experiments to examine the validity of the IQ framework including features developed on the basis of the selected criteria from the user study. Based on statistical support and feasibility, we devised 65 features to evaluate the IQ of slides. By performing Learning to Rank approach with the features, we confirmed that the experimental results support the validity of the proposed framework. The results also showed that the proposed feature set from the IQ criteria based on the user study effectively improves the retrieval performance(27.3%) over BM25 algorithm and Google slides search results. According to analysis result about the level of dimension, representational and informativeness dimension were effective to the automatic assessment. In addition, we expanded the framework by including difficulty assessment, that lead to a deep understanding of IQ of presentation slides. We proposed a difficulty model comprising content difficulty and presentation difficulty. With the model, we suggested 59 features across 5 difficulty dimension and performs a classification task to distinguish easy and hard slides in terms of content and representation. The results showed that the proposed difficulty model outperforms 6.2% in content difficulty and 20.5% in representation difficulty and also confirmed that the feature set have an crucial impact for classification performance, admitting the effectiveness of our comprehensive IQ framework. According to the results, readability is important for evaluation content difficulty and several dimensions such as structural completeness and formatting style were effective. The study showed the proposed comprehensive IQ framework is promising to evaluate quality and difficulty for educational presentation slides. The IQ framework is flexible and extensible for other types of documents such as textbook. This study had an impact on the related academic field for IQ assessment by providing a model, criteria, evaluation sets, and so on.