Evaluations of a wearable inertial sensors based fall risk assessment system for the elderly

Abstract

The world is facing a major challenge on the population aging and the substantial health problems associated with the occurrence of fall. The fall can lead to serious physical and psychological consequences and even death. In order to counter this problem effectively, a proactive wearable inertial sensor based fall risk assessment system (FRAS) was recently developed. The aim of this study was to evaluate the newly developed FRAS system for assessing the elderly fall risk in terms of its functionality, accuracy, and usability, which is an important and necessary step before deployment of the newly developed system into future applications. Functionality evaluation includes the whole system proper working, accuracy check of the FRAS raw sensor data with the golden reference (Xsens 3D Motion Tracking System), and the quantification of the error (absolute and relative) involved in the measures of the testing protocols. For checking the model accuracy in the system, the confusion matrix is used for the accuracy quantification by comparing the FRAS prediction results with a history of fall. System diagnosis is evaluated with the help of conventional direct methods, where the results from the profile graph and decision tree were compared. System usability is assessed with system usability scale and mobile application rating scale questionnaires. In addition, system acceptability is also evaluated with the help of open-ended questions based on the technology acceptance model and unified theory of acceptance and use of technology.

Two-stage experiments were conducted for this evaluation, first on the 20 younger adults (age: 20-30 years) and the second on 50 older adults (age: 65 years or above). For FRAS sensors under static conditions, the highest relative error for acceleration was in the anteroposterior direction ($0.158 \pm 0.11$) and for the angular velocity in the mediolateral direction ($1.261 \pm 0.28$). This high error is due to the noise involved in the sensor data when there is no movement in the body. However, under the dynamic conditions, this relative error is reduced significantly for both acceleration in the anteroposterior direction ($0.027 \pm 0.19$) and angular velocity in the mediolateral direction ($0.118 \pm 0.18$). There is no statistically significant difference between the FRAS testing protocols output measures and the measures extracted from Xsens. In addition, the relative error involved in all the measures is within 10%. The FRAS prediction accuracy based on the majority voting (support vector machine, random forest, decision tree) for the external cohort data is 66% with 100% sensitivity and 43.33% specificity. However, the overall accuracy of the models based on 10-fold cross validation for the mixed data is 90% with 93% sensitivity and 86% specificity. The FRAS diagnosis accuracy is $86 \pm 21%$. The system usability score from SUS was $63.5 \pm 18.1$ (out of 100). This average SUS score lies at the high marginal end on the scale, which is close to the acceptable range. According to SUS adjective rating, this score lies in the good region, which is above ok. The intended elderly users were agree to use ($4.21 \pm 0.61$ out of 5) the system based on the technology acceptance evaluation.

Through this study, the accuracy of the FRAS in terms of basic sensor data, derived measures, model prediction, and technology acceptance is evaluated to check the potential of this newly developed system. The results will be used to improve the system functionality, and to add the well-trained, suitable machine learning models in the FRAS to enhance the accuracy of fall risk assessment. Based on the findings, the usability of the system should be improved further. The practical applications of the improved FRAS could not only improve the quality of life of the elderly but will also reduce the healthcare cost.