Development of a Manual Handling Task Recognition and Holistic Fatigue Estimation System

Abstract

The overarching goal of this dissertation was to contribute to the body of research on mitigating work-related musculoskeletal disorders (WMSDs) and occupational accident risk by identifying opportunities to leverage machine learning (ML) techniques, then develop an effective intervention strategy. A scoping review on the role of ML in the primary prevention of WMSD was conducted and identified fatigue modelling during physical labour as a notable and novel intervention strategy. Some shortcomings of existing efforts include the dichotomization of fatigue, the over-reliance on self-reported fatigue as a ground truth, and the inability to automatically adapt estimations to various tasks. Thus, this doctoral research aimed to build off these shortcomings and develop the primary components of an automatic system to adapt to the performance of various manual handling tasks using minimal wearable sensors, then estimate holistic fatigue as a continuous variable. To achieve this, four research objectives were formulated: 1) Perform a scoping review to document how ML techniques have been used to improve efforts to mitigate WMSD risk and identify areas for further research and development. 2) Develop human activity recognition (HAR) models that utilized either raw (i.e., linear acceleration, angular velocity, magnetometer) or fused (i.e., orientation and position) IMU data to classify manual handling tasks. 3) Perform a sensitivity analysis of the quantity and placement of IMUs for classification of the manual handling tasks. 4) Develop task-specific ML models to estimate continuous fatigue levels during the performance of the six manual handling tasks, where fatigue was quantified holistically using a composite index. The scoping review returned 130 English studies published since 1990 regarding the use of ML techniques and were classified as contributions to the six steps of WMSD prevention (van der Beek et al., 2017). ML techniques were found to contribute most commonly to the development of interventions (48 studies; step 4), where a notable intervention approach was fatigue modelling during physical work. For studies 2 – 4, a simulated, fatiguing manual handling protocol was conducted, where participants performed 5 min manual handling sets of pulling, lifting, carrying, and pushing a crate, followed by walking and standing. Between each set, they performed a fatigue assessment, then continued to perform manual handling sets and fatigue assessment without breaks until a stopping criterion was reached. In the second study, HAR models were trained to classify the manual handling tasks with strong performances (accuracy ≥ 94.39%). However, HAR models performed significantly (yet marginally) better when using raw versus fused IMU data (p < 0.01). The third study developed 36 HAR models, with 18 sensor combinations and two ML model architectures to provide a sensitivity analysis. A greater number of IMUs lead to greater classification performances, while using one IMU achieved similar classification performance compared to using six (i.e., T8 and 6 IMU accuracy = 93.17% and 94.26%, respectively). Marginally better performances were observed with the deeper neural network (four convolutional and two long short-term memory layers) compared to the shallower neural network (three dense layers) until 3 or more IMUs were involved, after which the shallower neural network was equal or better. In the fourth study, a holistic fatigue composite index (FCI) comprised of 6 fatigue variables was computed for all participants and reflected an increase in fatigue throughout the manual handling protocol. Then, task-specific ML regression models were trained to estimate the FCI using kinematic data from 1 or 3 IMUs (combinations informed by study 3), with and without heart rate. On average, using only the T8 IMU and heart rate data allowed for estimation of the FCI with small errors (RMSE = 0.06, MAE = 0.05) and moderate correlation (r = 0.59) across all tasks and participants. The contributions of this dissertation include a thorough scoping review, knowledge surrounding the development of HAR models, and a novel approach to estimate task-specific holistic fatigue. This work represents the primary components of an unobtrusive and inexpensive system that can inform workers and employers of near real-time fatigue progression during the performance of manual handling work tasks, to ultimately mitigate WMSD and occupational accident risk.