Development and Validation of Novel Methods to Track Three-Dimensional Spine Kinematics During Dynamic Trunk Movements Using a Time-of-Flight RGB-D Camera - Ramos Junior, Wantuir Carlos (Université d'Ottawa / University of Ottawa, 2021)
Low back pain (LBP) is a common and costly musculoskeletal disorder. Clinicians and researchers frequently assess dysfunction in LBP by observing patient’s movement quality during pre-determined tasks using technically challenging and expensive motion capture devices. However, the cost and expertise needed to use these devices are direct barriers to their widespread adoption in clinical settings where direct benefits could be realized by LBP patients. RGB-D cameras are a potential inexpensive and less time-intensive motion capture alternative, which have been validated for analyses of gait, postural control, ergonomics, and human anatomy. However, as manufactured, RGB-D cameras are not ready for specific kinematic analyses, such as lumbar spine motion, as the native artificial intelligence (AI) models do not adequately detect the number and location of trunk joints to accurately track three-dimensional kinematics. In this thesis a proof of concept framework to adapt RGB-D cameras to accurately measure lumbar spine kinematics is presented, which is separated into two research studies: 1) validation of an RGB-D depth-based algorithm against an optoelectronic motion capture system of reference for measuring spine motion, and 2) the development of a markerless method of measuring lumbar spine kinematics. In study one, 12 healthy young adults (6M, 6F) performed 35 cycles of repetitive flexion-extension with infrared reflective marker clusters placed over their T10-T12 spinous processes and sacrum, while motion capture data were recorded simultaneously by one RGB-D camera and a 10-camera optoelectronic motion capture system. Lumbar spine joint angle range of motion (ROM) were extracted and compared between systems. Root mean squared error (RMSE) values were very low across all movement axes (RMSE ≤ 2.05° ± 0.97°), and intraclass correlations (ICCs) were considered excellent across all axes (ICC2,1 ≥ 0.849), while Bland-Altman plots revealed that, on average, the RGB-D camera slightly underestimated flexion-extension angles (≈ -1.88º) and slightly overestimated lateral bending and axial twisting angles (≈ 0.58º). In study two, a single RGB-D camera was used to capture infrared, depth, and colour image data of 15 participants performing two batteries of 10 cycles of repetitive trunk flexion-extension under two conditions: marked (i.e. hand drawn markers on key anatomical locations on the back) and unmarked. The collected data were used to create a custom four module convolutional neural network (CNN; SpineNet) to segment the human back into upper back, lower back, and spine regions and to subsequently extract lumbar spine kinematics. SpineNet was trained and tested on ten marked participants in a train:test ratio of 80:20. Images of five additional participants without markers were used to evaluate SpineNet’s generalizability. Quantitative image segmentation analysis on marked data had good similarity and accuracy between their prediction and ground truth across all individual modules (mPAFG ≥ 0.8855; mJBKG ≥ 0.8391; mJFG ≥ 0.7884; fwJBKG ≥ 0.9672; fwJFG ≥ 0.8087) when the background class was ignored. Qualitative image segmentation analysis on unmarked data showed that Colourized and Surface Normal modules presented a more uniform and robust class morphology throughout frames than infrared (IR) and Fusion modules. All modules extracted kinematics similarly and were compared with their ground truth labels, showing low error levels across all movement axes (RMSE ≤ 3.66º), and good agreement on the flexion-extension and axial twist axes (ICC2,1 > 0.907; CI 95% [0.640, 0.990]); however kinematic extraction in the lateral bend axis was poor (-0.212 < ICC2,1 <0.630; CI 95% [-6,990, 0.910]). Kinematic analysis on unmarked participants, done by inspecting average ensemble curves, showed that flexion-extension angles exhibited movement profiles (i.e. shape, timing, and peaks) that are comparable with previous research where similar data were collected. In conclusion, this thesis provides a foundational proof of concept framework for using a single RGB-D camera for collecting and analyzing lumbar spine kinematics with or without markers. The raw depth data were considered adequate for extracting lumbar spine kinematics from markers when compared to an optical motion capture system, and the application of an image segmentation AI model followed by the proposed method of kinematic extraction achieved adequate results for measuring lumbar spine kinematics on unmarked participants.
Keywords: Computer Vision; Artificial Intelligence; RGB-D Camera; Kinect; Depth Camera; Low Back Pain; Movement Quality; Biomechanics; Motion Capture; Machine Learning; CNN; Convolutional Neural Networks; U-Net; Fusion; Image Segmentation; Deep Learning; Spine Kinematics; Image Processing