Please refresh us on your project.
Our project aims to develop technology for optimizing physical therapy and rehabilitation services. Specifically, we are coupling cameras and wearable sensors with machine learning to automate exercise assessments.
There are a number of ways in which such technology-enabled assessment could prove useful. It can help providers to ensure adherence and track improvement over time, by providing information about how patients perform prescribed exercises at home. This can facilitate remote treatment or tele-rehabilitation, the importance of which has become increasingly clear during the COVID-19 pandemic. It can also furnish information that allows providers to use their time effectively, by identifying and focusing on patients that are having difficulty with their treatment program. Eventually, it could ultimately provide patients with simple feedback about exercise performance, without a clinician present. Finally, it can generate data that is useful for monitoring outcomes and demonstrating the value of physical therapy.
What was most exciting or unexpected that you learned during the project?
A better understanding of the clinicians’ needs and the challenges of developing this sort of technology motivated our research group to shift priorities. We came into this project with a strong interest in developing technology for automated exercise coaches — user-centered devices that can observe and provide real-time feedback about the quality of human movements. This is a challenging problem, as it requires sensors and algorithms that can detect subtle differences in movement, and that are robust to highly dynamic movement contexts. As our research progressed, however, both the input from clinicians and the data prompted us to focus more on generating automated summaries of patient performance that can enable providers to optimize how they use their time. This technical problem can be readily addressed, and we believe it has potential for meaningful impact in the short term.
Did you encounter any roadblocks that you did not anticipate?
Without question, the process of collecting data from patients was the most challenging part of this project. It was during data collection that we experienced the most unanticipated setbacks. Had we not had such effective clinical partners, data collection issues might very well have de-railed the project entirely.
It was also significant that patients committed errors in exercise motions at a lower rate than we had anticipated, and that exercise errors were not well distributed throughout the population. Certain individuals committed a lot of errors, while others committed none at all. This contrasts with prior investigations of technology-enabled exercise assessment, which often collect data from healthy subjects. In studies with fatigue-induced or conditioned errors, both the distribution and frequency of errors among participants is more conducive to training generalizable machine learning algorithms. This is a unique challenge for clinical studies.
What are the biggest takeaways from your project?
From an optimistic perspective, this project reinforces the idea that technology-enabled exercise assessment is a viable technology for near-term clinical applications. We’re still a long way from AI-driven personal exercise coaches, but we’ve made progress, and there are real applications of this technology now.
Any final thoughts?
As I mentioned previously, Prof. Adam Popchak and Keelan Enseki have been outstanding clinical partners. This project wouldn’t have happened without them, along with the trainees and volunteers that provided us with data. Professor Jessica Hodgins got this project started and has made critical contributions throughout. In addition, Professors Eni Halilaj and Emily Kim have contributed a lot during the final analysis stage of this project. Of course, this would not have happened without the Center for Machine Learning and Health at CMU, and the Pittsburgh Health Data Alliance, more generally.