Bailey Russ ’18 and Frankie Capraro ’18 are laying the foundation for an Xbox program that could change how physical therapy patients recover at home.
Although workouts available on the Xbox game console award points for completed exercises, none that Russ or Capraro found actually required players to complete routines or provided coaching. The Xbox’s Kinect camera can detect players’ motions, but without accountability, the games’ energetic commands ultimately amount to an exercise video—not necessarily a possible at-home therapy tool.
Together, Russ and Capraro have created the first phase of a virtual simulation that could fill that gap.
What if a game could “see” that a patient performed only four bicep curls, when their regiment asks for eight, and encourage them to finish? Russ, a kinesiology major with an emphasis in physical therapy, can envision how such an artificial intelligence program would be useful.
“For example: elderly people, sometimes they have physical therapy and it’s not always easy for them to travel,” Russ said. “So maybe we could put this system inside their home and this avatar can monitor their exercises at home and send the information back to the physical therapist, and see if they are doing everything that they are supposed to do.”
To build the virtual physical therapy simulation, Russ found a partner in Capraro, a self-designed film and physics major through the Whittier Scholars Program who could lend her coding and animation skills to the project. With guidance from Assistant Professor of Kinesiology Joshua Haworth, they developed a simulation, connected the program to both a Kinect and a separate motion-capture system in the Yao Yuan Sze Foundation Movement Laboratory, and collected study participants’ movement data.
In December, they plan to take that data to the mathematics department and approach them about taking the project to the next level: giving their virtual trainer the artificial intelligence it needs to become effective. Instead of running on a script, the program can be aware of patients’ movements and coach and encourage them, Haworth said.
“It’s amazing and it’s going to be revolutionary the more and more we develop it,” Capraro said.
The project has been an exciting opportunity for Russ to combine her kinesiology studies with coding. As she sets her sights on graduate school, she hopes to continue investigating the benefits of combining physical therapy and the virtual world.
The project is also a great asset for Capraro, who’s gained internship-like experience with crucial visual effects skills—including modeling and lighting—right on campus. Eventually, she hopes to work at a visual effects production house to create the dazzling worlds within movies or video games.
