Researchers developing an app to prevent workplace injuriesMarie Donlon | August 21, 2019
A team of researchers from the University of Washington are using machine learning to devise a new system for monitoring factory and warehouse workers for risky behaviors that have the potential to result in work-related injury.
Largely thought to be among those who run the highest risk of work-related injuries, factory and warehouse workers often perform repetitive tasks that can result in strain on the body over time. As such, the team from the University of Washington has developed a machine learning algorithm that is capable of detecting high-risk movements that might result in injury to the worker.
The algorithm breaks down activities likely to be conducted in such an atmosphere into individual motions, assigning each activity risk scores. To do this, the team developed a dataset of 20 three-minute videos of people performing a series of 17 different activities commonly performed in a factory or warehouse setting. Among the assigned activities, participants were tasked with picking up boxes from racks and placing them on tables. Using a Microsoft Kinect camera, the team captured footage for the dataset, recording 3D videos that enabled the team to map out the impact on the participants’ joints as they performed each of the 17 tasks. Using that footage, the algorithm computed the risk scores for each frame of video. It then learned to identify when a task would begin and end, making it possible to calculate a risk score for the complete action.
Eventually, the algorithm labeled three of the 17 actions in the dataset as risky. Those behaviors included picking up a box from a high shelf, placing a box on a high shelf or placing a rod on a high shelf.
The team is currently attempting to develop an app based on the algorithm that will enable factory and warehouse workers and supervisors to monitor the real-time behaviors that put them at risk of injury.
"Right now workers can do a self-assessment where they fill out their daily tasks on a table to estimate how risky their activities are," said senior author Ashis Banerjee, an assistant professor in both the industrial and systems engineering and mechanical engineering departments at the University of Washington. "But that's time consuming, and it's hard for people to see how it's directly benefiting them. Now we have made this whole process fully automated. Our plan is to put it in a smartphone app so that workers can even monitor themselves and get immediate feedback."
The team envisions that the app will offer warnings about moderately risky behaviors and issue actual alerts for behaviors thought to be high risk. The team also wants to eventually equip warehouse and factory robots with the algorithm to help monitor and keep human workers healthy.
"Factories and warehouses have used automation for several decades. Now that people are starting to work in settings where robots are used, we have a unique opportunity to split up the work so that the robots are doing the risky jobs," Banerjee said. "Robots and humans could have an active collaboration, where a robot can say, 'I see that you are picking up these heavy objects from the top shelf and I think you may be doing that a lot of times. Let me help you.'"
The research is detailed in the IEEE Robotics and Automation Letters.