Researchers at the University of Washington have developed a wearable camera system coupled with artificial intelligence (AI) that detects potential errors in medication delivery.

Source: University of Washington

The system’s developers believe that the system could become an important safeguard against errors made in operating rooms, intensive-care units and emergency-medicine settings.

To prevent errors — such as syringe and vial-swaps wherein the wrong vial is selected, or a syringe is mislabeled or when the drug is labeled correctly but administered incorrectly — the team is attempting to build a deep-learning model paired with a GoPro camera that recognizes the contents of cylindrical vials and syringes and issues a warning before the medication is injected into the patient.

To accomplish this, the researchers collected 4K video of 418 drug draws by 13 anesthesiology providers in operating rooms where setups and lighting varied. The video was used to capture clinicians handling the vials and syringes of select medications. The video clips were logged and the contents of the syringes and vials denoted to train the model to recognize the contents and containers.

Rather than directly reading the wording on the vials, the video system scans for other visual cues like vial and syringe size and shape, vial cap color and label print size, for example.

“It was particularly challenging, because the person in the OR is holding a syringe and a vial, and you don’t see either of those objects completely. Some letters [on the syringe and vial] are covered by the hands. And the hands are moving fast. They are doing the job. They aren’t posing for the camera,” the researchers added.

Additionally, the computational model was trained to focus on medications in the foreground of the frame while ignoring vials and syringes in the background — for instance, focusing on the syringe in the provider’s hand and not the syringe lying on the table.

The team’s findings are detailed in the article, “Detecting clinical medication errors with AI enabled wearable cameras,” which appears in the journal npj Digital Medicine.