Source: Ken Turner CollectionAn assistant professor of computer science at Virginia Tech has created a software platform that can identify Civil War soldiers using facial recognition technology.

Kurt Luther, in collaboration with historians, the Virginia Center for Civil War Studies and Military Images magazine, developed the free platform called Photo Sleuth, which relies on crowdsourced data to enable algorithms to identify the faces of Civil War soldiers that appear in photos.

Originally launched in August 2018 with a reference database of 15,000 identified Civil War soldier photos and profiles that contain detailed military history records, the platform lets users upload pictures of Civil War soldiers from personal collections or found in books and museums. Once uploaded, the user can assign tags to the image with keywords associated with photos including data such as format, inscription and details like shoulder straps, chevrons, coat color, and hat or collar insignias.

The tags are then linked to certain search filters. For instance, an image of a soldier tagged with the phrase “hunting horn” hat insignia might return images of soldiers who had served in the infantry. The software would thus hide results from cavalry or artillery. The site then uses facial recognition technology to rule out faces that don’t match, while sorting through similar ones. The combination of tagging and facial recognition technology serve to narrow down the possibilities.

The final step involves a comparison tool that enables users to view the matched images with pan and zoom controls in greater detail. If the user determines that the picture is a match, the user can tag the image to one of the previously identified photos in the database and their accompanying profile.

"Typically, crowdsourced research such as this is challenging for novices if users don't have specific knowledge of the subject area," said Luther. "The step-by-step process of tagging visual clues and applying search filters linked to military service records makes this detective work more accessible, even for those that may not have a deeper knowledge of Civil War military history."

Luther will present his research about the Photo Sleuth platform on March 19 at the Association for Computing Machinery's Intelligent User Interfaces conference in Los Angeles. He will also demonstrate the platform at the grand opening of the expanded American Civil War Museum, in Richmond, Virginia, on May 4.