Sandia National Laboratories’ Pattern Analytics to Support High-Performance Exploitation and Reasoning (PANTHER) team is developing solutions to enable analysts to work faster and more effectively when looking at large, complex data sets.

The team’s fundamental research in cognitive science will inform the design of software and tools to help those viewing the data and make information of interest, or trends, easier to spot.

One of PANTHER’s accomplishments includes rethinking how to compare motion and trajectories. Current approaches to obtaining meaningful information from trajectories focus on comparing one trajectory to another. The approach can consume a lot of time and computing power if there are millions of trajectories to consider.

The team developed the Tracktable code to automate the observation of motion and trajectories. It stores and represents trajectories so that computers can compare them quickly and effectively. Instead of trillions of comparisons, the software does the same job in millions of comparisons, which is more manageable.

For example, an analyst concerned about the number of airliners stuck in holding patterns near airports could ask Tracktable about aircraft trajectories that are making a certain pattern of turns. Tracktable then calculates geometric features, such as the number of 90-degree turns an aircraft flew or the length of a straight line. By associating a type of motion with these features and assigning a number to each feature, the computer can group flights that behave in similar ways and show them to the viewer for interpretation.

PANTHER’s geometric and temporal trajectory analyses of air traffic over the U.S. on April 4, 2014. Image credit: Andy Wilson.The code could be applied to any problem that involves movement, such as airliners, cars or people.

PANTHER has also examined the predictive capability of information buried in data. If an analyst looks at the first half of a flight, considers historical data about similar flight paths and then looks at the second half of the flight, any deviation from the pattern might cue him or her to take a closer look. Finding that outlier from millions of flights that have flown before takes about a second with Tracktable.

The analyst is alerted because PANTHER team members use advances in cognitive science to design visual results that highlight the odd behavior of a single aircraft. By studying how analysts use visual data, Sandia researchers are figuring out ways to make an outlier pop out of a screen full of detail to demand an analyst’s attention.

The Sandia team is now looking at integrating motion and trajectories into a system called GeoGraphy, which helps analysts search for items of interest and shows changes over time. GeoGraphy is a software that converts remote sensing images expressed in pixels into nodes and edges in a graph to show changes over time and make the data searchable.

GeoGraphy breaks the images into categories, such as buildings, trees or rivers. This pre-processing creates a graphic resembling a complex paint-by-number scheme that shows the categories of everything in the image. The program uses nodes and edges to describe relationships between objects, such as distance or time.

For example, to find a school, the analyst tells the program to search for large buildings near regions that look like parking lots, play fields and tennis courts and defines those items. The analyst can then choose from among the results the computer provides.

The PANTHER team has also focused on enhancing viewer experience by conducting cognitive science experiments to learn how analysts’ expertise affects their visual cognition and to create a model of how top-down visual attention—when a user approaches an image with a goal in mind—works. The researchers hope to use the answers they find to such fundamental cognitive science questions to inform the design of new tools that will improve interactions between humans and computers.