A new resource is now available for the development of 3D models for deep learning applications in robotics, artificial intelligence and other fields. The open-source software library assembled by Nvidia supports multiple representations of 3D data.

Kaolin is a 3D deep learning library for PyTorch designed to allow researchers to load, preprocess and Applications (clockwise from top-left) include 3D object prediction with 2D supervision, 3D content creation with generative adversarial networks, 3D segmentation, automatically tagging 3D assets from TurboSquid, and 3D object prediction with 3D supervision. Source: J. Krishna Murthy et al.manipulate 3D data before it is used to train deep learning algorithms. The system includes graphics modules to edit 3D images, with functions such as rendering, lighting, shading and view warping. Moreover, it supports a wide range of loss functions and evaluation metrics are supported to streamline evaluation of deep learning algorithms under development.

With their 3D dataset ready for deep learning, researchers can choose a neural network model from a curated collection that Kaolin supplies. The interface provides a rich repository of models, both baseline and state of the art, for classification, segmentation, 3D reconstruction, super-resolution and more. Users can select a neural network model from the curated collection, which includes source code and pre-trained models for specific applications.

Future improvements and additions to Kaolin are expected to include newer differentiable rendering tools, large scale semantic and instance segmentation lidar datasets and models for 3D object detection.