Diagram showing data transmission of a five-layer model. Source: HKBU

A team of researchers from Hong Kong Baptist University (HKBU) along with a team from Tencent Machine Learning have successfully developed a new method for training artificial intelligence (AI) machines faster than ever while still ensuring that they maintain accuracy.

To demonstrate, the team trained two deep neural networks. The first deep neural network AlexNet was trained in just four minutes while the second network called ResNet-50 was trained in just 6.6 minutes. In earlier tests, AlexNet had achieved 11 minutes and ResNet-50 had achieved 15 minutes as their fastest training times.

According to study author Professor Chu Xiaowen: "We have proposed a new optimised training method that significantly improves the best output without losing accuracy. In AI training, researchers strive to train their networks faster, but this can lead to a decrease in accuracy. As a result, training machine-learning models at high speed while maintaining accuracy and precision is a vital goal for scientists."

The time it takes to train an AI machine relies on both computing time and communication time, according to Professor Chu. As such, the team adopted a simpler computational method called FP16 to improve the speed of computation. Additionally, because communication time is impacted by the size of data blocks, the team employed “tensor fusion,” which is a method for combining smaller slices of data into larger ones, a process that optimizes the transmission pattern and that results in improved communication efficiency during the training.

To read the study, go to arXiv.