IBM Unveils Breakthrough Optical Data Transmission Technology That Enables 'Light-Speed' AI Training

IBM has developed a new optical technology that can help train AI models at the 'speed of light' while also saving significant energy. IBM says that by applying these new breakthroughs to data centers, the energy saved when training a typical AI model is equivalent to the annual energy use of 5,000 US households.

According to IBM, while data centers connect to the outside world using fiber optic cables, copper wires are still the predominant material used inside. These copper wires are responsible for connecting GPU accelerators that spend a lot of time waiting for data from other devices while using power, which significantly increases operating costs. Commenting on the new optical data transmission technology, Dario Gil, senior vice president and director of research at IBM, said:

As AI creates a need for more power and processing power, data centers must evolve – and generic packaged optics technology could help large-scale data centers better adapt to future demands. With this breakthrough, processors of the future will communicate in the same way fiber optic cables carry data in and out of data centers, ushering in a new era of faster, more sustainable communications that can handle massive AI workloads.

IBM Unveils Breakthrough Optical Data Transmission Technology That Enables 'Light-Speed' AI Training Picture 1

IBM outlines its new co-package optics (CPO) prototype in a technical paper. Essentially, by dramatically increasing bandwidth in data centers, GPU downtime can be minimized, accelerating AI processing.

IBM claims the technology can reduce the training time for large language models (LLMs) from three months to three weeks. At the same time, the increased energy efficiency will cut down on energy usage and reduce the costs associated with LLM training.

David Pac

Update 12 December 2024