The AI ​​chip has 20 times more power than Nvidia H100

According to Etched, Sohu is a form of application-specific integrated circuit (ASIC) and can beat Nvidia's H100 GPU in large language model (LLM) inference. The performance of a system with 8 Sohu chips can be equivalent to 160 H100.

Etched said that current AI acceleration systems, whether CPU or GPU, are designed to work with different AI architectures such as Short-term Memory Network, Neural Network. Convolution Neural Network or State Space Model.

Etched estimates that only 3.3% of the transistors of Nvidia's H100 GPU are used for the main task, while the remaining 96.7% are used for other tasks. With general-purpose AI chips, this is necessary but causes great waste.

So startup Etched introduced Transformer AI Architecture (Transformer architecture) into Sohu to be able to allocate more transistors for on-chip artificial intelligence computing. They strive to create chips that are optimized to run a specific type of AI model rather than accommodate every single AI architecture. In fact, the Transformer architecture is also the foundation of GPT (Generative Pre-trained Transformer), similar to AI systems of major technology companies including Sora, Gemini, Stable Diffusion and Dall-E.

Etched said it has bet on the Transformer architecture. If Transformer disappears, this startup will die, but if it continues to exist Etched has the potential to become one of the biggest companies of all time.

Sohu's price has not been announced, and the Nvidia H100 is currently priced at $40,000. According to Tom's Hardware, if Sohu meets expectations, it will help data processing centers save billions of dollars in initial costs and operating costs. This product even threatens Nvidia's leading position, but only when Etched's chip goes into operation.

Marvin Fry

Update 28 June 2024