The world's first Transformer-specific AI chip Sohu has been released: it is 20 times faster than NVIDIA's H100

Industry News|Industry Trends|

2024-06-28 17:16:23 701

On June 26th, it was reported that the emerging American chip startup Etched has launched its first AI chip, named Sohu. This chip demonstrates astonishing performance when running large-scale models, surpassing industry giant NVIDIA's H100 by up to 20 times. Even compared to the top chip B200 released in March of this year, Sohu shows over 10 times superior performance.

The major breakthrough of the Sohu chip lies in its direct integration of the Transformer architecture into the chip itself. According to Etched's CEO Uberti, Sohu is manufactured using TSMC's advanced 4-nanometer process. It not only far exceeds GPUs and other general-purpose AI chips in inference performance but also achieves new heights in power efficiency.

For Transformer technology, Sohu undoubtedly stands as the fastest chip on the market, showing significant orders of magnitude difference in performance compared to other products. It's worth noting that current mainstream AI products such as ChatGPT, Claude, Gemini, and Sora are all powered by Transformer architecture.

As the world's first Transformer ASIC (Application-Specific Integrated Circuit) chip, Sohu's integrated applications are particularly noteworthy. Tests have shown that a server equipped with 8 Sohu chips can match the performance of a cluster of 160 H100 GPUs. Specifically, a server with 8 Sohu chips can process up to 500,000 Llama 7B tokens per second, a figure unprecedented in the industry.

Etched, the company behind this outstanding chip, is located in Cupertino, California. Despite being founded only two years ago with a team of just 35 people, its founders Gavin Uberti (a Harvard dropout and former OctoML and Xnor.ai employee) and Chris Zhu, along with seasoned technical experts Robert Wachen and former CTO of Cypress Semiconductor Mark Ross, have successfully developed this chip designed for large-model AI, demonstrating their formidable technical prowess and boundless potential.

Prev:: China has successfully developed the world's first open-source on-chip brain-machine interface intelligent interaction system, MetaBOC, with the research published recently in the international journal "Brain" in the field of neuroscience

Next:: Alibaba's latest open-sourced Qwen2-72B instruction micro-tuning version has topped the charts in the open model rankings

The world's first Transformer-specific AI chip Sohu has been released: it is 20 times faster than NVIDIA's H100

Advisory Service