The reason why you are able to ask ChatGPT "how to eat a mango without spilling it" and much more is Nvidia's H100 and B200 Graphics Processing Units (GPUs). These magical chipsets that power AI chatbots have propelled Nvidia to become the frontrunners of the AI hardware industry, with market capitalization reaching the 3 trillion-dollar mark-more than Microsoft and Apple last month.
But now, a relatively young startup founded by two Harvard dropouts have set their eyes on their share of the AI hardware pie. Etched, the California based startup, is looking to disrupt the AI chipset market with their transformer ASIC (Application Specific Integrated Circuits) chip called Sohu.
Sohu claims to be 20 times faster in running transformers like ChatGPT than Nvidia's flagship- H100. The B200, which is the more powerful Nvidia offering than H100, is reportedly 10 times slower then Sohu, according to the claims made by the company based on emulation tests.
Source: X/@Etched
Sohu is taking an entirely different approach to providing high computational power to run billions of parameters (Variables that are used in training an AI Model) for transformer models. Unlike GPUs that can do multiple computationally heavy tasks (like rendering graphics in real time), Etched is choosing to create a specialised chip that caters to only transformer AI models - ones that run ChatGPT, Sora (OpenAI's text to video AI model) and Google's Gemini.
What this means is it cannot run other AI models like Convolutional Neural Networks (used for image recognition). This opens up the possibility of exploring new AI products by developers that were till now not possible due to limited power on GPUs.
For example, Sohu can potentially lead to a real-time translator that can hear and read Hindi, Gujarati or Tamil and respond back in French, English and German. Of course, such multimodal and multilingual translation needs more than just the computational power, but in theory, it opens up the possibility.
Another multimodal application of transformers that the chipset might tap into is integrating visual and language areas. This essentially means that such a model will understand both text and images simultaneously, throwing open the possibility of visual questions and answers- like an interview.
But all of this remains a theory. Etched has raised 120 million USD on 25 June to make it a reality, with a real timeline as to an actual release of the Sohu ASIC remaining elusive.
Etched has claimed it already has "tens of million dollars" worth of hardware reserved in preorders. The company has also secured a deal with TSMC (Taiwan Semiconductor Manufacturing Company) to fabricate the 4-nanometer chip, promising the deal is going to help "ramp up our first year of production."