A fresh investment round led by Blackrock has secured $640 million for Groq, a business that is creating chips to run generative AI models faster than traditional processors. The company announced this information on Monday. Participating companies included Samsung Catalyst Fund, Cisco, KDDI, Neuberger Berman, and Type One Ventures.
Groq, which was allegedly hoping to raise $300 million at a somewhat lower ($2.5 billion) valuation, is celebrating a significant victory with this tranche, which puts the company’s total raised to over $1 billion and values it at $2.8 billion. Groq raised approximately $1 billion in April 2021 and $300 million in a funding round headed by D1 Capital Partners and Tiger Global Management. This valuation more than doubles that amount.
Groq also announced today that Stuart Pann, the former head of Intel’s foundry business and former CIO at HP, will join the startup as chief operational officer. Yann LeCun, the main AI scientist at Meta, will advise the company technically. Given Meta’s investments in its own AI chips, LeCun’s appointment is a little surprising, but it definitely provides Groq a strong ally in a competitive market.
Groq is developing an LPU (language processing unit) inference engine after coming out of stealth in 2016. The business asserts that their LPUs can operate generative AI models that are currently in use at 10 times the speed and 1/10th the energy of OpenAI’s ChatGPT and GPT-4o.
Jonathan Ross, the CEO of Groq, is renowned for his contribution to the creation of the tensor processing unit (TPU), Google’s proprietary AI accelerator processor used for model training and execution. Nearly ten years ago, Ross co-founded Groq with Douglas Wightman, an entrepreneur and former engineer at Google parent firm Alphabet’s X moonshot lab.
Groq offers an LPU-powered developer platform called GroqCloud, which includes an API that lets users use its chips in cloud instances, as well as “open” models like Google’s Gemma, OpenAI’s Whisper, Mistral’s Mixtral, and Meta’s Llama 3.1 series. (Groq also runs GroqChat, an AI-powered chatbot playground that it introduced at the end of last year.) More than 356,000 developers were using GroqCloud as of July. According to Groq, some of the money raised during this round will be utilized to expand the company’s capacity and introduce new models and features.
“Many of these developers are at large enterprises,” stated Stuart Pann, COO of Groq. “By our estimates, over 75% of the Fortune 100 are represented.”
As generative AI becomes more popular, Groq will have to contend with competition from rival AI chip startups as well as Nvidia, the industry leader in AI hardware.
Nvidia is attempting to preserve its dominance in the market for AI processors, which are needed to train and implement generative AI models. The company is expected to dominate between 70% and 95% of this industry.
As opposed to every other year, like in the past, Nvidia has pledged to release a new AI chip architecture annually. Additionally, it is apparently starting a new business unit dedicated to creating custom chips for cloud computing companies as well as other businesses, including AI devices.
Groq faces competition from Nvidia as well as Amazon, Google, and Microsoft, who now provide or plan to provide customized chips for artificial intelligence workloads on the cloud. Customers of Google Cloud can use the aforementioned TPUs as well as Google’s Axion chip in due course. Microsoft recently introduced Azure instances in preview for its Cobalt 100 CPU, with Maia 100 AI Accelerator instances to follow in the coming months. Amazon offers its Trainium, Inferentia, and Graviton processors through AWS.
Analysts predict that in the next five years, the AI chip market might exceed $400 billion in sales, and Groq may see competition from Arm, Intel, AMD, and an increasing number of startups. Due in large part to cloud vendors’ increasing capital expenditures to meet the capacity demand for generative AI, Arm and AMD in particular have seen their AI chip businesses flourish.
Late last year, D-Matrix secured $110 million to launch a platform for inference computation that it described as unique in the market. With $120 million, Etched came out of stealth in June to unveil a processor designed specifically to accelerate the transformer, the most popular generative AI model architecture available today. Masayoshi Son of SoftBank is purportedly trying to fund $100 billion for a chip business in order to take on Nvidia. Additionally, it’s said that OpenAI is in discussions to start a chip-making venture with investment firms.
Groq is significantly spending on industry and government outreach in an effort to carve out its niche.
To create Groq Systems, a new business unit, Groq bought Definitive Intelligence, a Palo Alto-based company that provides a variety of business-oriented AI solutions, in March. Serving entities, such as sovereign nations and U.S. government agencies, that want to construct new data centers employing Groq processors or integrate Groq chips into already-existing ones falls within stems’ jurisdiction.
More recently, the startup Groq signed an agreement to install tens of thousands of its LPUs in the Norway data center of the European company Earth Wind & Power. Additionally, Groq teamed with Carahsoft, a government IT contractor, to market its solutions to public sector clients through Carahsoft’s reseller partners.
Additionally, Groq and Aramco Digital, a Saudi Arabian consulting business, are working together to install LPUs in upcoming Middle Eastern data centers.
Groq, a company located in Mountain View, California, is developing new ties with customers while simultaneously moving forward with the development of its microprocessor. The company said in August of last year that it will be working with Samsung’s foundry division to produce 4nm LPUs, which are anticipated to outperform Groq’s initial 13nm chips in terms of efficiency and performance.
By the end of Q1 2025, Groq intends to deploy over 108,000 LPUs.