One of the signs of the times is the number of start-ups dedicated to building silicon to process machine learning and AI workloads. It’s a fun challenge to take on NVIDIA in data center training, CPUs or add-in cards on data center inference, or novel compute methods for edge inference. One of the benefits of using ‘AI focused’ chips is usually in power, performance, efficiency, or cost – or a mixture of all of them. Hopefully the software and compiler is also there to meet that need.
However, one founders of such a startup, Ljubisa Bajic, who founded Tenstorrent, says that even the dedicated AI silicon today is too generalized for what it needs to be. His new startup, called Taalas (meaning locksmith in Hindi), is promising to break that efficiency barrier by several orders of magnitude again, by developing an architecture and chips that end up model-specific.
Drago Ignjatovic, Lejla Bajic, and Ljubisa Bajic, co-Founders
The new company has raised $50 million in two mini-rounds ($12m and $38m), from Quiet Capital and Pierre Lamond, under the edict that silicon can be further optimized to be model specific at the point of manufacturing. While AI and ML is moving at a rapid pace, both in software and hardware, we are starting to see enough of a trend in ‘good enough’ models such that dedicated computational pathways could indeed herald a more dedicated, efficient, chip approach.
In speaking with peers, we think that Taalas will ultimately be using a form of hardened configurable hardware – in a space that exists between a truly fixed function ASIC/DSP or a fully reconfigurable hardware solution like an FPGA or a CGRA (both of which have found niches in AI as well). A number of silicon design companies in this space run eASIC, or a structured ASIC, businesses where the underlying hardware is configurable, but at the point of final manufacture, can be locked into a given configuration. That allows the manufacturing process to still create a general programmable chip, but then have the benefits of reduced reconfigurable overhead for deployment into customer markets.
According to Taalas, this solves two main issues with AI hardware today – power efficiency and cost. The expected pervasiveness of machine learning in a consumer’s day-to-day life is set to be as ubiquitous as electricity, and so it will exist in everything from cars to white goods to smart meters and everything within a stack that can be electrified. In order to meet those needs for cost, compute power/efficiency, and also the fact that some/most of these devices wouldn’t ever connect to the internet, this hardware needs to be dedicated and fixed at the time of deployment. This only happens when the compute workload is fixed (or simple), which Taalas and Ljubisa see as a frontier coming soon, if not already here today.
Simply put, if you need a Llama2 model with 7B parameters in a product, and the company is certain that is all it needs during the lifetime, then a dedicated hardcore Llama2-7B chip and model with the lowest power and lowest cost for that handheld device is all you might ever need.
Taalas is coming out of stealth this week. The team is based in Toronto, Canada, and features expertise from AMD, NVIDIA, and Tenstorrent. The company is taping out its first large language model chip in the third quarter of 2024 and planning to make it available to early customers in the first quarter of 2025.
“Artificial intelligence is like electrical power – an essential good that will need to be made available to all. Commoditizing AI requires a 1000x improvement in computational power and efficiency, a goal that is unattainable via the current incremental approaches. The path forward is to realize that we should not be simulating intelligence on general purpose computers, but casting intelligence directly into silicon. Implementing deep learning models in silicon is the straightest path to sustainable AI," said Ljubisa Bajic, Taalas' CEO.
A side note, I find it amusing that they’re calling the process of optimized models and hardware ‘Foundry’. Whoever had that idea deserves a medal.
Ian, what thoughts do you have about Ljubisa Bajic departure from Tenstorrent? I wonder why he is not still with Jim Keller.
This might be a dumb question but with the coming emergence of chiplets and advanced packaging aren’t Taalas et al taking a big risk in building custom dedicated chips instead of chiplets?