A close up of the IBM Artificial Intelligence Unit chip. Image: IBM
The hype suggests that artificial intelligence (AI) is already everywhere, but the technology that actually powers it is still being developed. Many AI applications are powered by chips that weren’t designed for AI—instead, they rely on general-purpose CPUs and GPUs created for video games. That mismatch has led to a wave of investment from tech giants like IBM, Intel and Google, as well as from startups and VCs — in the design of new chips designed expressly for AI workloads.
As technology improves, venture capital will surely follow. According to Gartner, AI chip revenue was over $34 billion in 2021 and is expected to reach $86 billion by 2026. Also, the research firm said, workload accelerators accounted for less than 3% of data center servers in 2020, while more than 15% are expected by 2026.
IBM Research, for its part, recently unveiled the Artificial Intelligence Unit (AIU), a prototype chip specialized for AI.
“We’re running out of computing power. AI models are growing exponentially, but the hardware to train these behemoths and run them on servers in the cloud or on edge devices like smartphones and sensors hasn’t evolved as fast,” IBM said.
Also: Can AI help solve education’s big data problems?
The AIU is the IBM Research AI Hardware Center’s first complete system-on-a-chip (SoC) designed expressly to run enterprise AI deep-learning models.
IBM argues that the “workhorse of traditional computing”, otherwise known as the CPU, was designed before the advent of deep learning. While CPUs are good for general-purpose applications, they are not so good at training and running deep-learning models that require massively parallel AI operations.
“There’s no question in our mind that AI is going to be a fundamental driver of IT solutions for a long, long time,” Jeff Burns, director of AI Compute for IBM Research, told ZDNET. “In this computing landscape, these complex enterprise IT infrastructures and solutions will be connected in a much broader and more diffuse way.”
For IBM, it makes the most sense to effectively build complete solutions globally, Burns said, “so that we can integrate those capabilities across different compute platforms and support a much, much broader variety of enterprise AI needs.”
An AIU is an application-specific integrated circuit (ASIC), but it can be programmed to run any type of deep-learning task. The chip has 32 processing cores and 23 billion transistors built in 5nm technology. The layout is simpler than that of a CPU, it is designed to send data directly from one compute engine to another, making it more energy efficient. It is designed to be as easy to use as a graphics card and can be plugged into any computer or server with a PCIe slot.
To conserve energy and resources, AIU leverages predictive computing, a technology developed by IBM to trade computational precision in favor of efficiency. Traditionally, calculations have relied on 64- and 32-bit floating point arithmetic, to provide a level of precision that is useful for finance, scientific calculations and other applications where detailed precision is important. However, that level of precision is not necessary for many AI applications.
“If you think about planning the trajectory of an autonomous driving vehicle, there is no exact location in the lane that the car needs,” explains Burns. “There’s a range of places in the lane.”
Neural networks are fundamentally inferential – they produce output with probability. For example, a computer vision program can tell you with 98% certainty that you are looking at a picture of a cat. Nevertheless, neural networks were still trained with high-precision arithmetic, consuming significant energy and time.
Also: I tested the AI Art Generator and here’s what I learned
AIU’s approximate computing technology allows it to drop a quarter of the information from 32-bit floating point arithmetic into a bit-format.
To ensure the chip is truly universal, IBM has focused on more than hardware innovation. IBM Research has a heavy emphasis on foundation models, with a team of between 400 and 500 people. In contrast to AI models that are built for a specific task, base models are trained on a broad set of unlabeled data, creating a vast database-like resource. So, when you need a model for a specific task, you can retrain the base model using a relatively small amount of labeled data.
Using this approach, IBM wants to solve different verticals and different AI use cases. There are a handful of domains for which the company is building foundational models—use case areas such as chemistry and time series data. Time series data, which simply refers to data collected at regular intervals of time, is important for industrial companies that need to observe how their equipment is performing. After building base models for a few key areas, IBM can develop more specific, vertical-driven offerings. The team has ensured that the software for AIU is fully compatible with IBM-owned Red Hat’s software stack.
