[비즈한국] Skepticism toward AI is emerging once again. It isn’t that AI is considered "useless," but rather a recurring concern from an industrial perspective that "AI does not seem valuable enough to justify the massive investments expected." Regardless of the final outcome, we are unlikely to lose faith in the potential of AI technology or reduce its usage. For the foreseeable future, it will only continue to grow.
The investment that AI-related companies are currently pouring into computing is beyond imagination. Looking at Google’s recent earnings report, capital expenditures for 2026 are estimated at around $200 billion. Google generated $120 billion in revenue last quarter, a 24% increase compared to the same period last year. For a company of Google's scale, this level of growth is monumental. Notably, the cloud division earned 82% more than in the same period last year. There is only one reason for this: AI.
Coincidentally, however, the stock market began expressing AI skepticism immediately after this announcement. While it is true that Google is delivering better-than-expected results and has secured strong growth engines, the concern lies in the fact that capital expenditures account for an excessively high proportion of its spending. Depending on the viewpoint, the assessment of AI-leading companies, including those in the semiconductor sector, is cooling down in an instant. It is almost frightening.
AI remains the key keyword driving the current industry. Companies categorized as hyperscalers, including Google, are reaping massive profits. They are further expanding AI-related infrastructure to build better models. In effect, they are sweeping up nearly all of the world’s computing resources. Infrastructure growth is more earnest than ever, characterized by a terrifying intensity and expansion.
A Technical Thirst to Increase 'Tokens Generated per Cost'
I recently attended AMD's annual AI event, "Advancing AI," held in San Francisco. It was an occasion for AMD to introduce new AI products and announce major strategies and roadmaps. Previously, this event revolved around GPUs for AI processing and the ROCm development framework. This was because the world of AI began based on NVIDIA’s CUDA framework and operated on NVIDIA GPUs.
For AMD, it was only natural to emphasize that "AMD GPUs offer excellent performance, and ROCm can create an AI environment as easy and powerful as NVIDIA's CUDA." Especially since demand for NVIDIA GPUs has outstripped supply, presenting AMD’s GPUs—including the "Instinct" line—as the most powerful and realistic alternative was a logical move.
However, AMD’s stance has shifted subtly this year. Instead of just being an alternative to NVIDIA, they have provided realistic answers to the current hottest keywords: "performance per cost" and "number of tokens generated per cost." The basic training of major AI models has already been established, and infrastructure performance for this training has found some breathing room. But the most pressing issue now lies in "inference"—providing answers according to user demands.
AMD CEO Lisa Su stated during her keynote, "Until two years ago, the ratio of training to inference was about 6:4, but starting this year, a reversal is taking place." This means that the demand for AI in the field is exploding. Large language models now go beyond simple writing to code for large-scale development projects, analyze scientific experiments, and create images and videos. Finding and organizing information, or writing more naturally, is a given. However, handling this ultimately depends on the inference capability of the computers powering each service.
Furthermore, AI is moving toward "Agent AI," where instead of working in isolation, multiple AIs optimized for specific fields are combined to handle a single complex task. To complete one job, more AI models must run simultaneously.
This year, AMD went beyond just releasing GPUs; they introduced "Helios," an AI supercomputer that integrates CPUs, GPUs, and the network environment connecting 18 of these servers. I mention this not just because the computer is impressive, but because the direction this system pursues captures the current reality of AI perfectly.
The core of this computer is the integration of flat blade servers designed for data centers. Each tray contains a server equipped with an EPYC processor featuring 256 cores per chip, paired with four high-performance GPUs called Instinct MI455X. These GPUs are coupled with a massive 432GB of HBM4 memory. By bundling 18 of these unit blades, they form a rack system about the size of a large refrigerator. This results in a system with a staggering 4,608 cores, 72 GPUs, and 31TB (terabytes) of HBM integrated into one.
'Rack-Scale Systems': A Methodology for Securing More Computing Power in the Same Space
Helios also appears in a form factor typical of data centers, so what makes it special? In reality, for modern computers—especially for AI—total throughput is a more important metric than simple processing speed. It’s about how many calculations can be performed. Therefore, the key is to pack more computers into a single system, or a limited space. But the challenge isn’t just about fitting more in. While servers look impressive, they aren’t fundamentally different in form from the PCs we use. They are centered around CPUs and memory, with GPUs attached.
In particular, GPUs exchange data through a terminal called PCI Express, and there is a limit to the speed and volume of data that can pass through this path. Thus, when you install two GPUs in one computer to process AI, you don't get exactly double the work; there is some loss. There was a time when many GPUs were packed into a single computer, but the loss increases as the number of GPUs grows. Additionally, there are limits to both the number of GPUs that can be installed in a single computer and the available physical space. Therefore, multiple unit servers are bundled to create a single system. This is how the shelf-like "racks" are typically constructed.
However, connecting server-to-server causes even greater losses. Within one server, processing happens across the circuit board, but when connecting server to server, the results processed within one computer must be transmitted over a network. Due to basic transmission limits and the physical time taken for data to move, performance degradation increases sharply. There is a saying that connecting 100 GPUs doesn't yield 100 times the performance; it’s hard to even reach 50 times.
But that is not all. As AI processing demands increase, requiring vast amounts of GPUs, data centers connecting tens or hundreds of these racks are becoming the standard. The goal of a data center is to fit more computers into a limited space. It is just as important to improve the efficiency of connections as it is to pack in CPUs and GPUs.
The reason AMD is not selling CPUs and GPUs as "individual parts" but rather as "complete systems" like Helios is to propose a universal standard for these connections. When one 256-core CPU is paired with four GPUs, it achieves optimal performance and can be connected without bottlenecks. Connecting multiple standardized systems this way also makes it easier to operate data centers stably.
This is a concept that NVIDIA has already been presenting through "Vera Rubin." While one could interpret this as a move to monopolize platforms and sell complete products to charge more for GPUs, it is much more advantageous under the current requirement to secure infrastructure, as it is the most effective way to stably concentrate a large amount of computing power into the limited space of a data center.
In fact, AMD has packed significantly more HBM4 memory into a single rack with Helios than ever before. CEO Lisa Su says that because future AI environments will inevitably require more memory, securing as much HBM memory as possible within the same space is the most urgent task for the market.
Agent AI Is Still Thirsty for Memory
While large language models have been centered on "chatbot" forms until now, the era of Agent AI is opening. As each agent requires more inference, the orchestration of gathering those results to create value becomes important. This means that not only inference performance but also the amount of memory needed to "remember" information will increase.
AI will continue to demand greater computing power and more memory. Even now, we have been yielding limited resources, such as processors and memory required for PCs and smartphones, to AI. The price for this is coming back in the form of astronomical semiconductor costs. However, the current AI market is sucking up even more resources than that. AMD’s Helios or NVIDIA’s Vera Rubin are technically enticing proposals to address this.
Just before starting the Advancing AI keynote, AMD also announced partnership and investment plans with Microsoft and Anthropic. In particular, they decided to make a massive strategic investment of $5 billion—over 7 trillion Korean won—in Anthropic, which services Claude. Based on this, Anthropic has decided to introduce 1 gigawatt-level Helios-based AI infrastructure by next year. While the exact cost of this infrastructure is unknown, it involves an enormous amount of money, and it can be read that Anthropic is securing resources by transferring part of its stake to AMD. AMD is also securing Anthropic's stake in the form of de facto payment in kind through hardware rather than immediate revenue. NVIDIA’s recent supply of the Vera Rubin system alongside an investment in Naver can also be interpreted as a similar process.
This proves that AI requires far more semiconductors than it does now, and that there is no choice but to urgently absorb the next generation of GPUs and memory, however burdensome it may be. The AI industry is preparing to swallow even more resources in the future. The current machine-learning-based AI is ultimately a fierce battle of "probability," and the methods for basic model generation and operation have been established. It would not be wrong to interpret that we have reached the technical limits of AI. The current single-model approach cannot easily become any smarter.
However, Agent AI is a new methodology that overcomes this and allows us to trust and delegate more tasks to AI. Agent AI creates an environment different from the large language models we have seen so far. The end of that long journey will be the completion of artificial intelligence, a concept first proposed 40 years ago through machine learning techniques. And in that process, we will likely have to pay the price in the form of even more semiconductors.