New AI Hardware Struggles to Find Its Place in Datacenters

The semiconductor industry has spent the last few years focused on creating faster and more efficient artificial intelligence chips. Companies like NVIDIA and AMD dominate this space, but a growing number of startups are trying to break into the market with new designs. However, having a superior chip is only half the battle. The critical question for any new hardware is where it actually goes. Most of these chips are not sold to individual consumers but are destined for massive data centers, cloud providers, and large enterprises. This shift in focus moves the conversation from silicon design to the physical reality of infrastructure.

As reported by GN technics/ai (en-US), the deployment of this new technology raises significant logistical questions. It is not enough to simply plug in a card; operators must account for power consumption, water usage for cooling, and the overall energy footprint of the facility. This creates a complex trade-off. While new hardware may offer better compute per watt, the physical constraints of existing data centers often limit how much of that potential can be realized. The industry is now realizing that the bottleneck is no longer just the chip itself, but the environment in which it operates.

Infrastructure defines actual utility

Many startups, including Tenstorrent, d-Matrix, and Cerebras, have developed impressive accelerators. Yet, the market remains skeptical because the end-user experience depends on the entire stack, not just the silicon. This includes the networking capabilities that allow data to move efficiently between chips and the software frameworks that developers use to build applications. If the surrounding infrastructure is weak, the advantage of a faster chip is diminished. This is why the industry is increasingly focusing on the integration of compute and networking as a unified system rather than isolated components.

Sovereign AI drives local demand

A new trend is emerging around the concept of sovereign AI, which refers to the ability of nations and large corporations to control their own AI infrastructure without relying on foreign technology stacks. This concept has gained traction as governments and industries seek to reduce dependency on external providers. In Japan, a startup called ai& has emerged to address this need. Founded by David Bennett, the company focuses on building and operating its own data centers to provide local access to AI services.

Bennett, a former executive at Tenstorrent, explains that the term sovereign AI has evolved. It no longer just means air-gapped systems for governments but extends to any entity that needs dedicated, customized solutions. The company is leveraging brownfield sites, which are existing industrial properties, to build its facilities. This approach allows them to repurpose existing structures rather than building new ones from scratch, which can reduce construction time and environmental impact. The goal is to offer a full-stack solution that includes not just the hardware, but also the post-training of models using local data.

Challenges of repurposing industrial sites

Using brownfield sites presents its own set of challenges. These locations often have outdated electrical grids and cooling systems that were not designed for the high-density compute loads of modern AI hardware. Upgrading these systems can be costly and technically difficult. However, the trade-off is that it allows for rapid deployment in areas where land for new construction is scarce or expensive. For companies like ai&, this strategy aligns with the sovereign AI message by ensuring that critical infrastructure is built and maintained locally, reducing the risk of supply chain disruptions.

The success of these new deployments will depend on whether the physical infrastructure can keep up with the demands of the hardware. While the chips themselves are becoming more powerful, the ability to power and cool them efficiently remains the primary constraint. As the industry moves forward, the focus is likely to shift from developing the next best processor to optimizing the data centers that house them. This shift underscores a broader reality in tech: innovation is not just about what you build, but where and how you deploy it.