d-Matrix is bringing NVIDIA NVLink Fusion into its next-generation Raptor inference accelerator, giving the chip a direct path into NVIDIA-connected rack-scale AI systems. Raptor combines d-Matrix’s 3D in-memory compute architecture with RISC-V orchestration, while NVLink Fusion extends that design into a broader scale-up infrastructure.

Raptor is being designed for NVIDIA-connected AI racks

d-Matrix is taking its next-generation Raptor inference accelerator beyond the accelerator card itself.

On September 10, the company said Raptor will adopt NVIDIA NVLink Fusion, giving the custom AI chip a direct path into NVIDIA-connected data-center systems.

The move is important because NVLink Fusion is not just a cable or a standalone interconnect.

NVIDIA built it as a platform for custom CPUs and XPUs that need to participate inside rack-scale AI infrastructure. By adopting that platform, d-Matrix is designing Raptor to become one component inside a larger heterogeneous system rather than an isolated accelerator.

That turns the story from a new chip announcement into a rack-level architecture story.

NVLink Fusion is the bridge between custom silicon and the rack

NVIDIA describes NVLink Fusion as a way for custom processors to connect into its scale-up infrastructure.

The platform combines NVLink connectivity with the surrounding rack architecture needed to integrate specialized CPUs and XPUs alongside other processors.

For d-Matrix, that means Raptor can keep its own inference-first architecture while gaining a standardized connection into a broader NVIDIA rack.

The accelerator does not need to become an NVIDIA GPU.

It can remain a d-Matrix processor with its own memory architecture, execution model and software stack, while NVLink Fusion handles the path into the shared scale-up fabric.

That separation is what makes the integration technically interesting.

Raptor starts with a very different memory architecture

The core of Raptor is d-Matrix’s 3DIMC architecture.

3DIMC stands for 3D In-Memory Compute. Instead of treating memory and compute as distant parts of the accelerator, d-Matrix is stacking DRAM and compute much more tightly together.

The company first validated the approach through its Pavehawk test silicon and plans to bring the technology into a commercial accelerator with Raptor.

Raptor is also chiplet-based, which lets d-Matrix combine multiple specialized pieces inside one accelerator design.

The result is a processor built around fast movement between memory and inference compute, with the memory system treated as a first-class architectural component rather than a supporting block.

RISC-V becomes the orchestration layer inside Raptor

Raptor also includes a separate control and orchestration layer.

d-Matrix selected Andes’ AX46MPV, a 64-bit Linux-capable RISC-V processor, for the architecture.

According to the companies, the RISC-V CPU will manage workload distribution, memory coordination and runtime control across Raptor’s compute fabric. It can also handle supporting vector operations around the inference pipeline.

That gives Raptor multiple levels of specialization.

The 3DIMC layer focuses on the memory-compute path. The RISC-V cores coordinate the system. The chiplet structure connects those pieces into the accelerator.

NVLink Fusion then extends the design one level higher, connecting the accelerator into rack-scale infrastructure.

The architecture now stretches from silicon to the rack

Put those layers together and the shape of the system becomes clearer.

At the lowest level, Raptor combines compute and stacked DRAM.

Above that, RISC-V processors coordinate work across the accelerator.

At the device level, chiplets form the larger Raptor package.

Then NVLink Fusion provides a path from the custom accelerator into NVIDIA-connected scale-up infrastructure.

That is a much broader design space than building a standalone PCIe card.

Each layer has a different job, but the full system is designed around one continuous inference path from local memory access to rack-scale communication.

d-Matrix has already been moving toward heterogeneous inference

The NVLink Fusion announcement also fits d-Matrix’s wider infrastructure direction.

Earlier this year, Parasail announced that it was deploying d-Matrix Corsair accelerators alongside NVIDIA Hopper and Blackwell systems.

In that deployment, different processors can be assigned different parts of an inference workflow while operating inside the same broader service.

Raptor pushes that idea further into the hardware architecture itself.

Instead of only placing separate accelerator types next to each other in a data center, the next generation can connect more directly through a common rack-scale fabric.

That makes heterogeneous inference less about separate islands of compute and more about coordinated infrastructure.

The next milestone is a 2027 rack-scale deployment path

According to Reuters, d-Matrix expects Raptor to complete its final design stage by the end of 2026.

NVIDIA-compatible rack systems using the new integration are expected in 2027.

d-Matrix is also working with Astera Labs on connectivity around the broader system, adding another infrastructure layer around high-speed data movement.

The timing matters because Raptor is still being built as a next-generation platform.

Today’s announcement is therefore about the architecture that d-Matrix wants ready before those systems arrive: specialized inference silicon, tightly integrated memory, local orchestration and a direct scale-up path into NVIDIA-connected racks.

The Upgrade Feeling

Raptor shows how the definition of an AI chip is expanding.

The accelerator itself still matters, but the surrounding system now matters just as much.

d-Matrix is combining its own 3D memory-compute architecture with RISC-V control, chiplets and NVIDIA’s rack-scale connectivity layer.

That creates a stack where each company contributes a different piece without forcing every part of the system into one processor architecture.

The upgrade is the connection between those layers.

Raptor is not only being designed to run inference. It is being designed to arrive already connected to the kind of rack-scale infrastructure where modern AI services are increasingly built.

Jalapeño Moves OpenAI From Models and Serving Software Into Silicon

OpenAI has spent years working above the chip.

Models.

Inference kernels.

Serving software.

APIs.

Products such as ChatGPT and Codex.

Jalapeño adds another layer underneath them.

OpenAI and Broadcom unveiled Jalapeño in June 2026 as OpenAI’s first custom inference processor. On August 25, OpenAI published its first measured performance results from engineering hardware running public language models.

The chip is not presented as a general consumer processor.

It was designed around large-language-model inference.

That distinction defines the project.

Training builds or updates model weights.

Inference uses those trained weights to answer requests.

Every ChatGPT response, API completion or agent step becomes an inference workload somewhere in the serving infrastructure.

OpenAI is now designing hardware specifically around that workload.

The company describes Jalapeño as the first generation of a multigenerational compute platform built with Broadcom and other infrastructure partners.

The shift is architectural.

OpenAI is no longer optimizing only the model that runs on the machine.

It is also designing part of the machine around the model-serving process.

Inference Has Several Phases With Different Bottlenecks

One reason to design custom inference hardware is that serving a language model is not one uniform operation.

OpenAI separates the workload into phases.

Prefill processes the user’s prompt and existing context.

Decode generates the response token by token.

Those phases stress the system differently.

OpenAI describes prefill as more compute-intensive.

Decode depends more heavily on memory bandwidth because the system repeatedly accesses model state while producing each next token.

Communication becomes another part of the workload when tensors, model state or cached information need to move between cores or accelerators.

The hardware can therefore spend time computing, moving data or waiting for another part of the system.

Jalapeño was designed around those transitions.

Instead of optimizing one isolated arithmetic peak, OpenAI says it designed compute, memory, networking and software together around the full inference request.

The goal is to keep the useful work moving through the system.

That makes inference performance a systems problem.

The chip matters.

The memory matters.

The network matters.

The serving software deciding where every piece of work goes matters too.

KV Cache Placement Becomes a Hardware Design Problem

The KV cache is one of the clearest examples of software behavior turning into hardware architecture.

During autoregressive generation, a transformer reuses information from earlier tokens instead of recomputing everything from the beginning for every new token.

That reusable state is stored in the key-value cache.

Long conversations and agent sessions can make that state substantial.

Where the cache lives affects how far the data must travel and how quickly it can be reused.

OpenAI says Jalapeño allows model state, including the KV cache, to be explicitly placed and kept local while the system activates the required combination of compute, memory and networking.

That is a hardware-software decision.

The serving layer knows what model state exists.

The hardware exposes a structure that lets the system place that state deliberately.

The network connects the pieces that need to communicate.

The architecture is therefore shaped around a pattern created by language models themselves.

The cache is no longer just an implementation detail inside inference software.

It becomes part of how the accelerator system is organized.

The Network Is Part of the Accelerator Architecture

A custom inference chip does not operate alone.

Large models can span many accelerators.

Requests can move through several devices.

Expert models may need to route work to different parts of the system.

Cached state can be distributed.

Jalapeño treats networking as part of the architecture rather than an external connection added afterward.

OpenAI says the chip was designed to minimize data movement and communication delays and that its network is integral to the system.

Broadcom contributes networking technology, including Tomahawk networking silicon, to the wider platform.

That matters because inference speed depends on more than tensor arithmetic.

A processing unit can only continue when the data it needs is available.

If the system can keep model state closer to the compute that will use it and move information efficiently when communication is necessary, more of the request can stay active.

OpenAI calls the resulting connected area a large domain in which the workload can remain inside one coordinated system.

The chip and network therefore form one serving architecture.

The accelerator does the computation.

The network helps make that computation available across the larger model.

OpenAI Designed Jalapeño Around Both Prefill and Decode

Some inference systems can be organized around separate resources for different phases.

Jalapeño takes another route.

OpenAI describes the accelerator as balanced and fungible across prefill and decode.

The same architecture is intended to handle both phases while adapting to the changing mix of compute, memory and communication requirements.

That becomes relevant for interactive agents.

An agent can receive context.

Generate a short output.

Call a tool.

Receive new information.

Generate again.

Repeat the sequence many times.

The workload moves through prefill and decode repeatedly rather than following one long static pattern.

OpenAI says this changing balance was part of the design target.

The hardware therefore reflects the behavior of the software being served.

A conventional chat completion is one inference pattern.

A long, multi-step agent session creates another.

The processor is designed around the idea that the mix can change during real use.

That is an example of custom silicon being shaped by product workload instead of being designed independently from it.

OpenAI Tested the Chip on Three Public Model Families

The August results were not limited to one OpenAI model.

OpenAI tested Jalapeño with GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T.

That matters because a chip designed by a model company could otherwise be interpreted as hardware for one internal model family.

OpenAI says Jalapeño was designed to support current and future language models across the industry.

The three public tests give the company a way to measure that claim on architectures developed outside OpenAI as well as its own open-weight model.

The models also differ in scale and serving behavior.

That gives the benchmark more than one workload.

OpenAI reports that the chip remained on the throughput-per-power and latency frontier across the tested operating range for all three.

Those are OpenAI’s reported results from the InferenceX benchmark environment.

The larger architectural point does not depend on one comparison.

Jalapeño is being programmed as a general language-model inference target.

The hardware is custom.

The model support is intended to remain broader than one model.

The Public Results Measure Throughput and Latency Together

Inference performance can be described in several ways.

Tokens per second.

Tokens per user.

Time between generated tokens.

End-to-end request latency.

Throughput per watt.

Maximum total serving capacity.

A system can look different depending on which measurement is chosen.

OpenAI says it evaluated Jalapeño at matched user experience using InferenceX, a public benchmark from SemiAnalysis.

The company measured how much AI work the system could complete per unit of power while meeting latency requirements.

Across GPT-OSS 120B, DeepSeek R1 and Kimi K2.5, OpenAI reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems used in those tests.

For highly interactive operating points, OpenAI reports 2.1 to 4.1 times higher performance.

Those figures belong to the specific benchmark configurations OpenAI published.

They are useful because they show what the company is optimizing.

Not only peak arithmetic.

Not only one-user latency.

The design target is the combination of serving volume, response time and power.

Power Efficiency Is Becoming a Serving Metric

Jalapeño also makes power part of the performance discussion.

OpenAI rates the processor package at 700 watts.

The company says measured sustained power remained at or below 550 watts on the workloads used in its August tests.

OpenAI then normalizes benchmark results using published chip power ratings for the compared accelerators.

That produces throughput-per-kilowatt figures.

The reason is practical.

AI infrastructure is limited by more than the number of chips a company can buy.

A data center has electrical capacity.

Cooling capacity.

Rack limits.

Network capacity.

The same amount of available power can support different amounts of useful inference depending on the complete system.

A custom accelerator can therefore be evaluated by how much model-serving work fits inside a power envelope.

This is especially relevant when the operator is also the company serving the model.

The hardware decision eventually reaches the product as response capacity.

More useful work per kilowatt means the same electrical infrastructure can process more inference requests.

That is the connection between chip architecture and service scale.

The Chip Was Co-Designed With Broadcom and Celestica

Custom silicon does not mean one company manufactures every layer itself.

OpenAI designed Jalapeño’s architecture around its model and serving requirements.

Broadcom provides silicon implementation and networking expertise.

Celestica contributes board, rack and system integration work.

OpenAI describes the collaboration as a multi-generation platform rather than one isolated processor.

That division of work is important.

A complete accelerator program requires more than a block diagram.

The design has to become physical silicon.

The chip needs packaging.

Boards.

Power delivery.

Memory.

Networking.

Rack integration.

Production systems.

Software.

Deployment tooling.

The Jalapeño project connects OpenAI’s model and serving knowledge to companies that specialize in turning those requirements into production infrastructure.

This is how a model developer can move into custom hardware without becoming every supplier in the semiconductor chain.

The architecture starts closer to the workload.

Partners industrialize the system around it.

AI Models Were Used During the Chip Development Cycle

AI was also used to build the processor that will run AI.

OpenAI says earlier generations of its models assisted engineers during Jalapeño design and bring-up.

The company reports moving from initial design to manufacturing tapeout in nine months for the chip-development phase it describes.

AI was used to explore implementations, shorten design and verification loops and optimize arithmetic circuits.

That creates a feedback loop.

Models run on hardware.

Model behavior tells engineers where the serving system spends time.

AI tools help engineers design and verify a new accelerator.

The new accelerator is then used to serve later models.

This does not remove semiconductor engineering from the process.

It changes some of the tools inside it.

The same class of software being optimized for deployment becomes part of the engineering workflow used to build the deployment hardware.

Jalapeño therefore represents two forms of hardware-software co-design.

The chip is designed around AI workloads.

AI is also used during parts of the chip-design process.

The Programming Model Was Designed for Humans and AI

OpenAI also designed the software interface with AI-assisted programming in mind.

The company describes Jalapeño as a predictable programming target based on local tensors, explicit communication and predictable synchronization.

Those properties make the hardware mapping problem more structured.

An engineer can describe the work.

An AI system can help decide where that work should be placed, scheduled and coordinated across the accelerator system.

This matters because new model families still require new kernels and model-specific optimization.

The hardware does not automatically run every architecture at maximum efficiency.

The software stack has to adapt.

OpenAI reports using Codex with GPT-Astra to bring GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 to high performance on Jalapeño within two months even though those models were not part of the original production plan.

For selected GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran 1.5 to 1.8 times faster than the existing expert-written implementations.

OpenAI explicitly limits those numbers to selected blocks rather than the complete model.

The important architectural point is the programming loop.

Custom hardware and AI-assisted kernel development are being designed together.

Serving Software Becomes Part of the Silicon Advantage

A custom accelerator only becomes useful when the serving stack can keep it busy.

Model weights have to be loaded.

Requests have to be scheduled.

KV cache has to be placed.

Communication has to be coordinated.

Kernels have to match the model architecture.

Batching and interactive traffic have to share the system.

That software layer is one reason OpenAI describes Jalapeño as a full-stack project.

The company operates ChatGPT, Codex and the API.

Those products generate real serving patterns.

OpenAI can observe how the workloads behave and use those observations when designing the chip and its runtime.

Then the software can be adjusted around capabilities added to the hardware.

The information moves both directions.

Product workload informs infrastructure.

Infrastructure changes what the product can serve.

That feedback loop is different from purchasing a processor whose architecture was designed independently from one company’s specific deployment patterns.

Jalapeño makes the inference operator part of the silicon-design process.

Custom Silicon Adds Another Compute Source to OpenAI's Infrastructure

Jalapeño adds another compute source to OpenAI’s infrastructure strategy.

OpenAI says it will continue widely deploying accelerators from NVIDIA and other partners for both training and inference.

The custom processor therefore sits beside external hardware in the wider compute fleet.

That is consistent with the scale of modern AI infrastructure.

Training and inference use different workload mixes.

Individual model families can benefit from different accelerator characteristics.

Capacity has to grow across several suppliers and data-center environments.

Jalapeño gives OpenAI a first-party architecture it can shape around its own serving requirements while partner accelerators continue supplying substantial compute capacity.

This makes the custom-chip strategy easier to understand.

It is another layer of infrastructure optimization.

Some workloads can run on OpenAI-designed silicon.

Others can continue running on external accelerator platforms.

The serving system can grow with both.

The First Generation Is Planned for Deployment by the End of 2026

The August results come from engineering hardware and pre-deployment qualification.

OpenAI says it plans to begin deploying Jalapeño within its compute infrastructure by the end of 2026.

Before that scale-up, the company says it is continuing production qualification, software maturation and validation across additional models.

That status matters.

The published benchmark results show working first-party silicon.

They do not mean the chip is already carrying the full production load of OpenAI’s services.

The project is moving from development into deployment.

Broadcom and OpenAI have described the platform as intended for large-scale data-center rollout across multiple generations.

The infrastructure transition will therefore happen over time.

First silicon.

Benchmarks.

Production qualification.

Software maturity.

Rack integration.

Deployment.

Then later generations.

That sequence is how a custom accelerator becomes part of an operating AI service rather than remaining a laboratory prototype.

Gen 2 and Gen 3 Turn Jalapeño Into a Roadmap

OpenAI already describes Jalapeño as the first generation of a longer processor roadmap.

In the August update, the company said Gen 2 was deep in development and Gen 3 was taking shape.

That changes the meaning of the first chip.

A one-off ASIC can optimize one moment in model architecture.

A multigenerational platform can learn from each deployment cycle.

The first generation reveals how real workloads behave on first-party silicon.

The software team learns which kernels need improvement.

The hardware team sees where data movement occurs.

The serving team sees which workloads map well to the architecture.

Those observations can feed the next design.

At the same time, model architecture continues changing.

Context lengths grow.

Agent workflows become more iterative.

Mixture-of-experts routing changes communication patterns.

New numerical formats alter memory and arithmetic requirements.

A recurring silicon roadmap gives OpenAI a way to incorporate those changes into later hardware.

The processor becomes part of the model roadmap instead of a separate infrastructure purchase.

Jalapeño Shows Why AI Inference Is Moving Toward Full-Stack Design

Jalapeño is useful as a case study because it connects parts of AI infrastructure that are often discussed separately.

The model defines the computation.

Inference software organizes the request.

The KV cache creates persistent state.

Memory bandwidth moves that state.

The network connects accelerators.

The scheduler decides where work goes.

Power limits determine how much of the system fits inside a data center.

The chip architecture has to support all of those layers.

OpenAI designed Jalapeño around that complete serving path.

Broadcom and Celestica help turn the architecture into deployable systems.

OpenAI models help program and optimize the hardware.

The same hardware is tested on several public model families.

The first measured results are now available, while production deployment is planned to begin later in 2026.

That makes custom silicon more than a chip story.

It is an inference-stack story.

As AI products depend on larger volumes of interactive model serving, the companies operating those products have more reason to optimize below the software layer.

Jalapeño is OpenAI’s first step into that layer.

That is the upgrade.