Reasoning is moving onto the device

NVIDIA’s latest Jetson update is about a shift that would have sounded much harder a year ago: running multi-step reasoning and agentic AI locally on edge hardware.

In a technical post published September 4, NVIDIA says compact open models released through 2026 have reached a point where Jetson can run capabilities that previously belonged to much larger systems. The company uses Nemotron 3.5 Lightning and Qwen3.8-27B to show what that looks like in practice.

The goal is not just a local chatbot. NVIDIA is talking about agents that can reason through several steps, call tools, and keep working inside machines and edge applications without sending every inference request somewhere else.

That gives the story a very different feel from another model launch. The model is becoming part of the device itself.

Nemotron and Qwen take two different routes

The two models in NVIDIA’s test are built differently, which makes the comparison useful.

Nemotron 3.5 Lightning is a mixture-of-experts model with 30 billion total parameters, but only about 3 billion are active for each token. That lets the model keep a larger total parameter pool while reducing the amount of compute used on any one step.

Qwen3.8-27B takes the dense route. All 27 billion parameters participate during inference.

NVIDIA does not present one architecture as the universal answer. The point is that Jetson can now host reasoning-oriented models from more than one design family, and each one can be tuned around the workload it is meant to run.

For developers building local agents, the model choice is starting to look less like “what can fit?” and more like “which architecture fits this application best?”

For an agent, that difference can shape the entire deployment. A sparse mixture-of-experts design can keep a broad model capacity while activating a smaller slice of the network for each token. A dense model spends compute across the full network every time. Jetson now supports both paths, which gives developers more room to balance responsiveness, model behavior, and the kind of reasoning their application needs.

NVFP4 cuts the work of each inference pass

The first major optimization is NVFP4 quantization.

Instead of keeping model operations at a higher numerical precision such as BF16, NVFP4 uses a much smaller 4-bit representation on supported Jetson hardware. That reduces the memory footprint and the amount of work needed for each inference pass.

For edge devices, that is a big lever. Models that look heavy on paper become much more practical when the runtime can move less data and use lower-precision tensor operations efficiently.

NVIDIA pairs that quantization with prebuilt checkpoints and optimized serving through vLLM. The Jetson AI Lab model catalog already lists Nemotron 3.5 Lightning and Qwen3.8-27B with platform-specific run options, so this is not only a benchmark recipe hidden inside a research post.

Developers can start from an existing optimized configuration and then measure it against the prompts their own application will actually use.

Speculative decoding adds another layer of speed

Quantization is only half of NVIDIA’s performance story.

The second technique is speculative decoding. A smaller draft process proposes several tokens ahead of time, and the main model verifies them. When enough of those proposed tokens are accepted, the system can produce multiple output tokens for the cost of fewer full-model verification steps.

NVIDIA tested several speculative-decoding methods rather than assuming one would win everywhere. Nemotron 3.5 Lightning performed best with DSpark in the company’s tests, while Qwen3.8-27B performed best with DFlash2.

That distinction is useful. The fastest setup is not simply “turn on speculative decoding.” The draft method and checkpoint have to match the model and the workload.

Combined with NVFP4, speculative decoding pushed the best Jetson configuration in NVIDIA’s tests to as much as 6.28× the decode throughput of the BF16 baseline.

That 6.28× number is a peak, not a universal constant

The headline number is strong, but NVIDIA’s own measurements make the more useful point: inference speed depends on what the agent is doing.

The company kept the fastest speculative-decoding configuration for each model and tested writing, reasoning, summarization, and retrieval-augmented generation workloads. Throughput moved between categories even when the model and decoding method stayed the same.

Nemotron 3.5 Lightning with DSpark ranged from 123.01 to 138.02 output tokens per second. Qwen3.8-27B with DFlash2 ranged from 27.69 to 34.44 output tokens per second.

That is why NVIDIA recommends testing with representative prompts instead of treating a general benchmark as the final answer.

For an edge agent, the practical target is not the largest number on a chart. It is the configuration that stays fast on the exact mix of reasoning, retrieval, tool use, and response generation the device will perform.

Local reasoning changes what edge systems can do

Once reasoning models can stay on the device, the list of possible applications gets much more interesting.

NVIDIA points to in-cab assistants, real-time anomaly detection, and robots operating in remote or demanding environments. These are systems where the AI is close to the sensors, machines, and people it is helping.

A local agent can inspect incoming data, reason through a situation, and act without turning every step into a round trip to a remote service. That is especially useful when the agent is part of a physical workflow rather than a browser tab.

The same idea applies to robotics. A robot that can run a reasoning model onboard has a different relationship with its environment: perception, planning, tool use, and response can stay closer to the machine.

Jetson has been moving in that direction for years. The difference now is the kind of model NVIDIA says can fit into that local loop.

The edge setting also changes what counts as useful latency. A machine reading cameras, microphones, telemetry, or industrial sensors may need to reason close to the moment the data arrives. Keeping the model on the same system means the AI can sit directly inside that loop. The result can be an assistant that responds to the current state of a vehicle, a robot that reasons over what its sensors are seeing, or an industrial agent that combines local signals before deciding what to do next.

Jetson AI Lab is turning the research into runnable recipes

NVIDIA is also making the deployment path unusually concrete.

The Jetson AI Lab model catalog lists supported models by Jetson platform and inference engine, with quick-start commands for models including Nemotron 3.5 Lightning and Qwen3.8-27B. Developers can see which Jetson modules are supported, choose an inference runtime, and generate a launch command from the page.

That closes an important gap between a performance blog and something a developer can reproduce.

The technical post also links to tutorials for running LLMs and VLMs on Jetson, benchmarking generative AI models, and getting started with speculative decoding. The message is clear: NVIDIA wants local reasoning to be a deployable developer workflow, not a one-off demonstration.

As the model catalog grows, the edge is starting to look less like a place where only tiny models run and more like another serious target for agentic AI.

The Upgrade Feeling

The most interesting part of NVIDIA’s Jetson update is not that one benchmark got faster.

It is that local reasoning is starting to feel normal.

Nemotron 3.5 Lightning can activate only a fraction of its total parameters per token. Qwen3.8-27B shows a dense model can live in the same edge conversation. NVFP4 cuts the work of each pass, speculative decoding pushes more tokens through, and Jetson AI Lab turns those pieces into something developers can actually run.

That combination changes the shape of the edge. Instead of treating the device as a sensor that hands intelligence off somewhere else, the device can increasingly keep the reasoning loop with it.

For robotics, industrial systems, and local AI agents, that is a much bigger upgrade than another raw model score.

Intel’s Hot Chips 2026 roadmap shows how the company thinks agentic AI changes hardware architecture across the stack. Diamond Rapids targets enterprise-scale orchestration with up to 256 cores, 16 memory channels, PCIe 6.0 and CXL 3.0. Crescent Island targets inference economics with up to 480 GB of LPDDR5X on a 350-watt air-cooled PCIe GPU. Wildcat Lake brings a smaller hybrid-AI design to mainstream clients and edge systems with integrated Xe3 graphics and an NPU rated at up to 17 TOPS. The three products are different, but the strategy is one: agents need orchestration, inference and local execution to work as a coordinated system rather than as one giant accelerator doing everything.

Agentic AI Changes the Hardware Question

Most AI-chip discussions ask one question.

How much acceleration can one processor deliver?

Agentic AI makes that question too narrow.

An agent does not simply generate one answer.

It can plan.

Call tools.

Run code.

Search memory.

Trigger other models.

Use databases.

Interact with the edge.

Repeat the loop.

That creates several different compute jobs inside one workflow.

Intel’s Hot Chips 2026 announcements are interesting because the company is not presenting one universal AI processor.

It is dividing the work across three architectures.

Diamond Rapids for large-scale orchestration and general compute.

Crescent Island for inference.

Wildcat Lake for mainstream client and edge execution.

The hardware is being designed around a system of agents rather than one model call.

Intel Presented Three Architectures, Not One AI Chip

Intel’s August 24 Hot Chips 2026 announcement centers on Diamond Rapids, Crescent Island and Wildcat Lake.

They sit in very different power and deployment classes.

Diamond Rapids is a next-generation Xeon server processor.

Crescent Island is a datacenter GPU optimized for inference.

Wildcat Lake is a Core Series 3 SoC for mainstream laptops and intelligent edge platforms.

Intel describes the portfolio as a way to scale agentic AI from the rack to the edge.

That framing matters because agent workloads naturally spread across infrastructure.

The orchestration may happen on a CPU.

Heavy generation may happen on a GPU.

A local model may stay on the endpoint.

A real system can use all three.

Diamond Rapids Is the Orchestration Layer

Intel positions Diamond Rapids as the compute foundation for enterprise-scale agentic AI.

That does not mean the CPU replaces accelerators.

The role is broader.

Agents need general-purpose compute for scheduling, tool execution, business logic, memory management, networking, databases and coordination.

Those jobs are not always best handled by a GPU.

Intel is betting that the CPU remains central even when most of the attention goes to AI accelerators.

The more agents an enterprise runs, the more orchestration work exists around the models themselves.

Up to 256 Cores Changes the Scale of General-Purpose Work

Intel says Diamond Rapids will offer up to 256 new cores.

That is a major increase in server-side general-purpose compute density.

Agent workloads can create a large number of parallel tasks.

One process is waiting on a model.

Another is running a database query.

Another is executing a tool.

Another is validating output.

Another is handling security policy.

A high-core-count CPU can absorb that surrounding work while accelerators focus on matrix-heavy inference.

The important point is not simply “256 cores.”

It is what those cores are being asked to coordinate.

The 1.28 GB Last-Level Cache Is Part of the Story

Intel lists up to 1.28 GB of last-level cache for Diamond Rapids.

Large cache capacity can reduce how often workloads have to reach external memory for frequently reused data.

That matters in server environments where many software layers are active at once.

Agentic systems may include model runtimes, retrieval systems, databases, orchestration frameworks and network services.

Not every workload benefits equally from a large cache.

But the specification shows that Intel is designing for high-density server workloads where keeping more data closer to the cores can matter.

Memory Bandwidth Is Becoming an AI Constraint

AI discussions often focus on arithmetic.

Data movement can be just as important.

Diamond Rapids supports 16 memory channels at up to 12,800 MT/s, according to Intel’s Hot Chips material.

More memory channels increase aggregate bandwidth.

That helps workloads that need to feed many cores with data continuously.

Agentic systems may create memory pressure from retrieval, context preparation, databases and concurrent services even when the largest model itself runs on a separate accelerator.

The CPU still needs a fast path to data.

PCIe 6.0 and CXL 3.0 Are About the Rest of the Rack

A modern AI server is not one processor.

It is a network of processors, accelerators, memory devices, NICs and storage.

Diamond Rapids includes 128 lanes of PCIe Gen 6 and support for CXL 3.0.

Those interfaces are important because they determine how much external hardware a CPU can connect to and how quickly data can move between components.

CXL can also support more flexible memory architectures.

For agentic AI, the server platform has to move information between many specialized devices without turning I/O into the bottleneck.

Foveros Direct and UCIe Show Where Intel Wants Packaging to Go

Intel is also emphasizing advanced packaging.

Diamond Rapids uses Foveros Direct 3D and UCIe-S interconnect.

UCIe is an industry standard for connecting chiplets inside a package.

The strategic idea is modularity.

Instead of building one huge monolithic die containing every function, designers can combine smaller compute blocks, I/O blocks and other components.

That can make future processors easier to scale and specialize.

Agentic AI increases the incentive for that approach because different workloads want different types of compute inside the same platform.

APX and AMX Keep the CPU Relevant to AI

Diamond Rapids also adds new Advanced Performance Extensions and enhanced Advanced Matrix Extensions.

AMX is specifically designed to accelerate matrix operations.

That means Intel is not treating the CPU only as a traffic controller.

Some AI work can remain directly on the CPU.

Smaller inference tasks.

Preprocessing.

Postprocessing.

Classical machine learning.

Vector and matrix operations embedded inside larger applications.

The architecture is trying to keep general-purpose compute useful even as specialized accelerators become more important.

Crescent Island Is About Inference Economics

Crescent Island attacks a different problem.

Inference cost.

Once a model is trained, the business problem becomes serving it repeatedly.

Agents can multiply inference demand because one user request may trigger several model calls.

A planning step.

A tool-selection step.

A verification step.

A second model.

A retry.

One agent workflow can consume far more tokens than a simple chatbot response.

Crescent Island is designed around that continuous serving workload.

480 GB of LPDDR5X Is the Headline Crescent Island Number

Intel says Crescent Island supports up to 480 GB of LPDDR5X memory.

That is a striking amount of memory for a PCIe accelerator.

Large memory capacity can allow bigger models to fit on one device.

It can also support longer context windows or more concurrent workloads.

Those are exactly the problems agent systems create.

One model may be large.

Several agents may need to run at once.

Each may carry substantial context.

Capacity becomes part of inference economics.

LPDDR5X Is an Unusual Choice for a Datacenter GPU

High-end AI accelerators often use HBM because it delivers very high memory bandwidth.

Crescent Island instead uses LPDDR5X.

That choice reflects a different target.

Intel is optimizing for lower power, large capacity and easier deployment rather than pursuing the highest possible accelerator class.

The trade-off is important.

Not every inference workload needs maximum bandwidth.

Some enterprises may value capacity, power and deployment simplicity more than peak performance.

Crescent Island is aimed at that middle ground.

350 Watts Keeps the GPU Inside Existing Air-Cooled Infrastructure

Intel lists Crescent Island as a 350-watt air-cooled PCIe card.

That matters because many enterprise datacenters are not designed for extreme accelerator power density.

A very high-power GPU may require liquid cooling or major rack redesign.

A 350-watt PCIe card can fit more naturally into existing air-cooled systems.

Intel is explicitly selling deployment economics.

The question is not only how fast the GPU is.

It is how much infrastructure has to change before an enterprise can use it.

32 Xe Cores and 256 XMX Engines Target Sustained Inference

Crescent Island uses 32 Xe cores and 256 XMX engines based on Xe3P.

XMX engines provide matrix acceleration for AI workloads.

Intel says the design is optimized for sustained inference performance and token throughput.

That distinction matters.

Training hardware is often judged by how quickly it can build a model.

Inference hardware is judged by how cheaply and reliably it can serve that model again and again.

Agentic AI pushes infrastructure toward the second problem.

Agents Make Concurrency a First-Class Requirement

A chatbot can serve one user request at a time.

An enterprise agent platform may run many agents simultaneously.

Each agent may generate several model calls.

That creates concurrency.

The accelerator has to keep many streams of work moving without wasting capacity.

Memory capacity, scheduling and token throughput all become important.

Crescent Island is designed around that environment.

The goal is not one spectacular benchmark run.

It is keeping many inference jobs moving efficiently across a shared accelerator.

Wildcat Lake Moves the Same Idea to the Edge

Wildcat Lake sits at the other end of the stack.

Intel launched it as Core Series 3 for price-sensitive laptops and intelligent edge platforms.

The chip combines x86 CPU cores, Xe3 graphics with XMX acceleration and an NPU rated at up to 17 TOPS.

That is far smaller than datacenter AI hardware.

It does not need to compete with it.

The point is to keep selected AI work close to the user or machine while larger work can move to cloud or enterprise infrastructure.

Two Performance Cores and Four Efficiency Cores Show the Target Market

Intel lists Wildcat Lake with two performance cores and four efficiency cores.

This is not a flagship workstation design.

It is meant to bring a right-sized AI platform to more affordable systems.

That matters strategically.

AI adoption becomes much larger when local acceleration stops being limited to premium devices.

A smaller chip can put hybrid AI into mainstream laptops, embedded systems and edge platforms where cost matters more than maximum compute.

The 17-TOPS NPU Is for Hybrid AI, Not Frontier Models

Intel says Wildcat Lake’s NPU provides up to 17 TOPS.

That is useful for supported local workloads.

Noise suppression.

Vision.

Small language models.

Local classification.

Background AI features.

It is not enough to treat the device as a replacement for datacenter inference.

Intel uses the phrase Hybrid AI for a reason.

Some tasks stay local.

Others move outward.

The edge becomes one tier in a larger system.

Wildcat Lake Is Intel’s First Processor to Use UCIe

Intel says Wildcat Lake marks the first use of UCIe in an Intel processor.

That is notable because UCIe is not only a datacenter technology.

Intel is bringing chiplet-style modular packaging into mainstream client hardware.

The company says the approach enables more cost-effective multi-chip package designs.

If that strategy scales, future processors can mix compute blocks more flexibly across product tiers instead of redesigning every chip from scratch.

The Same Packaging Idea Runs From Server to Client

This is one of the most coherent parts of Intel’s roadmap.

Diamond Rapids uses UCIe-S and advanced 3D packaging.

Wildcat Lake also uses UCIe.

The products are very different.

The packaging philosophy is shared.

Build systems from modular pieces.

Combine general-purpose compute with specialized acceleration.

Scale the architecture up or down depending on the market.

Agentic AI gives Intel a narrative that connects those pieces across the entire product line.

18A Connects the Three Products to Intel Foundry

Intel says the architectures are underpinned by its Foundry technology, including the Intel 18A process family.

Diamond Rapids uses the performance- and power-enhanced 18A-P variant.

Wildcat Lake is built on Intel 18A.

That gives the roadmap another strategic layer.

Intel is not only trying to sell processors.

It is trying to prove that its process technology and advanced packaging can support competitive AI hardware across servers, accelerators and client devices.

The AI roadmap doubles as a manufacturing roadmap.

This Is Intel’s Version of Heterogeneous AI

The most important word in Intel’s strategy is heterogeneous.

Different compute engines do different work.

CPU.

GPU.

NPU.

Real-time or edge processing.

Specialized matrix acceleration.

Open chiplet interconnect.

Fast memory and I/O.

The agentic AI story gives all of those components a role.

Instead of asking which chip wins, Intel is asking how many different processors can cooperate inside one workload.

That is a system-level argument.

Agentic AI Makes Orchestration More Expensive Than Chatbots

One chatbot prompt may create one inference job.

An agent can create a chain.

Plan.

Search.

Call a model.

Use a tool.

Read the result.

Call another model.

Verify.

Retry.

Log.

Escalate.

Each step creates compute and data movement.

At enterprise scale, the surrounding work can become substantial.

That is why Intel keeps emphasizing orchestration.

The more autonomous the workflow becomes, the more important the infrastructure around the model becomes.

Inference Cost Can Become the Real Scaling Limit

Frontier model training gets the headlines.

Enterprise AI spending often happens during inference.

Every production request consumes resources.

Agents multiply that consumption.

If a workflow triggers five or ten model calls instead of one, token economics change quickly.

Crescent Island is Intel’s answer to that problem.

The company is explicitly talking about token throughput, power and cooling rather than only peak compute.

That is a sign the market is moving from experimentation toward operational economics.

Edge Execution Reduces Latency and Data Movement

Wildcat Lake covers the other side of the cost problem.

Not every task should leave the device.

Sending audio, images or sensor data to the cloud creates bandwidth and latency.

A local NPU can handle selected tasks immediately.

The device can then send only the result or escalate the difficult work.

That can reduce network traffic and improve responsiveness.

Hybrid AI is therefore partly an economic architecture.

Use expensive centralized compute only when the workload needs it.

Intel Is Also Preparing an Agentic-AI Infrastructure Narrative Beyond Hot Chips

Intel’s August 26 AI Infra Summit preview extends the same strategy.

The company says agentic infrastructure is moving toward heterogeneous compute, disaggregated inference and intelligent orchestration.

It also highlights hybrid architectures that combine cloud-scale reasoning with edge-based inference and control.

That language is consistent with the Hot Chips hardware roadmap.

Diamond Rapids, Crescent Island and Wildcat Lake are not isolated product announcements.

They are components inside the same infrastructure thesis.

The Open-Standards Message Is Strategic

Intel repeatedly emphasizes UCIe and open infrastructure.

That is not accidental.

The AI accelerator market is dominated by tightly integrated hardware and software stacks.

Intel’s alternative argument is interoperability.

Open chiplet standards.

Heterogeneous compute.

Multiple accelerators.

Software that can span architectures.

Whether the ecosystem delivers that smoothly is another question.

But the strategic goal is clear: make openness part of the reason enterprises consider Intel hardware.

None of This Proves Intel Has Won the AI Hardware Race

A roadmap is not a market result.

Intel has announced specifications and architecture.

Customers still have to deploy the products.

Software has to mature.

Performance has to hold up under independent testing.

Crescent Island’s inference economics have to compete with established accelerators.

Diamond Rapids has to prove its server advantages in real workloads.

Wildcat Lake has to deliver useful local AI in cost-sensitive systems.

The Hot Chips announcement shows direction, not victory.

The Specifications Need to Stay Product-Specific

It is easy to combine the numbers into one misleading picture.

256 cores belongs to Diamond Rapids.

480 GB LPDDR5X and 350 watts belong to Crescent Island.

17 TOPS belongs to Wildcat Lake’s NPU.

They are not one chip.

They are three different product classes.

Keeping those boundaries clear matters because Intel’s whole argument depends on specialization.

The strategy only makes sense if each architecture is solving a different part of the agentic workload.

What Intel Has Actually Confirmed

Intel presented Diamond Rapids, Crescent Island and Wildcat Lake as complementary architectures for agentic AI.

Diamond Rapids is a next-generation Xeon design with up to 256 cores, 1.28 GB LLC, 16 memory channels at up to 12,800 MT/s, and 128 lanes of PCIe Gen 6 with CXL 3.0.

Crescent Island is an inference-oriented datacenter GPU with 32 Xe cores, 256 XMX engines, up to 480 GB LPDDR5X and a 350-watt air-cooled PCIe design.

Wildcat Lake is a Core Series 3 SoC with two performance cores, four efficiency cores, Xe3 graphics with XMX, and an NPU rated at up to 17 TOPS.

Intel also says the portfolio uses Intel 18A technologies, advanced Foveros packaging and UCIe.

What We Should Not Claim

We should not say Diamond Rapids alone runs the whole agentic AI stack.

Intel positions it as the orchestration and general-compute foundation.

We should not say Crescent Island is a training flagship.

Intel is positioning it for inference economics.

We should not compare 480 GB LPDDR5X directly with HBM accelerators without discussing different bandwidth and power trade-offs.

We should not say 17 TOPS means Wildcat Lake can run frontier models locally.

We should not say Intel has proven better performance than competing platforms without independent benchmarks.

And we should not treat Hot Chips specifications as evidence of broad production deployment today.

The Bigger Shift Is That AI Hardware Is Becoming a System, Not a Chip

The first AI hardware race was easy to describe.

Who has the fastest accelerator?

Agentic AI makes the answer more complicated.

Agents need CPU orchestration.

Accelerator inference.

Memory.

Networking.

I/O.

Local execution.

Cloud execution.

Security.

Scheduling.

The winning platform may not be the one with the largest single number.

It may be the one that moves work efficiently across the entire stack.

Intel’s Hot Chips roadmap is a bet on exactly that future.

Diamond Rapids handles the orchestration.

Crescent Island handles large-scale inference.

Wildcat Lake handles the edge.

Three architectures.

One agent workflow.

Qualcomm’s new Dragonwing Q-2390 and IQ-2390 processors are not built to run giant frontier models. Their AI accelerator is rated at just 1.1 TOPS. That is exactly why they are interesting. These chips combine modest on-device AI, machine vision, graphics, real-time control, deterministic networking and broad I/O in compact systems designed for kiosks, gateways, HMIs, PLCs, cameras, building automation and other devices that need to react locally. The industrial IQ-2390 adds ECC memory protection, a -30°C to 115°C operating range and long product-lifecycle support. The story is not about putting a chatbot in every machine. It is about pushing small, specialized AI into the control layer where latency, reliability, cost and connectivity matter more than model size.

This Is AI Hardware for Machines Nobody Calls AI Devices

When people hear “AI chip,” they usually imagine one of two things.

A giant accelerator inside a datacenter.

Or an NPU inside a premium phone or laptop.

Qualcomm’s Dragonwing Q-2390 and IQ-2390 target a less glamorous part of the market.

Industrial controllers.

Machine-vision systems.

Building-management equipment.

Retail terminals.

Gateways.

Kiosks.

Smart appliances.

PLC and CNC systems.

The chips are designed for devices that increasingly need to perceive, classify, decide and react locally without becoming expensive AI computers.

That makes them a useful signal of where edge AI may spread next.

Qualcomm Announced Both Chips Ahead of IFA 2026

Qualcomm introduced the Q-2390 and IQ-2390 on September 1, 2026.

The commercial Q-2390 joins the Dragonwing Q2 Series.

The industrial IQ-2390 becomes the first processor in the new Dragonwing IQ2 Series.

They share a common architectural idea.

Put application compute, graphics, machine vision, AI acceleration, a real-time microcontroller and extensive connectivity into one compact platform.

Then let device makers replace several separate chips and controllers with one more integrated design.

1.1 TOPS Sounds Tiny — and That Is the Point

Both product pages list a Qualcomm Hexagon NPU rated at 1.1 TOPS.

Compared with an 80 TOPS laptop NPU or a high-end datacenter accelerator, that number looks small.

It is.

These processors are not designed for local frontier language models.

They are aimed at smaller inference jobs.

Object detection.

Image classification.

Anomaly detection.

Sensor interpretation.

Simple predictive models.

Local automation decisions.

In many embedded systems, 1.1 TOPS can be useful if the workload is narrow and the software is optimized.

The target is practical intelligence per watt and per dollar, not benchmark supremacy.

The CPU Is a Four-Core Arm Design

Qualcomm’s detailed product pages list one Arm Cortex-A78 core and three Cortex-A55 cores running up to 1.9 GHz.

Qualcomm also markets the CPU under its Kryo branding.

That mix follows a familiar heterogeneous pattern.

The stronger A78 core can handle more demanding application work.

The A55 cores can run lighter tasks more efficiently.

For an industrial controller, that balance is more useful than simply maximizing core count.

The system may need to run Linux, a user interface, networking, device logic and AI inference at the same time.

The RISC-V MCU Is More Important Than It Looks

Both platforms include a real-time RISC-V microcontroller.

That detail separates industrial edge systems from ordinary consumer computing.

A language model or Linux application can tolerate variable timing.

A control loop often cannot.

A motor command may need to happen at a predictable interval.

A sensor must be sampled on schedule.

A safety state may need an immediate response.

The real-time MCU gives developers a deterministic execution resource alongside the larger application processor.

That allows AI and conventional real-time control to live inside the same platform without pretending they have identical timing requirements.

AI Should Not Sit Inside Every Control Loop

One of the biggest mistakes in physical AI is assuming a neural network should directly control everything.

Industrial systems are built around predictability.

A machine may use AI to classify an object or detect an abnormal vibration.

That result can then feed a deterministic state machine or control routine.

The AI contributes perception.

The real-time controller contributes timing and repeatability.

Dragonwing’s architecture reflects that separation.

The processor does not force every machine function through an AI model.

It gives designers several compute domains and lets them assign the right job to each.

Machine Vision Is a Natural Fit

Factories already use cameras for inspection.

Check whether a component is present.

Read a code.

Measure alignment.

Detect surface defects.

Confirm that a robotic process completed correctly.

Historically, some of that processing required a separate industrial PC or a specialized vision controller.

The Q-2390 and IQ-2390 combine camera support, an image signal processor, GPU resources and an NPU.

That creates a path toward smaller vision systems where more of the pipeline runs directly at the machine.

Local Vision Reduces the Need to Stream Everything to the Cloud

A factory camera can generate a large amount of data.

Sending every frame to a datacenter is expensive.

It adds latency.

It consumes network bandwidth.

It may create privacy or sovereignty concerns.

A local edge processor can analyze the stream near the camera.

Only an alert, classification, measurement or selected image needs to leave the device.

This is one of the simplest economic arguments for industrial edge AI.

Process the high-volume raw data locally.

Transmit the smaller result.

The IQ-2390 Is Built for Harsher Environments

The industrial IQ-2390 is not merely the commercial chip with a different name.

Qualcomm lists an operating range from -30°C to 115°C.

The platform uses industrial-grade packaging and includes ECC memory protection.

It is designed for systems exposed to heat, cold, vibration and shock.

Those requirements matter because factory and infrastructure equipment can operate for years in places where consumer electronics would never be expected to survive.

Reliability becomes a product feature, not a background specification.

ECC Memory Is a Quiet Industrial Requirement

ECC stands for error-correcting code.

Memory errors are rare.

In industrial systems expected to operate continuously for years, rare errors eventually matter.

ECC can detect and correct certain memory faults before they become corrupted data or unpredictable software behavior.

That is especially important when one platform combines AI inference, control logic, networking and machine vision.

A rugged AI edge system has to be more than fast.

It has to remain trustworthy over long operating periods.

Ten-Year Longevity Matters More Than Annual Chip Upgrades

Consumer devices are replaced quickly.

Industrial machines are not.

A factory controller may remain in service for a decade.

A building-management platform can stay installed even longer.

Changing the processor halfway through a product lifecycle can force expensive redesign and certification work.

Qualcomm says the IQ-2390 is designed around long product-lifecycle support, with current materials pointing to roughly a decade of availability and ecosystem support.

For industrial customers, that can be more important than receiving a faster chip next year.

Industrial AI Has a Different Upgrade Cycle

The AI industry moves quickly.

Factories move carefully.

That creates tension.

A model can improve every month.

Industrial hardware may be frozen for years.

The practical solution is not to replace every controller whenever a better model appears.

It is to design hardware with enough headroom, stable software support and standard frameworks so models can evolve within the existing platform.

Dragonwing’s value will depend heavily on whether Qualcomm and its ecosystem can sustain that software path over the promised product lifecycle.

Dual Gigabit Ethernet With TSN Is a Major Clue

The IQ-2390 includes two Gigabit Ethernet interfaces with Time-Sensitive Networking support.

TSN is designed to make Ethernet traffic more deterministic.

Ordinary Ethernet is excellent for moving data.

Industrial automation often needs more predictable timing.

A control packet arriving late can matter.

TSN adds mechanisms for scheduling and prioritizing time-critical traffic.

That means the same platform can participate in modern industrial networks while still handling AI and vision workloads locally.

Deterministic Networking Connects AI to Real Automation

An AI model can identify a defective part.

That information becomes useful only when another system can react.

Stop a conveyor.

Reject the item.

Alert an operator.

Adjust a process.

The network between perception and action therefore matters.

By including TSN-capable Ethernet, Qualcomm is targeting devices that sit inside real operational technology rather than simply collecting data at the edge.

The chip is meant to participate in the control environment.

The Q-2390 Targets Less Harsh Commercial Devices

The Q-2390 has a broader commercial and consumer focus.

Qualcomm lists applications such as retail point-of-sale systems, kiosks, access-control devices, smart appliances, smart agriculture, home robots, fitness equipment and enterprise terminals.

These products may need cameras, displays, connectivity and some local AI.

They generally do not need the same industrial temperature envelope or rugged lifecycle features as a factory controller.

The shared architecture lets Qualcomm cover both markets without treating them as identical.

The Q-2390M Adds Integrated Cellular

Qualcomm’s Q-2390 product page describes a Q-2390M variant with integrated LTE Cat 4 data connectivity.

That matters for systems installed where wired networking is inconvenient or unavailable.

A remote terminal.

A kiosk.

A field device.

A smart-agriculture controller.

Cellular connectivity can turn an edge processor into a self-contained connected product without requiring another modem subsystem.

That reduces board complexity and the number of components an OEM has to integrate.

The Platform Also Supports Wi-Fi, Bluetooth and Wired Expansion

The commercial Q-2390 supports a broad connectivity mix, including Wi-Fi, Bluetooth, dual Ethernet, PCIe, USB and serial interfaces.

The exact implementation can vary by product design and companion components.

That flexibility matters because IoT devices are rarely defined by one connection type.

A machine may use Ethernet for plant networking.

USB for peripherals.

PCIe for expansion.

Serial interfaces for legacy sensors.

Wireless connectivity for setup or remote management.

Integration is the product.

Reducing BOM Cost Is Part of the AI Strategy

Qualcomm repeatedly emphasizes bill of materials, board area and integration complexity.

That may sound like ordinary chip marketing.

It is actually central to edge AI adoption.

If adding AI requires a separate processor, vision chip, real-time controller, modem and networking device, the feature becomes expensive.

If one SoC can absorb several of those roles, AI becomes practical in lower-cost products.

The edge-AI market expands not only when models become smarter but when the hardware around them becomes cheaper to integrate.

A Smaller Board Can Create a New Product Category

Some machines have plenty of space for an industrial PC.

Others do not.

Door controllers.

Compact cameras.

HVAC systems.

Small HMIs.

Printers.

Scanners.

Embedded gateways.

A highly integrated processor lets developers add local intelligence where a large computer would be impractical.

That is why Qualcomm describes the IQ2 family around space-constrained, cost-constrained and power-constrained devices.

The AI opportunity exists precisely because the hardware cannot be large.

Android, Linux and Zephyr Cover Different Parts of the Stack

Qualcomm lists support for Android, Yocto Linux, Ubuntu and Zephyr.

Those operating systems serve different product styles.

Android can support rich touch interfaces.

Ubuntu can provide a familiar development environment.

Yocto gives embedded teams more control over a customized Linux image.

Zephyr targets resource-conscious real-time and embedded workloads.

Supporting several operating environments gives OEMs flexibility.

It also acknowledges that industrial AI is not one software stack.

The Hard Part Will Be Software, Not the NPU Number

A 1.1 TOPS NPU is useful only if the application can use it.

Developers need model-conversion tools.

Supported operators.

Runtime libraries.

Camera pipelines.

Drivers.

Debugging tools.

Board-support packages.

Long-term security updates.

The success of Q-2390 and IQ-2390 will therefore depend on the development ecosystem around the silicon.

Industrial companies do not buy TOPS.

They buy a path from prototype to a product they can ship and support.

Qualcomm Is Explicitly Selling the Path From Evaluation to Production

The IQ-2390 product page emphasizes BSPs, tools, evaluation hardware and documentation.

That is exactly what embedded teams need.

A chip can look impressive on a specification sheet and still fail commercially if integration takes too long.

Qualcomm is positioning Dragonwing around reducing that development burden.

The company says customer engagement has already begun through an early-access program.

Evaluation kits for both platforms are expected in early 2027.

Early Access Means These Are Not Yet Everywhere

The launch should not be interpreted as mass deployment across factories today.

Qualcomm says the Q-2390 and IQ-2390 are in early access.

Evaluation kits are expected to become available in early 2027.

Customers and module vendors can begin development before broad availability.

That makes September 2026 the start of the product cycle, not proof that industrial AI systems using these chips are already widespread.

The Industrial Edge Is Where Cloud Economics Change

Cloud AI works extremely well when a device has reliable bandwidth and latency is not critical.

Industrial systems often violate both assumptions.

Some factories isolate operational networks.

Some remote infrastructure has limited connectivity.

Some decisions have to happen in milliseconds.

Some video streams are too expensive to upload continuously.

Some data cannot leave the site.

Those constraints create a natural market for edge inference even when the local model is much smaller than a cloud model.

A Small Model Can Be More Valuable Than a Giant One

A factory does not need a frontier language model to decide whether a bottle cap is missing.

It may need a tiny vision model that runs reliably millions of times.

A building controller does not need a giant reasoning model to detect an abnormal sensor pattern.

It may need a compact anomaly detector that works offline.

AI value depends on the task.

The model that fits the machine, latency budget and power envelope can be more useful than the model with the largest parameter count.

Industrial AI Is Often Invisible to the User

A consumer notices an AI chatbot.

They may never notice an AI-enabled industrial controller.

The system simply works better.

A camera catches defects earlier.

A motor failure is predicted.

A building adjusts energy use.

A gateway filters sensor data before sending it upstream.

A machine reacts locally when connectivity disappears.

That invisible behavior may become one of the largest deployments of practical AI.

Not because the model is impressive to watch.

Because the system quietly makes better decisions.

Qualcomm Is Positioning Dragonwing as the Industrial Counterpart to Its Consumer AI Push

Qualcomm already sells AI-capable platforms into phones, PCs, automotive systems and higher-end industrial hardware.

Dragonwing extends the same edge-computing strategy downward into more constrained devices.

The company’s message is consistent.

Do more inference near the user or machine.

Reduce dependency on the cloud.

Combine connectivity and compute.

Use specialized accelerators for efficiency.

The Q-2390 and IQ-2390 show what that strategy looks like when cost and industrial reliability become the dominant constraints.

What Qualcomm Has Actually Confirmed

Qualcomm announced the Dragonwing Q-2390 and IQ-2390 on September 1, 2026.

Both platforms combine a quad-core CPU, Adreno 704 GPU, Hexagon AI acceleration and a real-time RISC-V microcontroller.

The detailed product pages list one Cortex-A78 core and three Cortex-A55 cores up to 1.9 GHz and an NPU rated at 1.1 TOPS.

The Q-2390 targets commercial, enterprise and consumer IoT devices.

The IQ-2390 targets industrial HMI, machine vision, gateways, PLC/CNC, building automation and related systems.

The IQ-2390 includes ECC memory protection, dual Gigabit Ethernet with TSN and an operating range from -30°C to 115°C.

Qualcomm is running an early-access program and expects evaluation kits in early 2027.

What We Should Not Claim

We should not say these processors run frontier language models.

Their NPU is rated at 1.1 TOPS and the target workloads are much smaller.

We should not say 1.1 TOPS guarantees a specific model speed.

Software, model architecture and memory all matter.

We should not say every Q-2390 product includes cellular connectivity.

Qualcomm specifically describes the Q-2390M variant as adding integrated LTE Cat 4.

We should not say the chips are already broadly shipping in finished industrial products.

They are currently in early access, with evaluation kits expected in early 2027.

And we should not say AI replaces deterministic industrial control.

The platform itself includes a real-time MCU because predictable control still matters.

The Bigger Shift Is AI Moving Into the Control Layer

The most visible AI systems live in datacenters and apps.

The next wave may be harder to see.

Inside a camera.

Inside an HMI.

Inside a building controller.

Inside a gateway.

Inside a factory machine.

Those devices do not need enormous models.

They need enough intelligence to perceive local conditions and act quickly without depending on a remote server.

Q-2390 and IQ-2390 are not exciting because they are powerful compared with an AI PC.

They are exciting because they are small enough, integrated enough and industrial enough to put AI in places where a big AI computer would never fit.

That is how intelligence becomes infrastructure.

AI Is Moving From One Location to Several

For a long time, most consumer AI had one obvious home.

The model ran in a data center. The phone or laptop sent a request. The server produced the result. The device displayed it.

That architecture is still important, but it is no longer the only one.

Modern laptops, phones and tablets now include CPUs, GPUs and NPUs capable of running useful AI models directly on the device. At the same time, cloud infrastructure continues to scale into larger models, longer context windows and more demanding multimodal workloads.

The result is a split computing stack.

Some AI can stay close to the user. Some AI can run remotely. The application can decide which execution path fits the job.

Microsoft reflects this directly in its Windows AI guidance, which includes local APIs, Foundry Local, Windows ML and cloud services as parts of the same broader development environment. Apple is moving in the same direction through its Foundation Models framework, on-device system models and Private Cloud Compute.

That changes the question.

Local AI and cloud AI are no longer two competing ideas.

They are becoming two places where the same product can think.

Local AI Brings the Model Closer to the User

Local AI begins with proximity.

The model is running on the same device as the file, camera, microphone, application or user interaction it is working with.

That can make certain AI features feel immediate.

OCR can read text from a local image. A search tool can index files on the machine. A camera feature can process a live feed. A writing tool can summarize or transform text. A small assistant can answer from information already available on the device.

Microsoft’s Windows AI APIs are designed around this kind of execution on supported hardware. The platform exposes local capabilities such as OCR, image description, summarization and access to local models. Apple’s Foundation Models framework likewise gives developers access to an on-device model for tasks such as summarization, entity extraction, refinement and structured generation.

The common idea is simple.

The AI feature can use the machine itself as part of the inference infrastructure.

The laptop is no longer only a window into AI.

It can be one of the places where the AI actually runs.

Low-Latency Tasks Fit Naturally on the Device

Some AI interactions happen often enough that speed becomes part of the product experience.

A camera effect is active continuously. A search box may respond dozens of times in one session. OCR may run whenever a document appears. A text tool may make small changes repeatedly while the user is writing.

Local execution gives these tasks a short path.

The application can send the input directly to the local model without waiting for a remote request to travel across the network first.

That is why many on-device AI features are small, frequent and interactive.

Microsoft has been building Windows AI around this pattern on supported PCs. Apple uses the same direction with on-device Foundation Models. The model can sit close to the application and respond as part of the normal interface.

This changes how AI can be designed.

Instead of saving AI for a large command, developers can use it inside smaller moments throughout the application.

The model becomes part of typing, searching, viewing, organizing and navigating.

That is one of the most important effects of local AI: it makes AI easier to weave into the software itself.

Local Processing Creates a Strong Privacy Architecture

Local AI also creates a useful privacy model for supported workflows.

When inference happens entirely on the device, the input can remain on the machine for that part of the process.

Microsoft states that supported Windows AI APIs process data locally, and its Foundry Local documentation describes inference paths where inputs and outputs stay on the device. Apple’s on-device Foundation Models are designed around the same principle: supported generative work can happen on the user’s hardware.

That creates new possibilities for applications working with personal content.

A document assistant can process local notes. A photo tool can understand images already stored on the device. A search feature can index local files. A writing tool can work with text before any cloud request becomes part of the experience.

This architecture is useful because it gives developers another way to design privacy-conscious products.

The application can decide that some steps belong on the device by default.

Then larger cloud services can be added around those local capabilities when the product wants additional scale.

Privacy becomes part of system design, not only a policy written after the product is built.

Offline AI Makes the Device More Independent

A local model can also continue working when the network is not part of the moment.

That makes offline AI useful in travel, field work, aircraft, remote locations and any workflow where connectivity changes throughout the day.

Microsoft describes Foundry Local as supporting local inference after the required model is available on the machine. Apple likewise exposes on-device Foundation Models for supported generative tasks.

The practical effect is easy to understand.

The user can open the application and keep using the supported AI feature even when the device is disconnected.

A local summarizer can work with a document. A classification model can organize content. OCR can continue reading text. A local assistant can keep working with information stored on the machine.

That is a meaningful change in how AI software behaves.

The application does not have to treat an internet connection as the beginning of every intelligent action.

Instead, the device can carry some of its own intelligence with it.

That makes AI feel more like a built-in computing capability and less like a remote service the machine has to reach before anything useful can happen.

Cloud AI Gives Applications Access to a Much Larger Compute Pool

Cloud AI has a different strength: scale.

A server platform can combine large accelerator fleets, high-capacity memory, fast networking and centralized model infrastructure. That gives applications access to models and workloads far beyond what a portable battery-powered device is designed to carry locally.

This becomes useful for long context, complex reasoning, large multimodal inputs, agentic workflows and other tasks that benefit from a larger compute envelope.

Microsoft’s local-versus-cloud guidance treats model size and complexity as major architectural inputs. Apple does the same in its Foundation Models work, pairing on-device models with Private Cloud Compute for workloads that can use larger server-side capability.

The cloud therefore expands the ceiling of the application.

The device can remain thin and portable while the product reaches infrastructure that may contain far more memory and compute than any laptop or phone.

This is why cloud AI remains central even as local AI improves.

The local device adds immediacy and proximity.

The cloud adds scale.

Longer Context Windows Create a Different Kind of AI Experience

One of the easiest ways to see the difference between device-scale and server-scale AI is context.

Context determines how much information a model can work with inside one interaction.

Apple’s WWDC26 developer material provides a concrete example inside its own stack. Apple describes an on-device system model with roughly a 4K context window and a Private Cloud Compute model with roughly 32K context in that specific framework.

Those figures are Apple-specific, but the architecture is useful to understand.

The on-device model is designed to be available locally. The server model can draw on a larger resource envelope and work with substantially more context.

That difference changes the kinds of experiences an application can create.

A local model can handle short transformations, extraction and immediate assistance. A larger cloud model can take in longer documents, larger conversations or more complex multimodal material.

The product does not have to choose one forever.

It can use the smaller local model for frequent everyday interactions, then move a larger request to the cloud when the task expands.

That is where the two execution paths begin to look complementary rather than separate.

Centralized Cloud Models Can Evolve Across an Entire Service

Cloud AI also gives providers a powerful deployment model.

The model lives in centralized infrastructure.

When the provider updates that model, expands the serving stack or adds new capabilities, the change can become available across the service without moving the full model weights onto every user’s device.

That creates a fast path for platform evolution.

A cloud service can scale capacity, introduce a newer model family, extend context, add tools or improve multimodal processing from the server side.

Developers can then expose those capabilities through the same application interface.

This is one reason cloud AI is especially useful for products that serve many users and need access to large shared infrastructure.

The application can remain relatively lightweight while the provider manages the deeper compute environment centrally.

Local AI and cloud AI therefore distribute responsibility differently.

The local path places more intelligence directly inside the device.

The cloud path places more intelligence inside the service.

Modern applications can use both.

Private Cloud Compute Shows That Cloud AI Can Have a Purpose-Built Privacy Architecture

Cloud AI does not have to mean one generic server model.

Apple’s Private Cloud Compute shows how a provider can build a remote AI architecture around specific privacy and security requirements.

Apple documents PCC as infrastructure designed so user data sent for a request is used for the computation and is not retained after the response, with additional verification and security mechanisms around the system.

That makes PCC an important example because it widens the architecture choices available to developers and platform designers.

An application can keep supported work fully on-device. It can use a purpose-built private cloud architecture for larger requests. It can also connect to other server models where the product design calls for them.

The important shift is choice.

Privacy-sensitive design is not limited to one execution location.

It can influence how the local path is built and how the remote path is built.

That gives modern AI systems more flexibility than the old binary of ‘device equals private’ and ‘cloud equals remote.’

The architecture itself can carry the privacy model.

Hybrid AI Lets the Application Route Work to the Right Place

The most interesting architecture is often the one that uses both locations.

A hybrid AI application can keep lightweight work on the device and send larger work to server infrastructure when the task expands.

The local side might handle OCR, indexing, classification, short summarization, image understanding or quick text generation. The cloud side might handle longer context, deeper reasoning, larger multimodal inputs or an agentic workflow that needs more compute.

Apple is moving directly toward this model through its Foundation Models framework and Private Cloud Compute. Microsoft is doing the same across Windows AI, Foundry Local, Windows ML and cloud services.

This creates a routing layer inside the product.

The user asks for one thing.

The application decides where each part should run.

Some work can happen immediately on the device. More demanding work can move to the server. The result comes back into the same interface.

That is a major architectural change.

AI products are beginning to manage compute location the way modern systems already manage storage, networking and graphics resources.

The location becomes part of the software design.

AI PCs Make Local Execution a Larger Product Category

Local AI is becoming more important because consumer hardware is changing underneath it.

AI PCs increasingly include NPUs designed for neural-network workloads. CPUs and GPUs continue to improve. Memory capacity is rising. Operating systems are exposing local AI APIs. Model developers are creating smaller models designed to run efficiently on endpoint hardware.

Those changes reinforce one another.

Better hardware makes more local AI possible.

More local AI gives developers a reason to target the hardware.

More software gives users a reason to care about the NPU, GPU and local model stack inside the machine.

Microsoft’s Windows AI work is part of that cycle. Apple’s Foundation Models are another example on its platforms.

The device is becoming an AI execution target in its own right.

That does not shrink the role of cloud AI.

It expands the total AI system.

Instead of one remote model doing everything, the product gains another compute layer close to the user.

The Future AI Stack Is Local, Cloud and Everything Between Them

The long-term shift is not difficult to see.

AI is becoming distributed.

The phone can run a model. The laptop can run a model. The operating system can expose local AI services. The cloud can run larger models. Private server architectures can handle sensitive remote workloads. Applications can route between those places as the task changes.

That gives developers more ways to design the experience.

Fast, repetitive and personal interactions can stay close to the device. Offline features can remain available while the network disappears. Larger reasoning tasks can use server-scale compute. Long context can move to infrastructure with more memory. The same application can combine all of those paths without turning them into separate products.

Microsoft and Apple are already building software frameworks around this model.

That is the important signal.

Local AI is no longer a small alternative to cloud AI.

Cloud AI is no longer the only place where intelligence lives.

They are becoming layers of the same computing stack.

The device thinks.

The cloud thinks.

The application decides how to connect them.

That is the upgrade.