Local inference is evolving into a distinct market segment that spans from portable devices to team stations. Manufacturers are marketing it as AI without usage meters, while subscription services are shifting a portion of their computations to this domain. For CIOs, the permanent agent paradigm transforms spending tokens into a capital investment.
On October 7th in San Francisco, Microsoft brought together Jensen Huang, CEO of Nvidia, and its own CEO Satya Nadella to unveil the first Windows PCs equipped with the new RTX Spark chip. Laptops from Acer, Asus, Dell, HP, Lenovo, Microsoft, MSI, and Gigabyte, available for preorder since the event, will officially hit the market on October 16th, with compact formats expected to follow in November. Microsoft is also introducing the Surface Laptop Ultra, its own device built on this chip, although pricing details remain under wraps. Nadella emphasized the goal: to make the workplace the most secure environment for running AI agents.
To operate these agents, the chip combines a Blackwell graphics processor with up to 6,144 cores, a Grace processor with up to 20 cores, and up to 128GB of unified memory, achieving one petaflop of computing power at FP4 precision. Nvidia showcases this chip running a model with 125 billion parameters, claiming it rivals many cloud models, functioning without usage meters or data transmission outside the machine. First announced at Computex in June, this platform is now entering its commercial phase. Pavan Davuluri, Microsoft’s executive vice president for Windows and devices, highlighted the potential to run models on portable devices that traditional machines cannot accommodate.
A Memory-Segmented Market
Manufacturers are now segmenting local inference by price and memory ranges, each associated with specific use cases. Nvidia is set to launch the DGX Spark with 64GB of memory starting October 23rd, priced from $4,999. This machine can run models with up to 100 billion parameters, and coupling two units allows it to handle 200 billion. It comes with Nvidia’s software stack and common execution environments for open models, including Ollama, vLLM, and LM Studio. At the top of Nvidia’s lineup, the pre-release DGX Station for Windows features 748GB of memory and 20 petaflops for handling models approaching one trillion parameters, aimed at labs and enterprises. Previously exclusive to Linux, this transition to Windows caters to the large majority of Fortune 500 companies, according to Nvidia.
AMD is carving its niche with its x86 architecture. Its Ryzen AI Halo features 128GB of memory for models with up to 200 billion parameters, while the upcoming Ryzen AI Max PRO 400 series, expected by year’s end, will offer 192GB and support for 300 billion parameters. AMD has expanded its partnership with Hugging Face to provide open models optimized for its hardware. This tiered approach creates a comprehensive market, ranging from individual laptops to team stations and developer chassis, all distinct from conventional hardware upgrades.
The Permanent Agent: Transforming Token Counting
The economic argument targets continuous usage scenarios, where agents operate autonomously without user intervention. Nvidia cites the DGX Spark’s capabilities for running always-on agents and offloading inference tasks from everyday PCs. On October 7th, Microsoft made its Execution Containers generally available, allowing agents to operate sustainably in the background at the system level. An always-active agent continuously consumes tokens, and local computing transforms this variable expense into a depreciable investment.
AI-as-a-service providers are also adapting to the workstation model. On PCs equipped with Ryzen AI Max processors, Perplexity’s Portable Computer handles local tasks that do not exhaust subscription inference credits, reserving high-end model reasoning for the cloud, according to AMD. Microsoft is promoting Windows as a hybrid intelligence platform, where Copilot utilizes local models to optimize computational usage. In September, AMD equipped this scenario with Tokenomics, a simulator that compares the costs of cloud, local, and hybrid models while excluding machine administration expenses.
Shifting Computation to the Client
This alternative to subscriptions also benefits subscription vendors, as Microsoft and Perplexity lighten their own computation costs by executing some queries on the local machine, while the client invests in hardware and memory. Clients gain two advantages: enhanced privacy, as data remains on the machine, and predictability, since expenses are no longer tied to query volume.
However, models that process queries on local machines require careful consideration regarding their source. Nvidia’s prominent model with 125 billion parameters belongs to the Qwen family, created by China’s Alibaba and released under open weight. Thus, choosing the model installed on the machine is as significant as selecting the hardware itself, as companies must be aware of its origin, licensing, and update cadence.
A Segment to Govern Before Deployment
This new segment addresses specific needs for CIOs, such as data that must remain within the company, agents that run uninterrupted, or developers who wish to prototype without incurring a fee per request. Windows facilitates the confinement and identification of agents, and AMD is exploring supervision in collaboration with Cisco, a partnership announced in July that is still in exploratory stages.
CIOs will need to make informed decisions before the machine reaches the end user, selecting authorized models, their sources, and the tasks assigned to local agents. They will also need to reassess the cost-effectiveness based on their own assumptions, including administration costs, since no neutral simulator currently compares local and cloud solutions.



