Local LLM Hardware Checklist: What Specs Matter

Running large language models on your own hardware has gone from a niche experiment to a business decision. Instead of sending every prompt off to a cloud API, more and more teams are choosing to run models like Llama, Mistral, Qwen and DeepSeek locally — keeping sensitive data in-house, cutting out per-token cloud bills, removing latency, and staying in full control of how and when their AI runs.

The catch? Local LLMs are hungry. Get the hardware wrong and you’ll be watching a spinner while your model crawls along at a few words per second — or worse, discover the model you wanted won’t load at all. Get it right and you’ve got a private, fast, always-available AI setup sitting under your desk.

This checklist walks through the four specs that actually matter — GPU, memory, storage and networking — plus the bits people forget, and how to match the kit to the job without overspending. If you’d rather skip the theory, our guide to AI PCs and NVIDIA DGX vs Apple M5 are useful companion reads.

1. GPU (and Unified Memory): The Component That Decides Everything

If you only optimise one thing, make it the GPU — or, on Apple Silicon and the newer AI desktops, the unified memory that does the same job. For LLMs, the number that matters most isn’t raw clock speed, it’s VRAM (video memory). VRAM determines which models you can run at all; bandwidth then determines how fast they run.

A rough rule of thumb for a 4-bit quantised model (the format most people use locally): budget around 0.6–0.75GB of VRAM per billion parameters, plus a few GB of headroom for the context window. In practice that means:

• A 7–8B model fits comfortably on a 12GB card
• A 13B model wants around 12–16GB
• A 30–34B model needs roughly 24GB
• A 70B model needs around 40–48GB, or a very large pool of unified memory

Push the context length up (long documents, big chat histories) and those numbers climb further, so always leave room.

This is where the hardware route really forks:

NVIDIA GPUs remain the default for anyone serious, thanks to the mature CUDA software ecosystem. A workstation like the HP ZBook Fury G1i pairs Intel Core Ultra processors with NVIDIA RTX PRO Blackwell graphics, which is plenty for running mid-sized models and light fine-tuning on the move.

Apple Silicon takes a different approach. Because the M-series chips share one big pool of unified memory between CPU and GPU, a machine like the Mac Studio with M3 Ultra can load surprisingly large models simply by specifying enough memory — no separate graphics card required.

Dedicated AI desktop “supercomputers” are the newest option. Compact systems like the Lenovo ThinkStation PGX and the HP ZGX Nano G1n are built on NVIDIA’s GB10 Grace Blackwell Superchip, offering up to 1,000 AI TOPS and 128GB of coherent memory in a box that sits on your desk. They’re purpose-made for local model development, fine-tuning and inference.

For a tailored balance of GPU, cores and memory, a custom-built PC or a proper mobile workstation is often the sweet spot.

2. Memory: Don’t Let System RAM Become the Bottleneck

Even when the GPU does the heavy lifting, system RAM still matters. It’s used to load models before they’re moved onto the GPU, to run the operating system and your other apps, and — crucially — to handle CPU offloading, where layers of a model that won’t fit in VRAM spill over into system memory. That’s what lets you run a model slightly bigger than your GPU can hold (albeit more slowly).

A sensible baseline: at least as much system RAM as you have VRAM, and ideally 1.5–2x. For most business use, 32GB is the realistic floor and 64–128GB gives you comfortable breathing room for larger models and heavier multitasking. Fast DDR5 helps if you’re leaning on the CPU.

On unified-memory machines (Apple Silicon, Grace Blackwell), this distinction largely disappears — the memory is the VRAM — which is exactly why those systems punch above their weight for local AI.

3. Storage: Model Files Are Enormous, So Plan for It

This is the spec people underestimate. Model files are big, and they multiply fast. A single 70B model in 4-bit is roughly 40GB; in full precision it’s closer to 140GB. Once your team starts collecting a few models, some fine-tuned variants and a dataset or two, you’re into terabyte territory quickly.

Two things matter here:

• Speed. Use an NVMe SSD, ideally PCIe 4.0 or 5.0. Loading a 40GB model from a fast NVMe drive versus an older SATA disk is the difference between a few seconds and an uncomfortable wait every time you switch models.
• Capacity. 1TB is the practical minimum; 2–4TB is far more comfortable if you’re experimenting regularly.

For a team, a shared network-attached storage (NAS) solution such as QNAP is worth considering, so everyone pulls from a single, well-organised model and dataset repository rather than duplicating hundreds of gigabytes across machines.

4. Networking: The One That Matters More for Teams

For a single local machine, networking is the least critical of the four — but it’s not nothing. You’ll appreciate a fast internet connection when downloading models (Hugging Face repositories run to tens or hundreds of gigabytes), and a solid local network if you’re pulling from that shared NAS.

Networking becomes genuinely important the moment you run a local inference server that colleagues connect to. If one workstation is serving a model to a whole team, 2.5GbE or 10GbE wired networking keeps responses snappy and avoids a bottleneck as more people query it at once. And remember: one of the main reasons to go local in the first place is keeping data on-premises, so your network design is about secure internal traffic, not round-trips to the cloud.

Don’t Forget: CPU, Power and Form Factor

A few supporting specs round things off. A modern multi-core CPU keeps the GPU fed and handles data prep and offloading. Power and cooling deserve real thought — high-end GPUs draw serious wattage and generate heat, so make sure the power supply has headroom and the case can breathe. 

And decide on form factor early: a desktop workstation for maximum power, a compact AI supercomputer for a tidy plug-and-play setup, or a mobile workstation if your AI work needs to travel. Our AI PCs breakdown is handy if you’re weighing up laptop tiers.

Matching the Hardware to the Job

• Getting started / smaller models (up to ~13B): a workstation laptop or desktop with a 12–16GB NVIDIA GPU, 32GB RAM and a 1TB NVMe drive.
• Serious local work / mid-to-large models: 24GB+ of VRAM (or a well-specced Apple Silicon machine), 64GB RAM, 2TB+ NVMe.
• Team-scale, fine-tuning and heavy inference: a dedicated AI supercomputer or multi-GPU workstation, 128GB memory, fast NVMe plus shared NAS, and 10GbE networking.

The Biggest Factor of All: Cost — & How to Handle It

Here’s the honest truth: capable local-LLM hardware is expensive. Top-tier GPUs, high-memory workstations and AI desktop supercomputers run into thousands of pounds — and this is fast-moving kit. Buy outright and you’re facing a large capital outlay, ongoing depreciation, and the near-certainty that something better lands within a year or two.

Lease with HardSoft

That’s exactly where leasing with HardSoft changes the maths. Instead of a painful one-off purchase, you spread the cost into predictable, tax-deductible monthly or weekly payments — protecting cash flow while you put the hardware to work. 

Our most popular Flexi-Lease even lets you upgrade to newer devices after just 12 months, so you’re never stuck on last year’s silicon in a field that reinvents itself constantly. Prefer the lowest monthly cost Pure Rental comes in around 30% cheaper. There’s a fuller breakdown in our buying vs leasing guide.

You also get the things that make an AI hardware project painless rather than stressful. HardSoft is a one-stop IT shop — we’re an official Apple partner, authorised across the major Windows brands, and we even build custom PCs to your exact spec, all delivered ready to run. 

In short: you choose the right GPU, memory, storage and networking — we make owning it affordable, supported and future-proof.

Ready to spec your local AI setup? Build your lease solution and get a customised quote, or browse our full range of workstations and leasing options to find the right fit for your team.