The GPU box, specified and commissioned properly.
Sized to the model you are actually going to run, not the benchmark number on the box. Sourced, built, burned in under real load and handed over with the runbook.
What this is
A GPU server for AI inference is a sizing problem before it is a purchasing problem. The number that matters is not a card's headline VRAM, it is the model's weights at the quantization level you intend to run, plus the KV cache overhead your context length actually needs, plus the room a second model or a batch of concurrent requests will want later. Get that arithmetic wrong and you either overpay for capacity nobody uses or discover the shortfall the week the model ships, when the fix is a second card you now have to fit and cool.
We spec, source and commission the box. Specification is the VRAM budget worked backwards from the workload, the CPU and RAM sized so they are not the bottleneck, and the storage fast enough that loading a multi-gigabyte checkpoint is not the slowest part of a restart. Sourcing means ordering from suppliers we actually have accounts with and building around what has a real lead time, rather than betting a project schedule on retail stock. Commissioning is the part a spec sheet cannot promise: the machine racked, cabled, powered from a circuit sized to sustained draw rather than nameplate TDP, and cooled well enough that the second card in the chassis is not idling in the first card's exhaust.
The failure mode we see most is a box that benchmarks fine in the first five minutes and throttles an hour into a real workload, because nobody ran it long enough to hit thermal steady state before calling it done. We do not call it done until it has. Multi-GPU adds a second trap: two or four consumer cards in one chassis usually means splitting a model across PCIe rather than NVLink, and the interconnect bandwidth, not the GPUs, ends up setting the ceiling on throughput. That gets decided at the spec stage, not discovered after the cards arrive.
Most of this work follows the assessment on our on-premise AI page, once a workload and a model are picked and the question becomes a specific box rather than a strategy. We also take it standalone: if you already know what you are running and just need the hardware side done right, that is this page.
What you get
Written hardware specification
GPU model and count, VRAM budget worked against the target model, quantization level and context length, CPU, RAM and storage sized so none of them becomes the bottleneck.
Sourced and procured hardware
Specified down to the part number and ordered against that spec, with the parts that carry a real lead time flagged before they become the reason the project slips.
Rack, cabling and power build-out
Wired to a circuit sized against sustained draw under load, not the number on the PSU box, with the facilities conversation had before the pallet arrives if the building needs it.
Cooling verified under sustained load
Airflow checked at thermal steady state, not at boot, so a second card is not quietly starving in the first one's exhaust.
Driver, CUDA and firmware stack, version-pinned
Installed and locked so an unattended kernel or driver update does not silently change clocks or throughput on a machine nobody is watching that day.
Out-of-band remote management
BMC or IPMI access configured, so a hung box gets power-cycled from a laptop instead of somebody driving to the rack.
Burn-in and acceptance test
Sustained inference load run long enough to reach thermal steady state, with throughput, temperatures and any throttling written down against the spec.
Handover documentation
Bill of materials, network and power diagram, access credentials, and the runbook for driver and firmware updates. You can operate it without us.
When this fits, and when it does not
A good fit
- You have picked a model, or narrowed it to two or three candidates, and need the VRAM and compute budget sized to it rather than to a generic recommendation.
- The data cannot leave the building: the same boundary that drives our on-premise AI work drives this one.
- You already tried a consumer GPU on a desk or a DIY build, and hit a wall: a tripped breaker, a throttling card, a chassis that cannot move enough air.
- The box has to run unattended in production, which means remote management and a driver stack that will not drift on its own.
- You proved a workload on a pilot node and are scaling it, and want the machine specified and built to a written spec rather than assembled from whatever a retail listing had in stock.
Not a good fit
- You want one GPU in a tower for personal experimentation or a hobby project. A local system builder is cheaper and this is not a reason to involve us.
- The workload is low and elastic and a cloud API's per-token cost has not crossed the cost of owning a machine. We will tell you that in the assessment rather than sell you a box anyway.
- You need a multi-node training cluster with an InfiniBand fabric between nodes. That is HPC-cluster integration, a different discipline, and we will say so rather than learn it on your budget.
- You want us to rent or colocate a GPU for you. We specify and commission hardware on premises you control, we are not a cloud or colocation provider.
- There is no dedicated circuit or cooling for the box and no willingness to have that conversation with facilities. The electrical and thermal reality has to be dealt with, not wished away.
How it runs
- 01
Size it
Workload, model, quantization and context length turned into a VRAM and compute budget, with a written spec before anything is bought.
- 02
Source it
Ordered against the written spec rather than substituted at the till. Lead times on the parts that have one are flagged up front, not discovered mid-project.
- 03
Build and power it up
Racked and cabled, every PSU rail checked, each GPU seated and detected individually before the full stack is assembled and the drivers go on.
- 04
Burn it in
Sustained load until the box hits thermal steady state, not a five-minute benchmark. Throttling and errors get logged, not eyeballed.
- 05
Hand it over
Documentation, remote access and the runbook. Support afterwards if you want it, and none if you do not.
Questions we get
Can you spec a box before we have picked a model?
Yes, and it is common. We usually narrow to two or three candidate models and quantization levels and size the hardware to the worst realistic case rather than the best benchmark number, so the box does not fall short the week you actually pick.
Do you supply the hardware or do we buy it?
Either. The specification is the same work regardless. We procure through our own accounts when that is faster, or hand you a bill of materials to buy against if you have existing vendor terms we should use instead.
Consumer GPUs or datacenter cards?
Depends on the workload and the budget, and we say which case is which rather than default to the expensive option. Consumer RTX-class cards cover a lot of inference work. Datacenter cards earn their price when ECC memory, NVLink bandwidth or the warranty terms actually matter to what you are running.
What happens when it breaks at 2am?
The out-of-band management we configure means most faults get diagnosed and the box power-cycled remotely without anyone driving to it. Beyond that, a support agreement is optional: the runbook is written either way, because the team operating it in a year may not be the team that watched us build it.
Have a model and need the box sized to it?
Tell us the model, the quantization you are targeting and the context length you need. An engineer works the VRAM budget and tells you straight what the hardware has to be.