What a local AI server actually needs
A local AI server is a computer in your home or office that runs open models for you and, often, for the other devices on your network. Four things decide what it can do.
- GPU memory. The single number that matters. It decides which models fit. See the self-hosted LLM guide for how memory is used.
- System memory. 32 GB is comfortable; 64 GB lets you offload parts of a model that does not fit the card, slowly.
- Fast storage. Models are 5 to 60 GB each. A 1 to 2 TB NVMe drive stops you deleting models to make room.
- Power and cooling. A 24 GB consumer card draws 350 to 450 W under load. That needs a proper power supply and somewhere the noise does not matter.
The three build tiers people actually use
| Tier | Card | Runs comfortably | Who builds it |
|---|---|---|---|
| Entry, 16 GB | RTX 4060 Ti 16 GB or 5060 Ti 16 GB | 8B to 14B chat models, SDXL images, gpt-oss 20B | Someone testing the water, or adding AI to a PC they already own |
| Standard, 24 GB | Used RTX 3090 or an RTX 4090 | 32B-class chat at Q4, Flux images, short video clips | The classic home server; most of the community runs this |
| Workstation, 48 to 96 GB | RTX 6000 Ada, or RTX PRO 6000 Blackwell | 70B chat with long context, video models at full resolution, fine-tuning | Professionals, and people who priced it and winced |
The step from 24 GB to 48 or 96 GB is where the price stops being a hobby purchase. That is the step renting is made for.
Running costs nobody puts on the parts list
- Electricity under load, at your local price per kilowatt-hour, for the hours you actually generate.
- Electricity at idle. A machine that stays on draws 40 to 80 W doing nothing. Over a year, at 8 idle hours a day, that is 120 to 230 kWh.
- Resale. Cards hold value better than most computer parts. A realistic resale after three years is 20 to 40% of the price, depending on what launched in the meantime.
- Your time. Drivers, CUDA versions, a runtime that broke after an update. Not money, but not nothing.
- Fixed memory. The card you buy is the card you have. Models get bigger every quarter.
Build vs rent, with your own numbers
Every field is yours to change. The defaults are plausible examples, not our prices or anyone's. The verdict shows the month where owning becomes cheaper than renting, or tells you it never does within your horizon.
What renting has to get right to be worth it
Renting only beats a home build if it removes the parts of a home build that hurt, without adding new ones. In practice that means:
- The setup persists. If every session starts with an empty machine, you are paying to rebuild your own server weekly.
- The meter stops itself. Hourly billing that keeps running after you close the tab is the number one complaint about GPU clouds, and it is avoidable.
- The apps are installed. Ollama, a web interface, ComfyUI, a trainer. The point is to make, not to administer.
- The privacy policy is specific. Who can see what, at each layer, in writing. A rented machine can be private in the ways most people need; it cannot be private by vibes.
That list is the design brief Nodegrove was built against. A grove is a saved workspace that a GPU attaches to when you wake it and leaves when you are done. Hours count only while the GPU is attached and your app is ready.
Verdict
Build if you will run models most of the day, need the machine offline, or already own a 24 GB card. Rent if you use a GPU a few hours a week, want 48 or 96 GB without paying for it up front, or want the setup to be there when you come back without maintaining it. The calculator above will tell you which of those you are, in months.
- Power draw ranges are typical board power figures for the named cards; measure yours if the decision is close.
- Nothing on this page is a price we set. The calculator uses only the values you enter.