The Server Test: 3 questions before you buy a small business AI server

This is the guide from the Cortex video on the Server Test. The three questions are the same as in the video, in the same order:

1. The fit. Does the model you need fit on the machine you are thinking of buying?
2. The upkeep. Who updates it, backs it up and fixes it when it stops?
3. The job. Is the work big enough, private enough or constant enough to be worth a server?

If you get to the end with three good answers, a local server may be right for you. If not, you have saved the cost of finding out the hard way.

Why the question comes up

Running AI on your own machine is real and it is getting easier. Ollama, a popular tool for running open models locally, says in its FAQ that it runs locally and its makers do not see your prompts or data when you use it that way. By default it only listens on the machine itself (127.0.0.1 on port 11434), and you have to change a setting to expose it to your network.

That is a strong privacy story. It is also the moment the server becomes yours: the hardware, the updates, the backups and the Saturday when it stops answering.

Most businesses are early with AI of any kind. The Census Bureau reported that 19.8% of businesses used AI as of May 2026, and in the Federal Reserve Banks' 2026 report on employer firms, 37% of firms planning to start named the time to implement and train employees as a challenge. A server adds to that time. It should earn its place.

Question 1, the fit

A model runs fastest when it fits in the graphics card's memory, with room to spare for the work itself. Start with the size of the model file. These are the sizes Ollama lists on its Llama 3.1 library page:

Model	File size (Ollama)	What it suits
llama3.1:8b	4.9 GB	Routine drafting, summaries, sorting messages
llama3.1:70b	43 GB	Harder reasoning; far more memory than a typical office PC

Both list a 128K context window. The "what it suits" column is our view from building with them, not a benchmark.

What to check before you buy:

• Memory against the file size. The file has to fit, plus working room. A machine that only just fits the file will struggle.
• What happens when it does not fit. The Ollama FAQ says a model that does not fit can load partly on the graphics card and partly in system memory, which is slower, and that it can spread across several cards.
• Queues. The same FAQ says queued requests are processed in order. If three agents ask at once, two wait. That is fine overnight and wrong for anything a customer is waiting on.

We do not quote hardware prices or speeds here. They depend on the model, the card and the job, and they change every few months. Ask anyone quoting you for their numbers on your own workload.

Question 2, the upkeep

Write a name next to every line. If a line has no name, the server is not ready.

• Updates. Who updates the operating system, the drivers, Ollama and the models, and how often?
• Backups. Where do the configuration and any stored data get backed up, and has anyone tried restoring them?
• Access. Is it still only listening on the machine itself? If someone changed that setting, who approved it and what protects it now?
• Monitoring. Who notices when it stops answering, and how? An agent that depends on it will simply go quiet.
• Power and network. Is it on a battery backup? What happens after a power cut?
• Replacement. What is the plan when a part fails, and how long can the business run without it?

This list is the real cost of a server. The purchase price is the easy part.

Question 3, the job

A server makes sense when the job is constant, private or large. It rarely makes sense for a couple of small workflows.

Situation	Local server	Cloud
Two follow-up workflows (estimates, reviews)	Not worth it	Start here
Overnight checks across several systems, every night	Can fit well	Also fine
Data you are not allowed to send out of the building	Strong reason	Check the provider's terms first
Anything a customer waits on in real time	Only with enough capacity to avoid queues	Usually simpler
A team with nobody to own the upkeep list	No	Yes

Our own position, from Cortex's private AI page: for two follow-up workflows we would not recommend a server at all. Where we do run local models, such as the overnight checks and morning briefing at Detail Driven on dedicated Linux hardware, the job runs every night and the upkeep has an owner.

A hybrid is often the answer

You do not have to choose once. Many builds run routine, private work on a small local model and send the occasional hard job to a cloud model when you allow it. We compare the options in local vs cloud AI for service businesses.

How to score the test

• Three good answers (it fits with room to spare, every upkeep line has a name, and the job is constant, private or large): a server is worth pricing properly.
• Two: start in the cloud and revisit in six months with real usage numbers.
• One or none: start in the cloud.

For the longer explanation of hardware and upkeep, read self hosted AI for business: what it takes. If you would like someone to run the test with you on your actual workflows, your free AI Operations Audit covers it.