Every vendor pitch you sit through now has the letters LLM in it somewhere. You nod. Later you search for what is an LLM and get a page about neural networks and “transformer architecture,” written for someone with a computer science degree. You close the tab no wiser about whether the thing is safe to put in front of your customers.
You do not need the math. What you need is a working understanding of what an LLM is, what it is actually doing when it answers you, and why it sometimes states something false with total confidence. Once you have that, most decisions about AI in your business become common sense.
So: what is an LLM? This article answers that in plain language, with an analogy you can use in a meeting, and then lays out the rules that follow from it.
What is an LLM, in plain terms
LLM stands for large language model. It is software that has been shown an enormous quantity of written text (books, websites, code, forum posts, manuals, transcripts) and has learned, from all of that, to predict what words are most likely to come next after any piece of text you give it. That is the entire mechanism. When you type a question and get an answer, the software is producing, one small piece at a time, the sequence of words most likely to follow your question, given everything it read during training.
That sounds too simple to produce what you see. The reason the output looks like understanding is that predicting the next word well, across billions of examples, requires the model to pick up grammar, facts, reasoning patterns, tone, and the structure of arguments. So the model ends up with something that behaves a lot like knowledge and a lot like reasoning, without being either in the way a person or a database has them.
The analogy we use with owners is this. Imagine you hired a temp with an extraordinary memory who had spent years reading everything in a very large library, and who is brilliant at one specific thing: producing fluent, plausible text on demand. Ask them anything and they will write you a confident paragraph. Much of the time it is right, because they read a lot. Sometimes it is wrong, because they never read that fact, or read two conflicting versions, and their instinct is to produce something plausible rather than say “I do not know.” They are not looking anything up. They are composing. That distinction runs through everything below.
The rules that follow from how an LLM works
1. It is not a database, and you must never treat it as one
A database stores facts and returns exactly what was stored. An LLM stores nothing in that sense. It has weights, a compressed statistical impression of everything it read. Ask a database for your customer’s phone number and it returns the number or nothing. Ask a bare LLM for your customer’s phone number and it will produce a plausible-looking phone number.
This is the single most important thing an owner should understand. The model does not have your price list, your policies, or your customer records unless a system puts that material in front of it at the moment of the question. Every good business use of an LLM pairs it with a real store of facts (a database, a document set, your actual systems) and has the model read from that store rather than from memory. The technique is called RAG, explained in What Is RAG? Retrieval-Augmented Generation for Business Owners.
2. It makes things up, and the industry calls this hallucination
Because the model composes rather than retrieves, it will sometimes produce statements that are fluent, specific, and false. A case citation that does not exist. A product feature you never shipped. A refund policy that sounds like yours but is not. The technical term is hallucination. It is not a rare malfunction. It is the ordinary behavior of a system whose job is to produce likely text, applied to a question it does not have the facts for.
The current models hallucinate far less than the ones from a few years ago, and they are better at saying “I am not certain.” But it has not gone away and, in our view, it will not, because it is built into what the technology is. So any business system has to assume it: the model answers only from material it was given, cites where the answer came from, refuses when the material does not cover the question, and a person checks anything that matters. The defenses are in AI Hallucination Guardrails for Business Applications.
3. It only knows what it read, up to a date
The model’s training has a cutoff. Everything it absorbed was published before that date, and it learned nothing after. It has never seen your business, and it has never seen this morning’s news unless a tool fetched that news and handed it over. When a chat product seems to know today’s headlines, the product is doing a web search behind the scenes and pasting the results into the conversation before the model answers.
For you this means a model’s general knowledge is a starting point, never a source of truth about anything current or anything specific to you. The value in a business comes from what you connect it to.
4. It works in tokens, and tokens are how you pay
The model does not read words. It reads tokens, chunks of text roughly three quarters of a word each in English. “Invoice” might be one token; “reconciliation” might be three. Everything you send the model and everything it sends back is counted in tokens, and providers bill per token, usually quoted per million.
This is why cost in a real system depends on how much text goes in and out, not on how many “questions” are asked. A short classification of an email might use 500 tokens. Summarizing a 40-page contract might use 30,000. The article on AI Cost for Small Business, What It Really Costs to Run in 2026 turns this into dollars per thousand customer conversations.
5. It has a limited working memory called the context window
The context window is the amount of text the model can consider at once: your instructions, the documents you give it, the conversation so far, and its own answer. The current models have large windows, enough for hundreds of pages, but quality degrades when you stuff the window with irrelevant material. A model asked to answer from 400 pages when the answer is on page 212 will do worse than one given the right three pages. Good systems find the relevant material first and give the model only that. When a vendor says “we just give it all your documents,” ask how they select what the model actually sees.
It also does not remember you between conversations. Each one starts from nothing. Products that seem to “remember” you are keeping notes in an ordinary database and inserting them into the prompt, which means someone decided what to store and for how long. That is a data question worth asking any vendor.
6. The instructions you give it are the product
The text you send the model, including the standing instructions about its role, its rules, and the format of its answer, is called the prompt. In a business system, the prompt is not a casual question. It is a specification: who the model is acting as, what it may and may not say, what to do when unsure, examples of a good answer, and the exact shape the output should take.
Most of the difference between an AI feature that works and one that embarrasses you lives in this document. It should be written down, versioned, tested against real examples, and changed deliberately, the way you would treat a contract template. When it lives in a text box someone edits on a Tuesday, nobody knows why the answers changed on Wednesday. See Prompt Engineering for Business Applications That Work.
7. It is not a calculator and it is not a search engine
Two common misuses. First, arithmetic: the likely text after “what is 4,378 times 291” is a plausible number, not a computed one. The current models are better at this, and some quietly call a calculator tool behind the scenes, but do not build anything that trusts the model’s own math. Extract the numbers, have real software compute them.
Second, search: the model is not scanning the internet or your files when it answers. If it needs to look something up, a tool has to do the looking and hand over the result. This is “tool calling”: the model is given a menu of actions (search the knowledge base, look up an order, check the calendar), asks for the one it needs, then writes its answer from what comes back.
8. Different models are different tools
OpenAI’s GPT-class models, Anthropic’s Claude models, Google’s Gemini, and the open-weight models you can run on your own hardware differ in quality, cost, speed, and terms. Within each provider there are tiers: small cheap models for sorting jobs, large models for reasoning and drafting. A well-built system uses the cheap one where it will do and is written so the provider can be swapped without rebuilding. See Choosing an LLM Provider for Your Business in 2026.
Picture a business like this one
The business below is a composite of the kind of company that writes to us, not a client. The numbers describe the shape of the problem, not a case study.
Picture a business like this one: a two-partner accounting firm with twelve staff and about three hundred small-business clients. Every January the phones and inbox fill with the same forty questions: what documents do I need, when is the deadline for this entity type, where is my engagement letter. The staff answer each one by hand, and the partners field the ones that sound complicated.
The firm’s first attempt was a chat assistant subscription and an instruction to “answer client questions.” It answered fluently and, twice in the first week, wrongly about deadlines, because it was composing from general training and not from the firm’s own calendar. That is hallucination, and it is why the experiment was shut down.
What gets built the second time is different in kind:
- The firm’s actual material (its client FAQ, its deadline calendar by entity type, its document checklists, its engagement letter templates) is collected into one place and kept current by one named staff member.
- When a client question comes in, the system finds the two or three relevant passages from that material and hands only those to the model, with a prompt that says: answer from these passages, quote the passage you used, and if the passages do not cover it, say so and route to a person.
- Anything involving a specific client’s numbers or a judgment call about deductibility is never answered by the model. It drafts a holding reply and creates a task for a staff member.
- Every answer is logged with the question, the passages used, and the reply, and a partner reads a sample every Friday.
What changes is that the forty routine questions get accurate, sourced answers within a minute, staff time in January drops, and the partners see only the questions that need them. The model did not get smarter. It was given the right material and forbidden from guessing.
What it costs to run
For a use like the one above, the model cost is small. Each answer involves perhaps 2,000 tokens in and 300 out; at current published rates for a mid-tier model that is a fraction of a cent per answer, so a few thousand client questions a month is a handful of dollars. Check the provider’s current pricing page for the exact rate. The larger costs are around the model: a place to store and search the firm’s material (a Postgres database with pgvector, or a hosted vector store, typically $0 to $50 a month at this scale), the automation or web app that ties it together (a no-code platform subscription or a small server at $10 to $30 a month), and the staff hour a week spent keeping the material current and reading the log. In our experience that last line decides whether the system is still working in six months.
The mistakes we see most
- Asking the model for facts it was never given. Prices, policies, deadlines, stock levels. If the system did not put the fact in front of the model, the model is guessing.
- Treating a fluent answer as a checked answer. Fluency is the model’s default. It tells you nothing about accuracy. Require citations and read them.
- Editing the prompt casually. One “small change” to the instructions can change every answer. Version it, test it, review it.
- Sending everything into the context window. More material is not better material. Select the relevant passages first.
- Letting the model do arithmetic or make the final call on money. Extract, compute with real code, and put a person on anything irreversible.
When to bring in help
If your goal is personal productivity, you do not need anyone. A subscription to Claude, ChatGPT, or Gemini, used with the understanding above (it composes, it does not look things up, check anything that matters), will save you hours a week on drafting and summarizing.
You need a developer the moment the model has to answer on your behalf from your material, touch your systems, or handle customer questions unattended. That is when the parts around the model (the document store, the retrieval, the prompt as a specification, the refusal rules, the human handoff, the log) have to exist and be right, and none of them come in the box.
Levelbrook builds those systems for businesses: the retrieval layer over your own material, the guardrails, the integrations with the tools you already run, and the plain web apps that put a person in the loop. Fixed price from a written scope, everything runs in accounts you own, and you keep the code and the prompts. If you want a second opinion on whether a use is safe to build, the form below is where that starts.