What Is a Large Language Model?

Lesson 9: What Is a Large Language Model?

Course: Local AI 101
Level: Beginner → Intermediate
Estimated reading time: 50–70 minutes
Prerequisites: Lessons 1–8
Next lesson: Lesson 10 — What Is a Token?

What You Will Learn

By the end of this lesson, you should be able to:

• Define a language model precisely.
• Explain what it means for a model to model language.
• Explain what makes a language model "large."
• Understand the relationship between tokens, sequences, and language models.
• Understand next-token prediction.
• Explain why predicting the next token can produce surprisingly complex behavior.
• Understand pretraining at a conceptual and technical level.
• Understand self-supervised learning.
• Understand how enormous collections of text can be turned into training examples.
• Understand the difference between a base language model and an instruction-tuned model.
• Understand why a chat model is not simply a database of conversations.
• Understand the role of context.
• Understand vocabulary and tokenization at a high level.
• Understand why LLMs can generate text one token at a time.
• Understand why an LLM can appear to perform many different tasks using the same underlying model.
• Understand the relationship between model parameters and language-model capability.
• Understand why language models can generate incorrect information.
• Understand why LLMs do not automatically "look up" facts while answering.
• Understand how training data becomes distributed statistical representations.
• Understand the difference between training knowledge and information supplied through context.
• Understand how LLMs relate to local AI.
• Understand what happens when an LLM is loaded into Loco.
• Develop a technically accurate mental model of what is happening when you type a prompt into a local LLM.

1. Introduction: We Have Finally Reached the LLM

The previous eight lessons built the foundation. We started with artificial intelligence. Then we moved through:

Now we can finally answer one of the most important questions in modern AI:

What exactly is a Large Language Model?

You have probably used one. You may have asked one to:

• write an email
• explain a scientific concept
• generate code
• summarize an article
• translate text
• brainstorm an idea
• analyze a document
• solve a mathematical problem
• role-play a conversation
• rewrite something
• answer a question

The interface makes all of this look incredibly simple. You type: Explain quantum mechanics to me like I'm a beginner.

and the model produces an answer. Underneath that interface, however, an enormous amount of mathematics is taking place. The model is not opening a little database of pre-written answers.

It is not retrieving a paragraph from a hidden encyclopedia every time you ask a question. It is not following a giant collection of hand-written rules.

Instead, it is using learned numerical parameters to transform a sequence of tokens into a probability distribution over possible next tokens.

That sentence is the foundation of this entire lesson. We are going to unpack every part of it.

2. The Short Definition

A Large Language Model, or LLM, is a large neural-network model trained to model and generate human language. That definition is intentionally broad. A more technical description would be:

A modern LLM is typically a large parameterized neural network, often based on the Transformer architecture, trained on large collections of tokenized data using objectives that teach it to predict or otherwise model language.

For many modern generative LLMs, a central training objective is next-token prediction. Conceptually:

The model receives a sequence and learns to estimate what token is likely to come next. That sounds almost absurdly simple. And yet this objective, combined with:

• enormous datasets
• large neural networks
• substantial compute
• sophisticated optimization
• Transformer architectures
• extensive post-training

can produce systems capable of remarkably flexible behavior. Understanding why requires going deeper.

3. What Is a Language Model?

Before understanding the word large, we need to understand language model. A language model is a model that assigns probabilities to sequences of language. Imagine a sequence:

The dog chased the

Possible next words include:

cat ball rabbit boy girl car ...

A language model can estimate probabilities for possible continuations. Conceptually:

"The dog chased the"

cat → 0.35 rabbit → 0.21 ball → 0.18 boy → 0.05 car → 0.01 ...

These numbers are illustrative. The actual vocabulary could contain tens of thousands or more possible tokens. The model does not need to output every probability to the user.

It internally produces numerical scores that can be transformed into probabilities. Those scores are commonly called logits.

4. Language Is a Sequence

Human language arrives as sequences. For example:

I love learning about artificial intelligence

A language model processes language as a sequence of discrete units. Those units are usually tokens. We will study tokenization in detail in Lesson 10.

For now, think of a token as a piece of text represented by an integer ID. Conceptually:

The exact token boundaries depend on the tokenizer. This gives the model something extremely important:

A language model does not directly process English words as words. It processes numerical representations of tokens.

5. The Simplest Language Model

Imagine a toy language model that has only learned one thing:

"Good morning" → "everyone"

It could be represented as a lookup table. But that would not be very useful. A real language model needs to generalize.

It should understand that:

Good morning, everyone. Good morning, John. Good morning, Sarah. Good morning to you. Good morning everyone!

are related patterns.

A neural language model learns distributed representations that allow it to model relationships across many different sequences.

That is where neural networks become much more powerful than simple lookup tables.

6. The Core Idea: Predict the Next Token

The central idea behind many modern generative LLMs can be expressed very simply:

Given the tokens that came before, estimate what token should come next.

For example:

The sky is

The model might assign high probability to:

blue

Another example:

The model may assign high probability to:

4

Another:

The capital of France is

The model may assign high probability to:

Paris

Again, these are examples of behavior, not a claim that the model contains a little fact table. The model has learned statistical relationships from training.

7. Why Next-Token Prediction Is More Powerful Than It Sounds

At first, next-token prediction sounds like autocomplete. And in a very narrow sense, it is. But consider what a model must learn to become good at predicting the next token across enormous amounts of text.

It must learn patterns involving:

• spelling
• punctuation
• grammar
• syntax
• word relationships
• sentence structure
• paragraph structure
• discourse
• style
• programming syntax
• mathematical notation
• factual associations
• common reasoning patterns
• relationships between concepts
• multilingual structures

If the model is trained on sufficiently diverse data, predicting the next token can require learning many underlying regularities of language.

That is the key insight. The model is not explicitly told: Learn grammar.

then: Learn history. then:

Learn Python.

Instead, these capabilities can emerge as useful internal representations for minimizing prediction error over diverse training data.

8. Language Modeling as Probability

At a mathematical level, a language model can be thought of as estimating:

P(next token | previous tokens)

Read that as:

"The probability of the next token given the tokens that came before it."

For a sequence:

x1, x2, x3, ..., xn

the model estimates relationships between the tokens. A simplified next-token objective is:

P(xₜ | x₁, x₂, ..., xₜ₋₁)

This means:

Probability of token at position t given all previous tokens

This equation is simple. The neural network implementing it is not.

9. From One Prediction to an Entire Paragraph

Here is where generative language modeling becomes interesting. Suppose the prompt is: The ocean was unusually calm that morning.

The model predicts a distribution for the next token. Suppose it selects:

as

Now the sequence becomes:

The ocean was unusually calm that morning as

The model predicts again. Maybe:

the

Now:

The ocean was unusually calm that morning as the

Then:

sun

Then:

rose

Then:

over

and so on. The model repeatedly performs:

A paragraph emerges from many individual token-generation steps.

10. Autoregressive Generation

This process is called autoregressive generation.

"Autoregressive" means that the model uses previously generated values as part of the input for generating future values.

Conceptually:

This is why generated text unfolds sequentially. The model does not normally generate an entire paragraph as one giant indivisible operation. It generates a sequence of tokens.

11. Why the Model Does Not Simply Pick the Most Likely Token Every Time

If the model always selected the single highest-probability token, generation could become repetitive or overly predictable.

Instead, inference can use sampling strategies. For example:

Token A → 0.60 Token B → 0.25 Token C → 0.10 Token D → 0.05

A deterministic strategy might choose:

Token A

A sampling strategy might sometimes select:

Token B

depending on the configuration. This allows generation to vary. Parameters such as temperature can alter the distribution.

We will study generation controls in greater detail later.

12. The Language Model Does Not "Write" Like a Human

When you ask an LLM:

"Write a short story about a robot exploring Mars."

the interface makes it feel like the model has decided:

"I will now write a story."

Technically, the underlying generation process is more mechanical. The model repeatedly predicts likely next tokens conditioned on the context. Yet the result can have:

• characters
• plot
• dialogue
• descriptions
• structure
• tone
• thematic consistency

This apparent complexity emerges from the model's learned representations and the way the Transformer processes context.

The model does not need a separate "story-writing program" for every possible story. It has learned patterns that allow it to generate many forms of text.

13. The "Large" in Large Language Model

Now we can discuss the word large. A language model can be considered large in several related senses. Historically, "large" often referred strongly to parameter count.

A model with:

100 million parameters

was large by older standards. Modern LLMs can have:

billions

or:

tens or hundreds of billions

of parameters. But scale is not only about parameters. Large language models can also involve:

• enormous training datasets
• substantial compute
• large context windows
• sophisticated architectures
• extensive training pipelines

So "large" describes the scale of the overall system, with parameter count being one of the most visible measures.

14. Why Parameter Count Became Such an Important Number

In Lesson 8, we learned that parameter count describes approximately how many learned numerical values a model contains.

LLMs became increasingly capable as researchers scaled:

Model size + Training data + Compute

under appropriate training conditions. This produced increasingly capable models. That does not mean scaling is infinitely efficient.

There are limits, tradeoffs, and diminishing returns. But model scale became a major factor in the development of modern language models.

15. Language Models Before Modern Chatbots

It is important to understand that language models did not begin with conversational assistants. Language modeling existed long before modern generative AI systems. Earlier systems included:

• n-gram language models
• statistical language models
• recurrent neural-network language models
• LSTM-based language models
• Transformer-based language models

The basic goal was related:

Model the statistical structure of language.

What changed dramatically was the scale, architecture, training data, and capabilities.

16. N-Gram Language Models

An old-style language model might estimate the next word using a fixed number of previous words. For example, a trigram model might look at:

"The cat"

and predict what comes next. It might learn:

"The cat sat"

is common. But:

"The cat drove"

is less common. The limitation is context. A simple n-gram model might only consider a small local window.

Modern Transformers can model relationships across much larger contexts. This is one of the reasons Transformer-based LLMs are so powerful.

17. Recurrent Neural Networks

Before Transformers became dominant, recurrent neural networks were widely used for sequence modeling. An RNN processes a sequence step by step while maintaining a hidden state. Conceptually:

The hidden state attempts to carry information from earlier tokens into later processing. RNNs were important, but long sequences created difficulties involving memory, parallelization, and optimization. LSTMs and GRUs improved several of these issues.

Then Transformers changed the landscape.

18. The Transformer Revolution

The Transformer architecture was introduced in the 2017 paper:

Attention Is All You Need

Transformers introduced a powerful way of processing relationships among tokens using attention mechanisms.

Instead of relying entirely on a sequential hidden state passed from one token to the next, the model can use attention to determine which parts of the sequence are relevant to one another.

Conceptually:

The animal didn't cross the road because it was tired. ↑ What is "it"?

Attention allows the model to consider relationships between tokens. The details are more complicated than this example, but the central idea is important:

Attention allows representations to incorporate information from other positions in the sequence.

19. Why Transformers Matter for LLMs

Transformers became highly influential because they work well with large-scale training. They allow substantial parallelization during training. They provide powerful mechanisms for contextual representation.

They can be scaled to very large parameter counts. They support architectures that can process large sequences. This made them an important foundation for modern language models.

Many contemporary LLMs are Transformer-based or use closely related architectural ideas.

20. The Transformer Is Not the Same Thing as an LLM

This distinction matters. A Transformer is an architecture. An LLM is a trained language model.

You can think:

Transformer architecture + Learned parameters + Language-model training = Transformer-based LLM

The architecture defines the computational structure. The learned parameters define the trained behavior. The training process determines how those parameters are obtained.

21. What Does an LLM Actually Learn?

This is one of the hardest questions to answer precisely. An LLM learns statistical structure in its training data. That structure can include:

• linguistic patterns
• relationships between concepts
• syntax
• semantic associations
• code patterns
• factual associations
• common styles
• formatting conventions
• problem-solving patterns
• multilingual relationships

But it does not necessarily learn explicit human-readable rules. Instead, these regularities become encoded in distributed numerical representations.

22. It Is Not a Database

This distinction is critical. A traditional database might contain:

If you ask: What is the capital of France? the database performs a lookup.

An LLM does not normally perform the same kind of operation. Its parameters encode learned statistical relationships.

When you ask the question, the model transforms the token sequence through its learned network and generates a likely continuation.

This can produce the correct answer. But the mechanism is different from a database query.

23. Why the Database Analogy Can Be Dangerous

If you think of an LLM as a database, several confusing behaviors become difficult to explain. For example:

Why can it generate something that never appeared verbatim in training?

Because it learned patterns that allow it to generate novel combinations.

Why can it make mistakes?

Because generation is based on learned statistical representations rather than guaranteed database lookup.

Why can it write a new poem?

Because it learned patterns of language and style.

Why can it answer in multiple languages?

Because multilingual patterns can be represented within the same model.

Why can it solve some problems but fail at others?

Because learned capabilities are uneven and depend on the model, data, task, and context. The neural-network perspective explains these behaviors better.

24. Pretraining

The initial large-scale training phase of many LLMs is called pretraining. During pretraining, the model is exposed to enormous amounts of data. Depending on the model, the data may include:

• books
• websites
• articles
• code
• documentation
• educational material
• other text
• multimodal data in some systems

The exact dataset is model-specific. The important concept is:

The model repeatedly processes training examples and updates its parameters to improve its objective.

For a causal language model, a major objective is next-token prediction.

25. What Does a Training Example Look Like?

Suppose the training text contains: The quick brown fox jumps over the lazy dog. The tokenizer converts this into tokens.

Conceptually:

The quick brown fox jumps over the lazy dog

The model can be trained to predict:

quick

from:

The

Then:

brown

from:

The quick

Then:

fox

from:

The quick brown

and so on.

In practice, training is performed using batches and optimized tensor operations rather than literally running one tiny training operation for each word independently.

But the conceptual structure is useful.

26. Self-Supervised Learning

This type of training is often called self-supervised learning. Why? Because the training data itself provides the target.

Suppose you have: The cat sat on the mat. You do not necessarily need a human to label:

Input: The cat sat on the Target: mat

The sequence itself provides the target. The model can hide or shift the next token and use the original text as the answer. This allows enormous amounts of unlabeled text to become training material.

That is one of the major reasons language models can be trained at huge scale.

27. Why This Is Such a Powerful Training Strategy

Imagine having a human manually label every sentence in a gigantic dataset. That would be enormously expensive. Next-token prediction provides a naturally generated training signal.

For every suitable sequence, the data can produce many prediction targets. For example:

Input: The weather in Durban is

Target: warm

Then:

Input: The weather in Durban is warm

Target: today

And so on. A large corpus can therefore generate a massive number of training examples automatically.

28. The Training Objective

For a causal language model, the objective is often expressed as minimizing negative log-likelihood or cross-entropy loss over the target tokens.

At a high level:

This is the same learning loop we studied earlier. The difference is that the task is specifically language modeling.

29. Why Predicting the Next Token Requires Context

Consider:

The bank was beside the

Possible continuation:

river

But:

I deposited money at the

suggests:

bank

The meaning of a token depends heavily on surrounding context. Therefore, a useful language model needs contextual representations. This is where Transformer attention becomes extremely important.

30. Context Is More Than the Last Word

A good language model cannot only look at the immediately preceding token. Consider: John gave Mark the book because he wanted to read it.

To interpret:

he

the model may need to consider earlier words. Likewise: The company announced a new product. It will launch next month.

The meaning of:

It

depends on context. A Transformer can build contextual representations in which token meaning depends on relationships across the sequence.

31. Contextual Representations

A token does not have only one fixed representation once it enters the Transformer. Its representation changes as the model processes the surrounding context. Conceptually:

So the representation of a word like:

bank

can participate differently in:

river bank

versus:

bank account

The model does not need two completely separate vocabularies. Context changes the internal representation.

32. Vocabulary

An LLM operates over a vocabulary of tokens. Suppose a tokenizer has:

50,000 token IDs

The model ultimately needs to produce scores corresponding to those possible tokens. Conceptually:

The actual vocabulary size varies by model.

The important point is that the model is selecting among token units, not directly selecting among "words" in a dictionary.

33. Why Tokens Matter So Much

Tokens affect:

• input length
• output length
• context usage
• computation
• memory
• pricing for cloud APIs
• generation speed
• vocabulary size
• embedding parameters

This is why tokenization deserves its own lesson. In the next lesson, we will go deep into:

• what tokens are
• how text is split
• subword tokenization
• token IDs
• vocabulary
• why one word can become multiple tokens
• why token counts differ across languages
• how tokenization affects local inference

34. The Input Pipeline

When you type: Explain local AI. a modern LLM system does not simply pass those characters directly into the neural network.

Conceptually:

Every stage matters.

35. Step 1: Text

The user provides text: Explain local AI. This is human-readable.

The model does not operate directly on these characters.

36. Step 2: Tokenization

The tokenizer converts the text into token IDs. For example, purely illustratively:

"Explain local AI."

might become:

[1823, 941, 5072, 13]

The numbers above are fictional. The real IDs depend on the tokenizer. The important concept is:

Text → integer sequence

37. Step 3: Embeddings

The token IDs are mapped into vectors. Conceptually:

1823 → [0.12, -0.03, ...] 941 → [0.77, 0.21, ...] 5072 → [0.08, -0.42, ...] 13 → [0.15, 0.04, ...]

The vectors have the model's hidden dimension. Now the model has numerical representations that can be processed by the Transformer.

38. Step 4: Transformer Processing

The embeddings move through multiple Transformer layers. Conceptually:

Each layer transforms the representations. Attention allows contextual interactions. MLP components perform additional learned transformations.

Normalization and residual connections help stabilize and structure the network. The exact architecture varies by model.

39. Step 5: Logits

At the end of the network, the model produces numerical scores for possible next tokens. These are commonly called logits. Imagine:

Token A → 4.8 Token B → 2.1 Token C → 0.7 Token D → -1.2

These are not probabilities yet. They are scores. A softmax operation can transform them into a probability distribution.

40. Step 6: Token Selection

The system uses the logits/probabilities and generation settings to choose the next token. It may use:

• greedy decoding
• temperature
• top-k sampling
• top-p sampling
• other decoding methods

The selected token is appended to the sequence. Then the model generates again.

41. Step 7: Decode Back to Text

The generated token IDs are converted back into text. Conceptually:

This is what the user sees. So the full loop is:

This is one of the most important diagrams in Local AI 101.

42. Why the Model Can Answer Questions

Suppose you ask: What is photosynthesis? The model processes the token sequence.

During training, it encountered enormous numbers of related patterns involving:

• plants
• biology
• sunlight
• chlorophyll
• carbon dioxide
• oxygen
• glucose
• educational explanations

Those relationships influence the learned parameters. During inference, the current context activates relevant learned representations. The model then generates a continuation that is statistically compatible with the context.

This can produce a coherent explanation.

43. Why the Model Can Write Code

Now ask:

Write a Python function that calculates factorial.

The same underlying model can generate code. Why? Because the training data may contain code and programming-related text.

The model has learned statistical structures such as:

def
function names
parentheses
indentation
operators
common libraries
documentation
programming patterns

It does not need a completely separate neural network called "the coding module" in the simple sense. The same parameterized network can model many forms of sequence data.

44. Why the Model Can Translate

Suppose you ask: Translate "Good morning" into French. If the model has learned multilingual patterns, it can generate:

Bonjour

The underlying mechanism remains language modeling. The context establishes the task. The learned parameters contain multilingual relationships.

The model generates the continuation.

45. Why the Same Model Can Perform Different Tasks

This is one of the most remarkable aspects of LLMs. The same model can be prompted to:

summarize translate classify write code explain extract brainstorm transform

Why? Because these tasks can all be expressed as sequences of tokens. For example:

Summarize this: [document]

and:

Translate this: [document]

are both simply token sequences presented to the same neural network. The prompt provides instructions that influence the output distribution.

46. The Prompt Is Part of the Input

A system prompt, user prompt, document, conversation history, or tool output can all become part of the model's context.

Conceptually:

The model processes the combined context. This is why context engineering becomes so important in AI systems.

47. Context Is Not the Same as Training

This distinction is essential. Suppose you tell the model: My favorite color is orange.

The model can use that information during the current context. That does not automatically mean:

The model permanently learned "the user likes orange."

Instead:

Information in context

is available to the model during that inference interaction. Permanent parameter changes require training or another explicit learning mechanism.

48. Context Can Temporarily Change Behavior

Suppose the same model receives: You are a chemistry tutor. It may respond in a teaching style.

Then another conversation might say: You are a concise software engineer. The model can respond differently.

The underlying parameters have not necessarily changed. The input context has changed. Therefore:

Same model + different context = different behavior

This is one of the most important ideas behind prompt-based AI systems.

49. Instruction Tuning

A raw pretrained language model is not necessarily optimized to behave like a helpful assistant. It may be good at continuing text without being particularly good at following natural-language instructions. For example, a base model might receive:

Translate the following sentence: and continue in unexpected ways. Post-training techniques can improve instruction following.

One important approach is instruction tuning. The model is trained on examples resembling:

Instruction + Input → Desired response

This teaches the model to respond to user requests more effectively.

50. Base Models

A base model is a model that has primarily undergone its foundational pretraining. It has learned language patterns from the training corpus. It may be capable of text completion but may not behave like a polished conversational assistant.

For example, given:

Question: What is gravity?

Answer: a base model may continue the sequence based on patterns it learned. It does not necessarily understand the interaction in the same way a heavily instruction-tuned assistant does.

51. Instruction-Tuned Models

An instruction-tuned model is further trained to follow instructions. The training data may include examples such as:

User: Explain photosynthesis.

Assistant: Photosynthesis is...

or:

User: Convert this list into JSON.

Assistant: { ... }

The model learns patterns associated with instruction following. This is a major part of why modern chat models feel dramatically more useful than raw text-completion models.

52. Chat Models

A chat-oriented LLM is typically configured and trained to operate in conversational settings. The conversation may be represented using structured roles such as:

system user assistant

Conceptually:

System: You are a helpful assistant.

User: Explain local AI.

Assistant: ...

These messages are converted into a token sequence that the model processes. The exact formatting is model-specific. This is one reason using the correct chat template matters for local models.

53. Chat Templates

Different models can expect conversations to be formatted differently. A model might conceptually expect:

... ... ...

Another model may use different special tokens. The exact representation depends on the model family. This is why an inference runtime needs to understand model metadata and chat-template conventions.

Incorrect formatting can produce poor behavior even when the model itself is perfectly capable.

54. Why a Chat Interface Is Not the Model

This distinction is important. When you see:

Chat interface

you are looking at an application layer. The LLM itself is underneath. A simplified stack is:

The interface is not the intelligence by itself. It is a way of interacting with the model and the surrounding system.

55. LLMs Are Part of Larger AI Systems

A modern AI application can contain:

User interface + LLM + retrieval + database + tools + web access + file processing + memory + workflow logic

This distinction will become extremely important later in Local AI 101. An LLM can be the central reasoning or generation component without being the entire application.

56. The LLM vs the AI System

Suppose Loco eventually uses a local model to answer questions about a company's internal documents. The complete system might look like:

The LLM itself is only one component. The retrieval system provides external information. The interface manages the interaction.

The runtime executes the model. This distinction prevents a lot of confusion about what an LLM can and cannot do by itself.

57. Why LLMs Can Hallucinate

An LLM can generate text that sounds confident and plausible but is incorrect. Why? Because its fundamental generation objective is related to producing probable continuations, not guaranteeing truth.

The model does not automatically possess a built-in truth database. If its learned representations or context support an incorrect continuation, it may generate it. For example:

User: Who invented X?

If the model has uncertain or conflicting patterns, it can produce an answer that sounds reasonable but is wrong. This is often called a hallucination.

58. Fluency Is Not Proof

An LLM can produce:

beautifully written

text that is:

incorrect

These are independent properties. A fluent answer is not automatically a factual answer. This is one of the most important habits for using AI responsibly.

You should distinguish:

Language quality

from:

Factual reliability

and:

Reasoning quality

59. Why Bigger LLMs Can Still Hallucinate

Increasing parameter count can improve many capabilities. But it does not transform the model into a perfect truth machine. A larger model can still:

• misunderstand a question
• lack current information
• misremember a detail
• combine unrelated facts
• follow misleading context
• generate plausible but unsupported claims

The underlying architecture remains probabilistic. This is why retrieval, tools, verification, and grounding can be important in practical AI systems.

60. Training Data Is Not the Same as Current Knowledge

Suppose a model was trained on data ending at some historical point. You ask: What happened yesterday?

The model cannot magically know events that occurred after its training unless current information is supplied through some external mechanism.

This could include:

• retrieval
• web access
• a connected database
• a tool
• user-provided documents
• another current information source

Without such a mechanism, the model is constrained by its learned parameters and current context.

61. Knowledge Cutoff and Local Models

This matters for local AI. A local model running entirely offline does not automatically have access to:

today's news live stock prices current weather new websites new documents

unless you provide an appropriate information source. This is not a defect unique to local AI. It is a property of how standalone model inference works.

A model's parameters are not a live internet connection.

62. LLMs and Retrieval

One way to provide external information is retrieval. Conceptually:

The LLM uses the retrieved text as context. This is commonly associated with Retrieval-Augmented Generation, or RAG. We will study RAG much later in the course.

For now, remember:

An LLM's parameters and its context are different sources of information.

63. LLMs and Tools

Another approach is to give the model access to tools. For example:

The model can generate an instruction for the tool. The tool performs an operation. The result is returned to the model.

This creates a larger AI system around the LLM.

64. LLMs and Agents

Agents add additional structure. An agent may:

The LLM can serve as a decision-making or language-generation component. But the agent system includes additional software. This distinction will become important when we later study local AI agents and workflows.

65. What Makes an LLM Different From a Normal Program?

Consider a calculator program. You give it:

and it executes a deterministic mathematical rule. An LLM is different. Its behavior is produced by:

learned parameters + input context + inference process + decoding strategy

The exact output can vary. The model is not following a manually written rule for every possible question. Instead, it learned a high-dimensional function during training.

66. Deterministic vs Probabilistic Behavior

A conventional function might be:

every time. An LLM can produce different outputs depending on generation settings. For example:

Prompt: Write a sentence about rain.

One generation: Rain tapped softly against the window. Another:

Heavy rain swept across the quiet street. The same underlying model can produce different valid continuations. This variability is a feature of generative modeling.

67. Temperature

Temperature is one way to influence the probability distribution used during generation. Conceptually:

The exact mathematical implementation applies a transformation to logits before sampling. Temperature does not change the model's learned parameters. It changes how output is selected.

This distinction is important.

68. Model Parameters vs Generation Settings

Compare:

Model parameters

weights biases embeddings

These define learned model behavior.

Generation settings

temperature top-k top-p maximum output tokens

These influence inference behavior. Changing temperature does not retrain the model. Changing a quantization setting does not retrain the model.

Changing the prompt does not retrain the model. These are inference-time changes.

69. The LLM as a Conditional Generator

A useful mathematical mental model is:

Output ~ P(output | context, parameters)

The model's output depends on:

learned parameters

and:

current context

and:

generation process

This explains why the same model can behave differently across prompts.

70. Why LLMs Can Generalize

Suppose the model has seen many examples of:

Python functions

during training. You give it a new programming problem it has never seen verbatim. It may still generate a useful solution.

Why? Because it has learned patterns rather than merely memorizing one exact answer. The model can combine learned structures in novel ways.

This is one of the major reasons neural language models are so useful. However, generalization is imperfect. The model can also fail on seemingly simple tasks.

71. Memorization vs Generalization

LLMs can exhibit both:

memorization

and:

generalization

Memorization means learned information can sometimes closely reproduce patterns from training. Generalization means the model can apply learned patterns to new inputs. Real model behavior contains both.

The distinction is important because neither extreme is a complete description. It is inaccurate to say:

"The model just memorizes everything."

It is also inaccurate to say:

"The model never memorizes anything."

The actual behavior is more complicated.

72. Why Novel Text Is Possible

Suppose an LLM has seen: The cat slept on the couch. and:

The dog ran across the garden. It can generate: The cat ran across the garden.

This exact sentence may never have appeared in training. The model can combine learned linguistic patterns. It does not need to retrieve the exact sentence from a database.

This is one reason generative models can produce novel outputs.

73. The Role of Composition

Complex behavior can emerge from combining learned patterns. For example:

language pattern + formatting pattern + domain knowledge + instruction + context

can produce a response that is not a direct copy of any one training example. This compositional behavior is central to generative AI.

74. Why Prompting Works

Prompting works because the prompt changes the input sequence. Consider: Explain gravity.

versus: Explain gravity to a 10-year-old using a simple analogy. The second prompt provides additional information.

The model's next-token probability distribution changes. Conceptually:

Same parameters + Prompt A → Output A

Same parameters + Prompt B → Output B

The model does not necessarily need to be retrained for each task. The task can be specified through context.

75. Prompting Is Not Reprogramming the Model

A prompt can strongly influence behavior. But it is important not to think of prompting as rewriting the model's parameters. If you tell a model:

You are a pirate. you are not changing the weights into pirate weights. You are supplying a context that influences the model's generated continuation.

The learned parameters remain the same.

76. Why System Prompts Matter

A system prompt can provide high-priority context such as:

You are a technical tutor. Use concise explanations. Do not invent sources.

The exact behavior depends on the model and surrounding system. But conceptually, the system instructions become part of the context that influences generation.

This is why applications can give a general-purpose LLM a specialized personality or task orientation without retraining it.

77. The LLM Is Not Necessarily "Thinking" in the Human Sense

People often say:

"The AI is thinking."

This can be a useful conversational shorthand.

Technically, however, an LLM is performing mathematical transformations over learned representations and generating tokens.

Whether and how that should be described as "thinking" is a philosophical and scientific question. For practical Local AI work, the safer technical model is:

This gives you something concrete to reason about.

78. Why LLM Reasoning Can Look Impressive

Some LLMs can produce multi-step explanations. For example:

Step 1 Step 2 Step 3 Conclusion

The model generates those tokens based on learned patterns and current context. The apparent reasoning can be useful. But the existence of coherent reasoning-like text does not guarantee correctness.

An LLM can produce a beautifully structured chain of reasoning that contains an incorrect assumption. This is why verification remains important.

79. LLMs as Learned Statistical Systems

A useful neutral description is:

An LLM is a learned statistical system that models patterns in token sequences using a large neural network.

That description avoids two common extremes. It is not:

just autocomplete

in the simplistic sense. But it is also not necessarily:

a human mind inside a computer

The actual system is more interesting than either description.

80. Why "Just Autocomplete" Is Misleading

It is technically true that autoregressive language models predict the next token. But saying:

"It's just autocomplete."

can hide the complexity of what the model has learned. A modern LLM may have to model:

• syntax
• semantics
• long-range dependencies
• code
• formatting
• multilingual patterns
• domain relationships
• task instructions

The next-token objective is simple to state. The internal function needed to perform it well is extremely complex.

81. Why "It Understands Everything" Is Also Misleading

The opposite mistake is to assume the model has human-like understanding. An LLM can demonstrate sophisticated behavior while still having serious limitations. It can:

• misunderstand context
• hallucinate
• make arithmetic errors
• fail on unusual inputs
• confidently state false information
• struggle with exact requirements
• be sensitive to phrasing

Therefore, the correct mental model is somewhere between:

"mere autocomplete"

and:

"human-like mind"

The technical reality is a large learned neural network that can model language remarkably well.

82. What Is an LLM's "Knowledge"?

This question can now be answered more precisely. An LLM's learned knowledge is not stored as a conventional knowledge base. It is reflected in the model's learned parameters.

Those parameters encode statistical relationships acquired through training. During inference, those parameters interact with the current context. So:

Then:

83. Training Data vs Context

This distinction deserves a simple comparison.

For example:

Training: Model learns programming patterns.

Context: User provides a specific codebase.

The model can use both.

84. Why Local LLMs Can Be Private

If the model runs entirely on your own computer and the surrounding application does not send the data elsewhere, your prompt and documents can remain local.

The model itself does not require an internet connection merely to perform inference. Conceptually:

This is one of the major reasons local AI is attractive for privacy-sensitive workloads. However, privacy depends on the complete system. A local model can still be connected to:

• network services
• APIs
• webhooks
• external tools
• cloud integrations

if the application is configured to use them. So "local model" and "completely offline system" are not automatically identical concepts.

85. LLMs on a Local Computer

When you run an LLM locally, several components work together. Conceptually:

The LLM is the learned model. Loco is the surrounding application layer. The CPU/GPU performs the mathematical computation.

The memory holds model data and runtime state.

86. What Happens When You Ask a Local LLM a Question?

Suppose you type: Explain how a GPU works. A simplified local pipeline is:

1. Loco receives the message.
2. Conversation context is assembled.
3. Text is tokenized.
4. Tokens become model inputs.
5. The inference runtime executes the model.
6. The model produces logits.
7. A decoding strategy selects the next token.
8. The token is appended.
9. Generation repeats.
10. Tokens are decoded into text.
11. Loco displays the result.

This process can happen without sending the prompt to a remote AI service when the system is configured for local-only inference.

87. The Model File Contains the Learned State

At the heart of this process is the model file. It represents the learned numerical parameters in a format that the inference runtime can load. For example:

The model file itself is not the entire application. It is the learned model representation.

88. The Runtime Is the Engine

The inference runtime performs the mathematical operations defined by the model architecture. A runtime can handle tasks such as:

• loading tensors
• allocating memory
• executing matrix operations
• managing attention
• handling context
• managing KV cache
• producing logits
• sampling or exposing logits for sampling

Open-source infrastructure such as llama.cpp is one example of technology used to perform local model inference across supported architectures and hardware.

The important conceptual separation is:

89. Why This Separation Matters

Imagine downloading a model file but having no compatible inference runtime. The file contains learned numerical information. But something still needs to:

interpret the architecture load the tensors perform the operations manage memory generate output

Likewise, a runtime without a trained model does not magically know how to answer questions. You need both:

trained model + inference software

for practical model execution.

90. LLMs and Hardware

The model's parameters must ultimately participate in computation. That computation can involve:

CPU GPU or combinations of both

The exact hardware path depends on the runtime and configuration. The larger the model, the more demanding the computation can become. This is why the parameter-count lessons from Lesson 8 matter directly here.

91. Why Quantized LLMs Are Important Locally

A full-precision model may be too large for a particular computer. A quantized representation can make it practical. For example:

The tradeoff is that quantization can introduce some numerical approximation. Modern quantization methods attempt to minimize the impact.

92. Context Length

An LLM does not have infinite context. A model may support a particular maximum context length. The context can include:

• system instructions
• user messages
• previous conversation
• retrieved documents
• tool outputs
• generated tokens

Conceptually:

+-----------------------------------+
| Context window                    |
|                                   |
| system                            |
| conversation                      |
| documents                         |
| current user message              |
| generated output                  |
+-----------------------------------+

The available context is finite. As context grows, computational and memory requirements can also change. We will study context windows in detail later.

93. Why Long Context Is Not Free

A common beginner assumption is:

"If a model supports a million tokens, it should be able to process a million tokens with no additional cost."

That is not how computation works. Longer context can increase:

• memory usage
• attention-related computation
• KV cache size
• processing time

Architecture and implementation strongly affect the scaling behavior. So context length is both a capability and a resource consideration.

94. The LLM's Basic Generation Loop

Here is perhaps the most useful operational diagram in this lesson:

This loop is the basic mechanism behind autoregressive text generation.

95. Why One Token at a Time Can Produce Complex Documents

Suppose the model generates:

A

Then:

A local

Then:

A local AI

Then:

A local AI model

Then:

A local AI model can

and so on. At every step, the previous generated tokens become part of the context. That means the model can build:

• sentences
• paragraphs
• sections
• code
• structured lists
• stories

through repeated conditional generation. The complexity of the final document does not require a single "generate entire document" operation. It emerges from the sequence.

96. Why Errors Can Compound

Autoregressive generation has an important consequence. If an early generated token is wrong, that token becomes part of the subsequent context. Then later predictions are made partly based on the model's own previous output.

Conceptually:

This is one reason long generated outputs can drift away from the intended answer. The model conditions on its own generated sequence.

97. Why Prompt Quality Matters

Because the model conditions on context, a clear prompt can make the task easier. Compare: Tell me about AI.

with:

Explain local AI to a complete beginner. Define CPU, GPU, RAM and VRAM. Use practical examples. Keep the explanation under 1,000 words.

The second prompt gives the model much more information about the desired output. It does not change the parameters. It changes the conditioning context.

98. LLMs and Structured Output

An LLM can also generate structured formats such as:

{ "name": "John", "age": 30 }

or:

Name: John

Age: 30

The model learned patterns of structured text. Some systems add additional constraints or grammar mechanisms to make structured output more reliable. The underlying model is still generating tokens.

99. LLMs and Code

Code is particularly interesting because it has strong structural constraints. For example:

def add(a, b):
    return a + b

The model learns statistical patterns involving:

• keywords
• indentation
• operators
• function definitions
• APIs
• comments
• documentation
• common algorithms

This can make code generation surprisingly capable. But generated code can still contain:

• syntax errors
• logical errors
• security issues
• incorrect assumptions
• nonexistent APIs

The LLM's fluency does not guarantee executable correctness.

100. LLMs and Mathematics

Language models can generate mathematical expressions because mathematical notation appears as structured sequences. For example:

can be represented as tokens. The model can learn patterns associated with mathematical transformations. However, language-model generation is not equivalent to a formal theorem prover or arbitrary-precision calculator.

For exact numerical work, external tools can be useful. This is another example of why an LLM can be powerful while still having domain-specific limitations.

101. LLMs and Multimodal Models

Not every modern model is text-only. Some models can process combinations of:

• text
• images
• audio
• video

These are often called multimodal models. The underlying architecture may combine different input-processing components with language-model generation. A multimodal model is still related to the language-model concept, but the input pipeline is broader.

For example:

We will explore multimodal local AI much later.

102. Why LLMs Became the Center of Generative AI

LLMs became extremely important because language is a universal interface for many tasks. Humans can describe:

what they want

in natural language. The model can then generate:

text code structured data instructions plans tool calls

This makes language a convenient control layer for software systems. The LLM itself is not necessarily the entire solution. But it can serve as a flexible interface between human instructions and machine actions.

103. LLMs as Interfaces

Imagine saying:

Find the three most important invoices from this folder and summarize them.

A complete AI system could:

The LLM may help interpret the instruction and generate the summary. Other software components perform file access and retrieval. This is why modern AI applications are becoming systems rather than isolated models.

104. LLMs and Local Business AI

The same principle applies to local business deployments. Imagine a company with:

customer database sales documents internal procedures product catalog support tickets

A local LLM can potentially serve as the language interface. For example:

"Summarize the support history for this customer."

The surrounding system can retrieve the relevant data. The LLM can then generate the explanation. This is the broader direction in which local AI becomes useful for organizations.

105. Why Local LLMs Matter

A local LLM gives an organization or individual the ability to perform inference on their own hardware. Potential advantages can include:

• data remaining within a controlled environment
• reduced dependency on external inference services
• predictable local availability
• customization of the surrounding system
• ability to operate in disconnected environments
• control over model choice

These benefits depend on the actual deployment architecture. A local model does not automatically make every system private or secure. The entire application stack matters.

106. Local Does Not Automatically Mean Offline

This distinction is worth repeating. A model can be local while the application still connects to:

the internet

for:

• model downloads
• updates
• authentication
• licensing
• web search
• email
• APIs
• integrations

So there are multiple concepts:

Local inference

and:

Offline operation

They overlap, but they are not identical.

107. Local Does Not Automatically Mean Secure

Security depends on:

• operating system security
• application permissions
• network configuration
• authentication
• encryption
• model source
• file permissions
• exposed APIs
• connected services
• user practices

A local model can reduce some classes of external data exposure. It does not eliminate all security risks. This distinction becomes particularly important when deploying local AI inside businesses.

108. LLMs and Model Choice

Now that you understand what an LLM is, choosing one becomes easier to reason about. You might compare models by:

Parameter count Architecture Context length Quantization Modality Training purpose Instruction tuning License Hardware requirements Performance

Parameter count is only one dimension. A model with fewer parameters can be more appropriate for a constrained computer. A larger model may be useful when greater capability justifies the resource requirements.

109. The Local AI Model Selection Problem

Imagine you have:

limited RAM no dedicated GPU

and want:

a general-purpose assistant

You probably do not start by downloading the largest model you can find. You start by asking: What size model can my hardware realistically handle?

Then: What quantization should I use? Then:

What context length do I need? Then: What model family is appropriate for my task?

This is the engineering mindset local AI requires.

110. Why LLMs Can Run on Ordinary Computers

You might wonder:

"How can billions of parameters run on a personal computer?"

The answer is a combination of:

• model compression
• quantization
• efficient inference runtimes
• CPU optimization
• GPU acceleration
• memory management
• smaller model architectures

Not every LLM can run on every computer. But many models can be made practical on consumer hardware. This is a major reason local AI has become accessible.

111. What Happens Inside a Local Runtime

A local inference runtime may perform operations such as:

Different runtimes implement these operations differently. The conceptual pipeline remains useful.

112. Prompt Processing vs Generation

When you provide a long prompt, the model first has to process the existing context. Then it begins generating new tokens. These are different workloads.

Conceptually:

This distinction matters for performance measurements. Later lessons will explore:

• prompt processing
• time to first token
• token generation speed
• context length
• KV cache

113. Why First-Token Latency Can Feel Different

Suppose your prompt is:

10,000 tokens

The system may need to process that large context before generating the first new token. So:

After generation begins, subsequent tokens may arrive at a different rate. This is why users sometimes experience:

initial pause

followed by:

streaming output

The technical reasons are important when evaluating local AI performance.

114. Streaming

Many LLM applications display output as it is generated. For example:

The The quick The quick brown The quick brown fox ...

The application receives generated tokens incrementally and displays them. This creates the feeling of an ongoing conversation. The model is still generating sequentially.

Streaming is primarily an interface and delivery behavior around the generation process.

115. Why LLM Output Can Stop Mid-Sentence

Generation does not necessarily continue forever. The model can produce an end-of-sequence token or reach a configured generation limit. The application can also stop generation based on:

• maximum token count
• stop sequences
• user cancellation
• system constraints

This is another reason the surrounding inference system matters.

116. The Importance of Special Tokens

LLMs can have special tokens used for purposes such as:

• beginning of sequence
• end of sequence
• padding
• role markers
• tool instructions
• other control functions

These tokens are part of the model's vocabulary or processing conventions. They help structure the input and output. This is another reason tokenization is more sophisticated than simply splitting text on spaces.

117. Why LLMs Have Context Limits

A model architecture has constraints on how much sequence information it can process effectively. The context window defines a maximum or practical amount of token context. For example:

8K tokens 32K tokens 128K tokens

These labels describe context scale. They do not describe:

parameter count

and they do not mean:

8K parameters

This distinction is crucial.

118. Parameter Count vs Context Length

Compare:

Model: 8B parameters

Context: 32K tokens

These numbers describe completely different things.

8B → learned model scale

32K → sequence/context capacity

One is about the model's learned parameter state. The other is about how much token context the model can process.

119. Parameter Count vs Vocabulary Size

Likewise:

8B parameters

does not mean:

8B vocabulary tokens

A model could have:

8B parameters 50K-token vocabulary

The vocabulary size is one architectural dimension. The parameter count is the total learned numerical state.

They are related because embedding and output matrices depend partly on vocabulary size, but they are not the same quantity.

120. LLMs and Model Compression

A trained LLM can often be compressed or transformed into representations that are easier to deploy. Methods include:

• quantization
• pruning
• distillation
• architectural optimization

These methods have different tradeoffs. Quantization is especially important for local inference because it can reduce memory and storage requirements. Knowledge distillation can create a smaller model trained to reproduce useful behavior from a larger teacher model.

These are different approaches to making AI more efficient.

121. Small Language Models

You may hear the term:

SLM, meaning Small Language Model.

The distinction between "small" and "large" is relative. A 1B model may be small compared with a 70B model. But 1B parameters is still an enormous numerical system compared with many traditional machine-learning models.

The important lesson is: LLM scale is relative to the technology and task. Local AI benefits greatly from smaller models because deployment resources are limited.

122. Why Smaller Models Can Be Surprisingly Capable

A model does not need to be enormous to learn useful language patterns. A carefully trained smaller model can perform well on specific tasks. For example:

text classification summarization basic coding document extraction simple assistants

may not require the largest available model. This is why local AI is increasingly about selecting the right model rather than simply selecting the largest one.

123. Model Specialization

Some LLMs are designed or fine-tuned for particular domains. Examples can include:

• coding
• mathematics
• medicine
• legal text
• scientific literature
• instruction following
• multilingual use

A specialized model may perform differently from a general-purpose model with a similar parameter count. Again:

Parameter count

is only one part of the story.

124. Fine-Tuning

Fine-tuning involves continuing training on a narrower dataset or objective. For example:

The model's parameters are adjusted. This is fundamentally different from prompting. Prompting:

same parameters + different context

Fine-tuning:

new training + updated parameters

This distinction will become important later in the course.

125. LLMs and LoRA

LoRA, or Low-Rank Adaptation, is a technique for adapting models more efficiently.

Instead of updating every original parameter directly, LoRA introduces additional trainable components that modify the model's behavior.

Conceptually:

This can greatly reduce the number of parameters that need to be trained for an adaptation. We will explore LoRA in the advanced section of Local AI 101.

126. Why LLMs Are Not Static in the Broader Ecosystem

A model file may be fixed. But the ecosystem around LLMs changes constantly. New:

• model families
• quantization methods
• runtimes
• hardware
• training techniques
• context techniques
• agent frameworks
• multimodal capabilities

appear over time. This is why learning the underlying concepts is more valuable than memorizing one particular model name. The model names will change.

The principles remain useful.

127. A Technical Summary of an LLM

We can now give a much more complete definition:

A modern Large Language Model is a large parameterized neural network, commonly Transformer-based, trained on large-scale tokenized data to model language. In a causal generative setup, it estimates a probability distribution for the next token conditioned on the preceding context. During inference, the model repeatedly processes context, produces logits over its vocabulary, applies a decoding strategy, selects a token, and feeds that token back into the sequence to continue generation.

That is the technical definition you should carry forward.

128. The Entire LLM Pipeline

Here is the complete simplified picture:

This diagram connects almost every concept from Lessons 1–9.

129. LLMs and Loco

Now let's bring the concepts into the local-AI environment. When you use Loco to run an LLM locally, the fundamental process is:

The model's parameters provide the learned behavior. The runtime provides the computation. Your hardware provides the resources.

The Loco interface provides the user-facing experience.

130. Why Understanding LLMs Helps You Use Loco

Once you understand what an LLM actually is, model selection becomes much less mysterious. You can look at a model and ask:

How many parameters? What architecture? What quantization? What context length? What modality? What task? What hardware?

Instead of:

"That model has a cool name, I'll download it."

You can make an engineering decision. That is one of the main goals of Loco Learn.

131. The Local Model Library Mental Model

Imagine a Loco model library containing:

Model A 3B 4-bit Text General

Model B 8B 4-bit Text Coding

Model C 14B 4-bit Text General

Model D 7B Vision Multimodal

A beginner can now interpret the fields. They understand:

3B / 8B / 14B

are parameter-scale indicators. They understand:

4-bit

is a numerical representation. They understand:

Vision

indicates additional input capability. They understand that:

context length

is separate from parameter count. This is practical AI literacy.

132. What an LLM Cannot Tell You From Parameter Count Alone

Suppose you only know:

You do not know:

• how good its coding is
• how good its reasoning is
• how accurate it is
• what languages it supports well
• what context it supports
• what license it uses
• how fast it will run on your computer
• what quantization is available
• whether it supports vision
• whether it is instruction-tuned
• whether it is designed for a specific task

This is why a good model library needs richer metadata.

133. What an LLM Is Not

An LLM is not automatically:

a search engine

It is not automatically:

a database

It is not automatically:

a calculator

It is not automatically:

a web browser

It is not automatically:

an agent

It is not automatically:

a knowledge base

It is a learned model. Those other capabilities can be added around it.

134. The LLM as a Core Component

A useful way to visualize a modern AI application is:

The LLM is often central. But it is not the entire system.

135. Why This Matters for the Future of Local AI

The future of local AI is not necessarily:

Download one model Open chat Ask questions

It can also involve:

local models + private data + retrieval + agents + tools + workflows + APIs + business applications

This is where local AI becomes infrastructure rather than merely a chatbot.

Loco is positioned conceptually at this broader layer: helping users manage and use local AI models and build useful systems around them.

136. Common Misconception: "An LLM Understands Language Exactly Like a Human"

Not necessarily. An LLM can model language remarkably well. It can produce useful explanations and interact fluently.

But its internal mechanisms are neural-network computations over learned representations. Human cognition involves biological processes that are fundamentally different.

For practical AI engineering, focus on observable capabilities and limitations rather than assuming human-like cognition.

137. Common Misconception: "LLM Means Any AI That Uses Text"

Not necessarily. A system can use text without being a large language model. For example:

• a rule-based chatbot
• a traditional search engine
• a small classifier
• a database query system

can all process text. An LLM is a specific kind of large neural language model.

138. Common Misconception: "Every LLM Is a Chatbot"

No. A language model can be used for:

• completion
• classification
• extraction
• summarization
• code generation
• translation
• agents
• retrieval systems
• structured output

A chat interface is one application pattern.

139. Common Misconception: "LLM Training Means Someone Taught It Every Fact Manually"

No. Large-scale pretraining can use self-supervised objectives derived from the data itself. Human involvement can still be substantial in:

• dataset construction
• filtering
• evaluation
• instruction data
• preference data
• safety work
• post-training

But the model does not require a human to manually label every individual fact.

140. Common Misconception: "The Model Searches the Internet When You Ask a Question"

Not automatically. A standalone LLM generates from:

parameters + current context

If the surrounding application gives it web access or retrieval tools, then it can use external information. That is a system-level capability. It is not inherent to every model.

141. Common Misconception: "The LLM Knows the Current World"

Not necessarily. A standalone model's learned parameters are not continuously updated with world events. Current information requires:

new training or external information

such as retrieval or tools.

142. Common Misconception: "If the Model Is Local, It Cannot Be Used Over a Network"

False. A model can run on one machine and expose an API or service to other machines on a local network. For example:

This becomes particularly important for businesses with dedicated AI hardware. The model can be physically located on one machine while users interact with it from other devices. We will explore local AI servers later in the course.

143. The Emerging Local AI Infrastructure Model

Imagine a business has:

Employees could interact with the model from their ordinary computers. This separates:

User device

from:

AI compute device

The model can therefore become shared infrastructure. This is an important concept for enterprise local AI.

144. Why the Network Does Not Change What the LLM Is

Whether you run the model:

directly on your laptop

or:

on another machine on your LAN

the model itself is still:

learned parameters + inference computation

The difference is where the computation occurs. This distinction will become useful when we study local AI servers.

145. A Complete Example

Suppose you have a local 8B instruction-tuned model. You ask: Explain why local AI can be useful for businesses.

The system may perform:

1. Loco receives the prompt.
2. The application builds the conversation context.
3. The tokenizer converts text to token IDs.
4. The inference runtime loads/uses the model.
5. Token IDs become embeddings.
6. Transformer layers process the context.
7. The model produces logits.
8. The decoding system selects a token.
9. The token is added to the context.
10. The process repeats.
11. Generated tokens are decoded.
12. Loco displays the answer.

At no point does the model need to explicitly execute a rule saying: If user asks about local AI, produce a paragraph about privacy. Instead, the learned network produces a probability distribution shaped by the prompt.

146. The Most Important LLM Mental Model

If you remember one sentence from this lesson, remember this:

An LLM is a large learned neural network that models patterns in token sequences and can generate language by repeatedly predicting what token should come next given the current context.

That definition is technically useful without pretending the system is either:

magic

or: nothing more than a lookup table.

147. The LLM Mental Model in One Diagram

This is the foundation for everything that follows.

148. Knowledge Check

Try answering these before reading the answers.

Question 1

What is a language model?

Question 2

What does an LLM primarily model?

Question 3

What does "large" refer to?

Question 4

What is next-token prediction?

Question 5

Why can next-token prediction produce surprisingly complex behavior?

Question 6

What is autoregressive generation?

Question 7

What is a Transformer?

Question 8

Is a Transformer the same thing as an LLM?

Question 9

What is pretraining?

Question 10

What is self-supervised learning?

Question 11

What is the difference between a base model and an instruction-tuned model?

Question 12

What is context?

Question 13

Does adding information to the context permanently modify the model parameters?

Question 14

Why can an LLM hallucinate?

Question 15

Does an LLM automatically have access to current internet information?

Question 16

What are logits?

Question 17

What is autoregressive token generation?

Question 18

Why can the same LLM perform translation, summarization, coding, and question answering?

Question 19

What is the difference between an LLM and an AI application?

Question 20

What is the difference between a local LLM and an offline application?

149. Knowledge Check Answers

Answer 1

A language model is a model that represents and predicts patterns in language sequences, commonly by estimating probabilities over possible tokens.

Answer 2

It models patterns in sequences of tokens.

Answer 3

Primarily the scale of the model, especially its large number of learned parameters, along with the scale of training and computation involved.

Answer 4

Given the preceding context, predict the probability distribution of the next token.

Answer 5

Because learning to predict language across enormous and diverse datasets requires learning many linguistic, semantic, structural, factual, and domain-related patterns.

Answer 6

Autoregressive generation repeatedly predicts the next token and then feeds that generated token back into the context for the next prediction.

Answer 7

A Transformer is a neural-network architecture centered around mechanisms such as attention for processing relationships among sequence elements.

Answer 8

No. A Transformer is an architecture. An LLM is a trained language model that may use the Transformer architecture.

Answer 9

Pretraining is the large-scale foundational training process in which the model learns from a broad dataset.

Answer 10

Self-supervised learning uses structure already present in the data to generate training targets, such as predicting a missing or next token.

Answer 11

A base model is primarily pretrained to model its training distribution. An instruction-tuned model has additional training intended to improve instruction following.

Answer 12

Context is the sequence of information available to the model during a particular inference operation, including relevant instructions, conversation, user input, and possibly retrieved information.

Answer 13

No. Context normally influences inference without permanently updating the learned parameters.

Answer 14

Because the model's generation objective does not guarantee factual truth. It can produce plausible continuations that are incorrect.

Answer 15

No. External information requires a retrieval system, web tool, database, API, or another mechanism that supplies current data.

Answer 16

Logits are numerical scores produced before they are transformed into probabilities for token selection.

Answer 17

It is the repeated process of predicting and selecting one token at a time while adding each selected token to the context.

Answer 18

Because all of these tasks can be represented as token sequences, and the model has learned patterns associated with many forms of language and instruction.

Answer 19

The LLM is the learned neural model. The application includes the interface and surrounding systems such as context management, retrieval, tools, files, authentication, and workflows.

Answer 20

A local LLM runs its model inference on local hardware. An offline application additionally avoids network communication during the relevant operation. Local and offline are related but not identical.

150. Practical Exercise: Trace a Prompt

Take this prompt: Explain why GPUs are useful for local AI. Try to trace it through the system.

Your answer should resemble:

Now add a system instruction: Explain technical concepts to a beginner using simple analogies. Ask yourself:

Did the model's parameters change?

No. The context changed. The model now conditions its output on additional information.

151. Practical Exercise: Training vs Inference

Imagine the model has learned the following relationship during training:

"Water freezes at" → "0°C"

Now during inference you ask: At what temperature does water freeze? The model uses its learned parameters to generate a response.

You did not train it during the question. The parameters remain fixed. This is the core difference:

Training: learn parameters

Inference: use parameters

152. Practical Exercise: Context vs Parameters

Suppose you tell the model: The fictional country of Norland has a capital called Riverton. Then ask:

What is the capital of Norland? The model may answer: Riverton.

But that does not prove that:

Riverton

was permanently added to the model's learned parameters. It may simply be using the information present in the current context. This is an important distinction when building document-based AI systems.

153. Practical Exercise: Why Retrieval Helps

Imagine a company has:

10,000 internal documents

and asks: What is our refund policy for enterprise customers? A standalone LLM may not know the company's current internal policy.

A retrieval system can search the documents. Then:

This is one of the foundations of practical enterprise AI.

154. Practical Exercise: Why a Local Model Can Be a Server

Imagine:

PC A: Employee laptop

PC B: Powerful AI machine

Both are connected to the same LAN. The LLM can run on PC B. PC A can send requests to PC B through a local service.

Conceptually:

This means users do not necessarily need the AI model physically loaded onto every workstation. This architecture becomes increasingly important as organizations deploy shared local AI infrastructure.

155. Final Key Takeaways

1. An LLM is a large neural-network language model.

2. A language model models statistical patterns in token sequences.

3. Modern generative LLMs commonly use next-token prediction as a central objective.

4. Autoregressive generation produces text one token at a time.

5. The generated token becomes part of the context for subsequent predictions.

6. Transformers are architectures, while LLMs are trained models.

7. Attention allows token representations to incorporate information from other positions in the context.

8. Pretraining teaches broad language patterns using large datasets.

9. Self-supervised learning allows text itself to provide many training targets.

10. Instruction tuning improves a model's ability to follow user requests.

11. A base model and an instruction-tuned model can have different behaviors even if they share a common foundation.

12. An LLM is not simply a database.

13. An LLM's learned information is distributed across its parameters.

14. Current information generally requires external context, retrieval, tools, or additional training.

15. Context influences inference without normally changing the model's parameters.

16. LLMs can hallucinate because plausible generation is not the same as guaranteed truth.

17. Parameter count is one measure of model scale, not a direct measure of intelligence.

18. Tokenization converts text into numerical token IDs that the model can process.

19. Logits are scores produced for possible next tokens.

20. Generation settings affect how tokens are selected but do not retrain the model.

21. An LLM is one component of a larger AI application.

22. Local inference means the model computation happens on local hardware.

23. Local does not automatically mean offline.

24. Local does not automatically mean secure.

25. Understanding LLMs is the foundation for understanding local AI systems.

156. Final Mental Model

At the end of Lesson 9, you should be able to picture an LLM like this:

The entire system is ultimately built around one deceptively simple idea:

Given the context so far, what token should come next?

The sophistication comes from the enormous learned function used to answer that question. That function has been shaped by:

• architecture
• training data
• optimization
• parameter scale
• post-training
• inference configuration

And when that model is brought onto your own computer, it becomes the core computational component of a local AI system.

157. Preview of Lesson 10

Lesson 10 — What Is a Token?

We have repeatedly used the word token throughout this lesson. Now it is time to stop treating it as a vague technical term. Lesson 10 will go deep into exactly what tokens are and why they matter.

We will examine:

• characters
• words
• subwords
• token IDs
• vocabularies
• tokenizers
• Byte Pair Encoding
• BPE-style tokenization
• SentencePiece-style approaches
• special tokens
• whitespace
• punctuation
• numbers
• code tokenization
• multilingual tokenization
• why different languages can use different numbers of tokens
• why token count affects context
• why token count affects inference cost
• how tokens affect local model performance
• how text becomes integers
• how integers become embeddings
• how tokenization interacts with the model architecture
• why two models can tokenize the same sentence differently
• why tokenization matters when choosing a model
• practical tokenization exercises

By the end of Lesson 10, you will be able to look at a sentence and understand that the LLM is not seeing it as ordinary human-readable text.

It is seeing a sequence of numerical token IDs. And that sequence is the doorway through which language enters the neural network.

Sources and Further Reading

1. Attention Is All You Need — Vaswani et al. (2017)
   https://arxiv.org/abs/1706.03762

2. Language Models are Few-Shot Learners — Brown et al. (2020)
   https://arxiv.org/abs/2005.14165

3. Improving Language Understanding by Generative Pre-Training — Radford et al. (2018)
   https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf

4. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding — Devlin et al. (2018)
   https://arxiv.org/abs/1810.04805

5. Training language models to follow instructions with human feedback — Ouyang et al. (2022)
   https://arxiv.org/abs/2203.02155

6. Deep Learning — Goodfellow, Bengio and Courville
   https://www.deeplearningbook.org/

7. Google Machine Learning Glossary
   https://developers.google.com/machine-learning/glossary

8. Hugging Face Transformers Documentation
   https://huggingface.co/docs/transformers/

9. Hugging Face Course
   https://huggingface.co/learn/nlp-course/

10. llama.cpp — GitHub
    https://github.com/ggml-org/llama.cpp

11. NIST Artificial Intelligence Resource Center
    https://www.nist.gov/artificial-intelligence

These resources provide progressively deeper explanations of Transformer architectures, language-model training, instruction tuning, and practical model implementation.

Lesson Complete

Completed: Lesson 9 — What Is a Large Language Model?

Next: Lesson 10 — What Is a Token?

Course: Local AI 101

Loco Academy: Lesson Status

Completed: Lesson 9: What Is a Large Language Model?

Next: What Is a Token?

Course: Local AI 101