What You Will Learn

By the end of this lesson, you should be able to:

• Explain what a parameter is in a neural network.
• Explain the relationship between parameters and weights.
• Understand why biases are also parameters.
• Explain what labels such as 7B, 8B, and 70B mean.
• Calculate the parameter count of simple neural-network layers.
• Understand where parameters live inside a language model.
• Understand the role of embedding matrices.
• Understand the major parameter-bearing components of a Transformer.
• Understand attention projection matrices at a conceptual and mathematical level.
• Understand why feed-forward or MLP layers contain so many parameters.
• Understand the purpose of normalization parameters.
• Understand the language-model output head.
• Understand tied and untied embeddings.
• Understand why Mixture-of-Experts models can have many total parameters but fewer active parameters per token.
• Estimate how much storage raw parameters require at different numerical precisions.
• Understand why parameter count is not the same thing as file size.
• Understand why parameter count is not the same thing as RAM or VRAM usage.
• Understand the difference between parameters, activations, and the KV cache.
• Understand why quantization changes the storage and memory requirements of a model.
• Understand why a larger parameter count does not automatically mean a better model.
• Understand why individual weights usually cannot be interpreted as individual facts or concepts.
• Connect all of this to choosing and running models locally with Loco.

1. Introduction: The Numbers Behind an AI Model

If you have spent any time looking at local AI models, you have probably seen labels such as:

• 0.5B
• 1B
• 3B
• 7B
• 8B
• 13B
• 14B
• 32B
• 70B
• 405B

The letter B means billion. So an 8B model contains roughly eight billion parameters. That sounds almost meaningless at first.

What exactly is a parameter? Why does a model need billions of them? Are the parameters pieces of knowledge?

Is one parameter equal to one word? Does a 70B model literally know ten times more than a 7B model? Do eight billion parameters require eight gigabytes of RAM?

If you download an 8B model and the file is 5 GB, where did the other three billion parameters go? These are exactly the kinds of questions that this lesson is designed to answer.

Parameters are one of the most important concepts in modern AI because they connect several things you have already learned:

Neural networks learn by changing parameters.

During training, an optimization process repeatedly adjusts parameters so that the network becomes better at its task. During inference, those learned parameters are used to produce outputs.

When you download a language model, the model file is essentially a stored representation of those learned numerical values, together with information describing how they should be interpreted.

This means that when you run a local model, you are not downloading an artificial brain in the science-fiction sense.

You are loading a very large collection of numerical values into a computer and executing a mathematical program around them.

That may sound less exciting than "AI brain."

It is, however, much more useful as a mental model.

2. The Most Important Definition in This Lesson

A parameter is a numerical value inside a machine-learning model that is learned during training. That definition contains three important ideas:

1. It is a number.
2. It is part of the model.
3. Its value is learned or adjusted during training.

A parameter is not necessarily:

• a word
• a sentence
• a fact
• a rule
• a neuron
• a token
• a piece of code
• a document
• a database record

It is a numerical value used by the model's mathematical computations. Consider a very simple equation:

Here:

• x is the input.
• w is a weight.
• b is a bias.
• y is the output.

If the network learns the values of w and b, then both w and b are parameters. For example:

The model might calculate:

Those numbers are parameters. A real neural network is vastly more complicated, but the basic principle remains the same.

3. Parameters Are Learned Numbers

This is worth emphasizing because it is the foundation of the entire subject. Suppose you create a tiny neural network with one weight:

At the beginning of training, perhaps:

The model makes predictions. The predictions are compared against desired outputs. A loss function measures the error.

Backpropagation calculates how changing w would affect the error. An optimizer then updates w. Maybe it becomes:

Later:

Eventually:

The network has "learned" by changing the parameter. The parameter did not suddenly become a sentence. It did not store the phrase:

"The capital of France is Paris."

Instead, its numerical value contributes to a much larger mathematical system. This distinction becomes increasingly important as models become larger.

4. Weights and Parameters

You will often hear the terms weights and parameters used almost interchangeably. They are closely related, but technically they are not always identical terms. A parameter is any learned value in the model.

A weight usually refers to a learned numerical coefficient that determines how strongly one mathematical signal influences another.

For example:

There are two weights:

w1 w2

If the model also has a bias:

then the parameters are:

w1 w2 b

So:

Weights are parameters, but parameters can include things other than what we casually call weights.

Biases are a common example. Some architectures also contain learned scaling or shifting values associated with normalization. The general term "parameter" is therefore safer when discussing the entire learned state of a model.

5. Why Do Neural Networks Need Parameters?

A neural network is fundamentally a parameterized mathematical function. You can think of it like this:

The operations themselves define the structure of the network. The parameters determine how those operations behave. Imagine a blank calculator with a configurable set of knobs.

The architecture tells you:

"Here are the calculations that will happen."

The parameters tell you:

"Here are the numerical settings that determine how those calculations behave."

During training, the model searches for parameter values that make its outputs useful. This is why training is so computationally expensive. You are not simply writing a few rules.

You are optimizing a huge collection of numerical values.

6. A Simple Neural Network Example

Let's construct a tiny network. Suppose we have two inputs:

x1 x2

And one output:

y

The network calculates:

There are:

• 2 weights
• 1 bias

Therefore:

Now imagine ten inputs:

x1 ... x10

with one output. We need:

Therefore:

Now imagine:

1,000 inputs

and:

1,000 output neurons

A fully connected layer needs one weight for every input-output connection. That produces:

plus:

1,000 biases

Total:

1,001,000 parameters

We have reached one million parameters with an extremely simple layer.

Modern language models contain many such mathematical transformations stacked together, along with embeddings, attention projections, feed-forward networks, normalization parameters, and other components.

That is how parameter counts can reach billions.

7. The Dense Layer Formula

A fully connected, or dense, layer is one of the easiest places to understand parameter counting. Suppose a layer has:

N input features M output features

The weight matrix has shape:

M × N

Therefore the number of weights is:

M × N

If each output also has a bias:

M

additional parameters are required. So the total is:

or:

For example:

Weights:

Biases:

256

Total:

131,328 parameters

This simple formula will become useful when we look at Transformer architectures.

8. What Does "7B" Actually Mean?

Now we can return to the model labels. Suppose a model is described as:

7B

That generally means the model has approximately:

7 billion parameters

Likewise:

The number is usually an approximate model-size designation rather than a guarantee that you should count exactly that many values in a particular file.

Different architectures and releases may count parameters differently, especially with structures such as tied embeddings or Mixture-of-Experts.

So treat the label as a useful scale indicator. It tells you:

"This model is operating in roughly this parameter-count class."

It does not tell you everything about the model.

9. Billion Is a Very Large Number

Let's make the scale concrete. One billion is:

1,000,000,000

Eight billion is:

8,000,000,000

Seventy billion is:

70,000,000,000

If you could write one parameter value on every sheet of paper, an 8B model would require an absurd amount of paper. But parameters are not stored as decimal numbers written on paper. Computers store them in binary numerical formats.

That is where memory and storage calculations become important.

10. Parameter Count Is Not the Same as File Size

This is one of the most important distinctions in local AI. Suppose you have:

8 billion parameters

You might incorrectly assume:

That is not necessarily true. The amount of storage required depends on how many bits are used to represent each parameter. For example, if each parameter is stored using 32 bits:

8,000,000,000 × 32 bits

Since:

we get:

8,000,000,000 × 4 bytes

which is:

32,000,000,000 bytes

Approximately:

32 GB

before accounting for other model metadata and storage details. Now suppose each parameter uses 16 bits:

8,000,000,000 × 2 bytes

That is approximately:

16 GB

At 8 bits:

8,000,000,000 × 1 byte

Approximately:

8 GB

At 4 bits:

8,000,000,000 × 0.5 bytes

Approximately:

4 GB

This is the basic reason quantization is so important for local AI.

11. The Basic Storage Equation

A useful first approximation is:

Since:

we can write:

For an 8B model:

32-bit

8B × 32 / 8 = 32 GB

16-bit

8B × 16 / 8 = 16 GB

8-bit

8B × 8 / 8 = 8 GB

4-bit

8B × 4 / 8 = 4 GB

These are simplified theoretical values. Real model files do not always equal these numbers exactly. Quantized formats also have scales, metadata, group information, and other overhead.

The actual memory required to run a model can be higher because the model needs more than just its stored parameter tensors.

Still, this equation gives you an extremely useful first estimate.

12. Bits, Bytes and Why Local AI Discussions Get Confusing

You will encounter:

• bit
• byte
• kilobyte
• megabyte
• gigabyte
• KiB
• MiB
• GiB

A bit is a binary digit:

0 or 1

A byte is:

8 bits

Therefore:

4 bits = half a byte 8 bits = 1 byte 16 bits = 2 bytes 32 bits = 4 bytes

When people casually say:

"A 4-bit 8B model is around 4 GB."

they are applying the simple parameter-storage equation. It is useful, but it should not be interpreted as an exact runtime-memory measurement.

13. Why Parameters Can Be Stored at Different Precisions

During training, models are often represented using numerical formats such as:

• FP32
• FP16
• BF16

For inference, parameters can also be represented using lower-bit formats. Examples include:

• INT8
• 8-bit floating-point formats
• 4-bit quantization schemes
• other specialized quantization formats

The central tradeoff is:

This is not a simple rule that "lower bits always make the model worse."

Modern quantization methods can preserve model quality surprisingly well for many use cases.

The important point is that the stored numerical representation can be changed without changing the basic conceptual architecture of the model.

14. Parameters Are Not Tokens

This is another common beginner mistake. A language model works with tokens. Parameters are different.

A token might represent:

"hello"

or:

"hel"

or:

"lo"

depending on the tokenizer. A parameter is a numerical value such as:

0.0137

or:

-1.842

or another value represented in the model's numerical format. Therefore:

and:

There is no simple one-to-one mapping.

15. Parameters Are Not Individual Facts

Suppose an AI model can answer:

"What is the capital of France?"

and responds:

"Paris."

It would be tempting to imagine that somewhere inside the model there is a parameter saying:

That is not how the model should be conceptualized. Knowledge in a neural network is generally distributed across many parameters and many layers. The model's behavior emerges from interactions among enormous numbers of numerical values.

You should therefore avoid thinking:

There generally isn't a useful interpretation like that. Instead think:

This idea will become especially important in later lessons about language models and transformers.

16. What Does a Weight "Mean"?

This is a surprisingly difficult question. At the simplest level, a weight controls the strength of a mathematical connection. For example:

The weight is:

0.8

A larger magnitude means the input contributes more strongly to that particular calculation, while a smaller magnitude means it contributes less strongly.

But in a modern Transformer, a single numerical weight is usually not something a human can interpret cleanly. A weight might contribute to a representation that participates in:

• token relationships
• syntactic patterns
• semantic relationships
• positional interactions
• factual associations
• style
• multilingual behavior
• code patterns
• reasoning-like transformations

and many other behaviors. The model does not divide itself into neat compartments labeled:

grammar history science sarcasm Python South Africa mathematics

with one parameter assigned to each. Representations are distributed.

17. A Useful Analogy: A Massive Mixing Board

Imagine a gigantic audio mixing console. It has billions of tiny controls. One control by itself may not produce anything meaningful.

But when millions or billions of controls interact, the resulting signal can become highly structured. A neural network is somewhat like this, although the analogy has limits. Each parameter contributes to mathematical transformations.

The model's capabilities emerge from the combined configuration. During training:

After enormous numbers of updates, the model contains a learned numerical configuration. That configuration is what we use during inference.

18. Parameters and the Neural Network From Lesson 6

In Lesson 6, we introduced a simplified artificial neuron:

Let's examine the parameters. The inputs:

x1, x2, ..., xn

are not parameters. They are the values being processed. The weights:

w1, w2, ..., wn

are parameters. The bias:

b

is a parameter. The activation function:

f(...)

is usually part of the architecture rather than a learned parameter. This distinction is extremely useful:

Data flows through the network. Parameters define the learned transformation applied to that data.

19. Parameters vs Hyperparameters

Do not confuse parameters with hyperparameters. They sound almost identical, but they are different.

Parameters

Parameters are learned by the model during training. Examples:

weights biases learned embedding values normalization scales

Hyperparameters

Hyperparameters are settings chosen by the training process or system designer. Examples:

learning rate batch size number of layers hidden dimension number of attention heads training steps optimizer configuration

A learning rate is not something the model normally learns in the same way that it learns its weights. It is a training setting. So:

This distinction will appear repeatedly throughout Local AI.

20. Parameters and Architecture

Another important distinction:

Architecture tells you what kinds of computations exist. Parameters tell you the learned numerical values used by those computations.

Imagine two identical neural-network architectures. They have:

same number of layers same hidden size same attention structure same vocabulary size

but different parameter values. They can behave very differently. For example:

Architecture A + Weights A = Model A

and:

Architecture A + Weights B = Model B

The architecture is the same. The learned parameters differ. This is one reason model checkpoints matter.

A model architecture by itself is not a trained model. You also need the learned parameter values.

21. Where Do the Parameters Live?

In a modern language model, parameters are distributed across many tensors. You may encounter tensors representing:

• token embeddings
• attention projections
• feed-forward layers
• normalization parameters
• output projections
• other architecture-specific components

A model file may contain thousands of tensors. A tensor is essentially a structured collection of numerical values. For example:

Tensor shape: [4096, 4096]

contains:

4096 × 4096

values. That is:

16,777,216 values

If each value represents a parameter, that single matrix can contain more than sixteen million parameters. Modern models contain many such matrices. This is how billions become plausible.

22. Embeddings: One of the First Huge Parameter Tables

Language models need a way to turn tokens into numerical representations. This is commonly handled using an embedding matrix. Suppose a tokenizer has:

50,000 tokens

and the model uses:

4,096-dimensional embeddings

The embedding matrix has approximately:

50,000 × 4,096

values. That equals:

204,800,000

values. So the embedding layer alone can contain more than 200 million parameters. This is a major reason vocabulary size and hidden dimension matter to model size.

23. The Embedding Matrix

Conceptually:

Each token is associated with a learned vector. For example:

"cat"

might map to something conceptually like:

[0.12, -0.42, 0.07, ...]

The real vector may contain thousands of dimensions. The individual numbers are learned parameters. The vector as a whole represents the token in a learned numerical space.

We will study embeddings in much greater depth later.

24. Attention Contains Parameters Too

Transformers use attention mechanisms. The original Transformer architecture introduced learned projection matrices associated with:

• queries
• keys
• values
• output projections

You can think of the process conceptually as:

Each projection is represented by learned numerical matrices. Those matrices contain parameters. This means that attention is not simply a mathematical formula with no learned information.

The formula defines the computation. The learned projection matrices determine how the model transforms representations before and after attention.

25. Attention Projection Parameter Counting

Suppose a Transformer has hidden dimension:

A simplified attention block may use four dense projections, each roughly:

4096 × 4096

The number of values in one matrix is:

4096 × 4096 = 16,777,216

Four such matrices contain approximately:

67,108,864

parameters. That is roughly:

67 million

for one attention block in this simplified example. Real architectures can differ.

Modern models may use grouped-query attention, multi-query attention, separate dimensions for query and key/value projections, biases or no biases, and other architectural variations.

So do not use this simplified calculation as a universal parameter-count formula. Use it to understand the mechanism.

26. The Feed-Forward Network Is Huge

Transformer blocks usually contain a feed-forward or MLP component. A simplified version can be represented as:

If the hidden dimension is:

4096

and the intermediate dimension is:

16,384

one projection is approximately:

4096 × 16,384

which is:

67,108,864

parameters. A second projection of similar size adds another:

67,108,864

So even a simple two-matrix MLP can contain roughly:

134 million parameters

before accounting for biases or architectural differences. This is one reason Transformer blocks can contain enormous numbers of parameters.

27. Why MLPs Matter So Much

Attention gets most of the attention in discussions about Transformers. But the feed-forward network is also extremely important. A simplified Transformer block can look like:

Input
  |
  +-----------------------------+
  |                             |
  v                             |
Attention                        |
  |                             |
  v                             |
Residual -----------------------+
  |
  v
Normalization
  |
  v
MLP / Feed-Forward
  |
  v
Residual
  |
  v
Output

The exact ordering varies by architecture. The MLP performs learned transformations on the representations. Its matrices can contain a very large fraction of the model's parameters.

This is why understanding parameter counts requires looking beyond attention.

28. Normalization Parameters

Modern neural networks frequently use normalization techniques. One common family is Layer Normalization. Depending on the implementation, normalization may contain learned parameters such as:

scale bias

If the hidden dimension is:

4096

then a normalization layer with one learned scale vector and one learned bias vector might contain:

parameters. Compared with a 100-million-parameter MLP, that is tiny. Nevertheless, they are still learned parameters.

Some modern architectures use normalization schemes with different parameter structures. Again, architecture matters.

29. The Output Head

A language model eventually needs to convert its internal representation into scores for possible next tokens. Suppose the model has:

A simple output projection might have:

4096 × 50,000

parameters. That is:

204,800,000

parameters. So the output layer can itself be hundreds of millions of parameters. This is sometimes called the language-model head or LM head.

The model uses the output representation to produce logits for possible tokens. Conceptually:

We will explore this process in much greater depth in later lessons.

30. Tied Embeddings

Some language models tie the input embedding weights and output projection weights. Conceptually:

Input embedding matrix
          |
          +------------------+
                             |
                             v
                       Model processing
                             |
                             v
Output projection ----------+

The same learned matrix can be reused. This can reduce the number of independent parameters. If the input embedding and output head would otherwise each contain:

204.8 million

parameters, tying them can avoid storing two separate matrices.

This is one reason simple parameter-count estimates do not always perfectly match what you might calculate from a model architecture diagram.

Architecture details matter.

31. Parameter Count Is an Accounting Problem

When researchers describe a model as having a certain parameter count, they are effectively answering:

"How many learned numerical values does this model contain under the relevant parameter-counting convention?"

That means parameter counting can depend on:

• vocabulary size
• hidden dimension
• number of layers
• intermediate dimension
• number of attention projections
• attention design
• normalization design
• whether biases are used
• whether embeddings are tied
• whether the model is dense
• whether it uses experts
• architecture-specific components

This is why two models with similar labels can have different internal structures.

32. A Simplified Transformer Parameter Budget

Let's build a conceptual model. Suppose a model has:

Vocabulary: 50,000 Hidden size: 4,096 Layers: 32 MLP size: 11,008

We will not try to reproduce a specific production model exactly. Instead, we will use this to understand where parameters come from.

Embeddings

50,000 × 4,096 ≈ 205 million

Attention

A simplified four-projection estimate:

4 × 4,096 × 4,096 ≈ 67 million

per layer. Across 32 layers:

MLP

If there are two major matrices:

2 × 4,096 × 11,008 ≈ 90 million

per layer. Across 32 layers:

Already we are in the multi-billion-parameter range. Add:

• output layers
• normalization
• biases if used
• architectural details
• possible gating
• embeddings
• other components

and the model can reach several billion parameters. This exercise demonstrates something important:

Billions of parameters do not require billions of separate conceptual features. They naturally arise from large matrices repeated across many layers.

33. Why Repeating Layers Produces Billions of Parameters

A language model may contain dozens of Transformer blocks. Suppose one block contains:

100 million parameters

Then:

32 blocks × 100 million

3.2 billion parameters

If the model has:

60 blocks

the same rough block size gives:

6 billion parameters

This is a simplified example, but it explains the scale. Modern models are not necessarily one enormous matrix. They are usually collections of many parameterized components.

34. Parameter Distribution

It is useful to imagine a model as a parameter budget. For example:

+---------------------------------------+
|              Model                    |
|                                       |
| Embeddings             ███            |
| Attention              ███████        |
| MLP / Feed-Forward     ███████████    |
| Normalization          ▏              |
| Output Head            ██             |
| Other Components       █              |
|                                       |
+---------------------------------------+

The exact distribution varies dramatically by architecture. The important idea is:

The parameter count is distributed across many tensors and layers.

This also explains why modifying one part of a model can have very different consequences from modifying another.

35. Parameters and Tensors

You will frequently see the word tensor in model files and AI software. A tensor is a general numerical data structure. You can think of it as:

scalar = one number vector = one-dimensional collection matrix = two-dimensional collection tensor = generalized multi-dimensional collection

For example:

[3.2]

is a scalar.

[1.0, 2.0, 3.0]

is a vector.

[ [1, 2], [3, 4] ]

is a matrix. Higher-dimensional structures are tensors. Model parameters are often stored in tensors.

Therefore:

A model can contain thousands of tensors, depending on architecture and how the checkpoint is represented.

36. Why Parameters Are Usually Floating-Point Numbers

During training, neural networks need numerical precision. Suppose a parameter is:

0.123456789

and another is:

0.123456700

Those values are extremely close. Small numerical differences can matter because training repeatedly applies tiny updates. Floating-point formats provide a practical way to represent a large range of numerical values.

Common formats include:

FP32 FP16 BF16

The exact format determines how many bits are used and how those bits are interpreted. This is why "16-bit model" does not simply mean that the model has fewer parameters. It means each stored numerical value is represented with fewer bits.

37. FP32

FP32 uses:

32 bits

per value. Therefore:

4 bytes per parameter

For 8 billion parameters:

8B × 4 bytes ≈ 32 GB

This is a rough parameter-storage estimate. FP32 is useful when high numerical precision is desirable, but storing very large models at FP32 is expensive. This is one reason lower-precision representations are so important for modern AI.

38. FP16

FP16 uses:

16 bits

per value. Therefore:

2 bytes per parameter

For an 8B model:

8B × 2 bytes ≈ 16 GB

Again, this is only the raw parameter estimate. Runtime memory can be higher.

39. BF16

BF16 is another 16-bit floating-point format. It uses:

16 bits

per value. Its bit layout differs from FP16 in a way that gives it a different balance between range and precision. BF16 has become important in machine-learning training because of its numerical properties and hardware support.

For the simple storage calculation:

So an 8B model represented at BF16 requires approximately:

16 GB

for the raw parameter values.

40. INT8

An 8-bit integer representation uses:

8 bits

per stored value. That is:

1 byte

per parameter. An 8B model therefore has a raw parameter-storage requirement of approximately:

8 GB

Additional quantization metadata and runtime requirements mean the final file or memory requirement may differ.

41. 4-Bit Quantization

At approximately 4 bits per parameter:

So:

8B × 0.5 bytes ≈ 4 GB

This is why you will encounter 4-bit versions of models that are dramatically smaller than their full-precision counterparts.

But remember:

4-bit does not mean the model has four billion parameters.

An 8B model quantized to 4 bits still represents approximately eight billion parameters. The representation of those parameters has changed.

42. Quantization Does Not Delete the Parameters

Imagine a model has:

8,000,000,000 parameters

Quantization does not necessarily transform that into:

4,000,000,000 parameters

Instead, it changes how the parameter values are represented. Conceptually: Before:

Parameter 1 = high-precision number Parameter 2 = high-precision number Parameter 3 = high-precision number ...

After:

Parameter 1 = lower-precision representation Parameter 2 = lower-precision representation Parameter 3 = lower-precision representation ...

The exact implementation of quantization can be more sophisticated than simply rounding every number independently. Modern quantization methods may use:

• groups
• scales
• zero points
• block structures
• specialized numerical formats

But the conceptual principle remains:

Same learned model, more compact numerical representation.

43. Why Quantization Matters for Local AI

Cloud data centers can contain enormous amounts of memory and specialized hardware. A typical personal computer has much tighter resource constraints. Suppose your computer has limited RAM.

An FP16 model might be too large. A quantized version might fit. For example, a rough conceptual progression for an 8B model is:

FP32 ≈ 32 GB raw parameters FP16 ≈ 16 GB INT8 ≈ 8 GB 4-bit ≈ 4 GB

This does not mean:

It means that numerical representation creates a tradeoff between:

• memory
• storage
• bandwidth
• speed
• numerical fidelity
• hardware compatibility

This tradeoff is fundamental to practical local AI.

44. Parameter Count vs Model File Size

Let's say you download:

An 8B model

and the file is:

approximately 5 GB

You might ask:

"Why isn't it 8 GB?"

Because the model may be quantized. For example:

8 billion parameters × roughly 4 bits ≈ roughly 4 GB

Then additional data and quantization metadata can push the actual file size upward. A file around 5 GB is therefore not contradictory. Likewise, an 8B model could exist in multiple representations:

8B FP16 8B 8-bit 8B 6-bit 8B 5-bit 8B 4-bit

They all represent roughly the same parameter count. They can have dramatically different file sizes.

45. Parameter Count vs RAM

Even if the model file is 5 GB, you should not automatically conclude:

"I need exactly 5 GB of RAM."

Runtime memory can include:

• model parameters
• temporary computation buffers
• activations
• KV cache
• runtime overhead
• tokenizer data
• metadata
• other application memory
• operating-system memory

Therefore:

Model file size ≠ Exact runtime RAM requirement

The file size is useful. It is not the complete answer.

46. Parameters vs Activations

This distinction is critical.

Parameters are learned values stored as part of the model.

Activations are intermediate values produced while the model processes an input.

Imagine:

The parameters are relatively stable. The activations change depending on the input. For example:

Prompt A

produces one set of activations.

Prompt B

produces another. Therefore:

This is an extremely important distinction when understanding memory usage.

47. Parameters vs KV Cache

For autoregressive language models, another major memory consumer is the KV cache. During generation, the model repeatedly processes a sequence of tokens. Attention needs information about previous tokens.

Rather than recomputing everything from scratch at every step, inference systems can store key and value information from previous processing.

That stored information is the KV cache. Conceptually:

The KV cache grows as the context becomes longer. Therefore:

This is one reason a model that fits comfortably for a short conversation can require substantially more memory when given a very long context.

48. The Three Different Memory Questions

When working with local AI, ask three separate questions.

Question 1: How large is the model?

This is largely related to:

parameter count + numerical representation

Question 2: How much memory does the model need to run?

This includes:

parameters + runtime overhead + activations + KV cache + other memory

Question 3: How much storage does the model file occupy?

This is approximately:

parameters × bits per parameter + metadata + quantization overhead

These are related but not identical.

49. Why a 4 GB Model Does Not Mean You Only Need 4 GB of RAM

Suppose you have:

4 GB model file

The model still needs to be loaded into memory. Then inference requires additional memory. Imagine, purely as a conceptual example:

Model parameters 4.2 GB KV cache 0.8 GB Runtime buffers 0.4 GB Application overhead 0.3 GB

Total 5.7 GB

These numbers are illustrative, not a universal formula. The point is that the model file is only one component. This is why a computer with exactly the same amount of RAM as the model file is not necessarily sufficient.

50. Why More Parameters Usually Increase Resource Requirements

If you double the parameter count while keeping representation precision similar, the raw storage requirement roughly doubles.

For example:

8B at 4 bits ≈ 4 GB raw parameter storage

while:

16B at 4 bits ≈ 8 GB raw parameter storage

Likewise:

32B at 4 bits ≈ 16 GB raw parameter storage

and:

70B at 4 bits ≈ 35 GB raw parameter storage

These are simplified estimates. They illustrate the fundamental scaling relationship. More parameters generally mean:

• more storage
• more memory pressure
• more data movement
• more computation

although architecture and hardware implementation strongly affect actual performance.

51. Parameter Count and Computation

A model with more parameters generally has more mathematical work to perform. Imagine:

It would be unreasonable to expect the 70B model to have exactly the same computational requirements as the 8B model. More learned matrices and larger transformations usually mean more operations. However, parameter count alone does not completely determine inference speed.

Performance also depends on:

• architecture
• quantization
• context length
• hardware
• memory bandwidth
• compute throughput
• implementation
• batch size
• number of active parameters
• offloading strategy
• prompt length

This is why you should not treat "B" as a direct speed rating.

52. Parameter Count Is Not an Intelligence Score

This deserves its own section. Suppose:

It is tempting to conclude:

That conclusion is not justified. Parameter count tells you approximately how many learned values the model contains. It does not directly measure:

• reasoning ability
• factual accuracy
• coding ability
• language ability
• instruction following
• mathematical performance
• hallucination rate
• efficiency
• usefulness for a specific task

A smaller model can outperform a larger one on a particular benchmark or task. A larger model can have capabilities that a smaller model lacks. The relationship is complicated.

53. Why Architecture Matters

Consider two models:

Model A: 8B parameters Architecture A Training data A Training process A

Model B: 8B parameters Architecture B Training data B Training process B

They have similar parameter counts. They can still behave very differently. Why?

Because parameters are only one part of the story. Model behavior depends on:

Architecture + Parameters + Training data + Training objective + Optimization + Instruction tuning + Alignment / post-training + Inference configuration

Parameter count is therefore one dimension of model design, not a complete description.

54. Why Model Families Matter

You may encounter several model sizes within a single family:

3B 8B 14B 32B 70B

A model family typically shares some architectural and training philosophy while changing scale. This makes the parameter number useful as a rough indicator of model scale. But even within one family, different sizes can have different:

• memory requirements
• performance
• speed
• context capabilities
• hardware requirements

The best way to select a model is therefore not:

"Pick the biggest number."

Instead, think in terms of requirements:

55. Mixture-of-Experts Changes the Picture

Dense models are relatively straightforward conceptually. If a dense model has:

70B parameters

then those parameters are generally part of the computation path for each token, subject to architectural details. Mixture-of-Experts, or MoE, models introduce a different idea.

Instead of having every token use every parameter in the same way, the model can contain multiple expert networks and route different tokens to selected experts.

Conceptually:

Only some experts may be activated for a given token. This means a model can have:

Large total parameter count

while having:

Smaller active parameter count per token

56. Total Parameters vs Active Parameters

Imagine an MoE model with:

100B total parameters

but only a subset of experts is activated for each token. You might see a description such as:

100B total 20B active

The exact figures vary by architecture. The important concept is:

Total parameters ≠ Parameters used for every token

This affects:

• computation
• memory
• inference performance
• model loading
• hardware requirements

However, an MoE model still needs access to the model components required by the implementation. So you cannot automatically conclude:

"Only the active parameters exist in memory."

Memory and compute are separate questions.

57. Why MoE Is Important for Local AI

When evaluating local models, you may eventually encounter a model whose parameter label seems enormous but whose active parameter count is much smaller.

This can be confusing if you have only learned the dense-model interpretation. The better mental model is:

Dense model: Most model parameters participate in each token's computation.

MoE: A routing mechanism selects a subset of expert parameters for each token.

The model still contains a large learned parameter space. The computation is selectively routed. We will return to this in the advanced section of Local AI 101.

58. Parameters Are Distributed Knowledge

Let's return to the question:

"Where is the model's knowledge?"

There is no single file inside the model saying:

knowledge.txt

There is no database table such as:

fact_id | fact | confidence

Instead, training adjusts enormous numbers of parameters. The result is a distributed statistical representation. A useful conceptual diagram is:

This is why deleting or changing one arbitrary weight does not normally correspond to deleting one fact. The information is distributed.

59. Why Individual Parameters Are Hard to Interpret

Suppose a model has:

8,000,000,000 parameters

and you inspect one:

0.017382

What does it mean? Usually, by itself, not much. You could change it slightly.

Maybe the model's output changes. Maybe it barely changes. Maybe it affects several downstream representations.

But you generally cannot say:

Parameter #2,391,824 means "Paris."

Modern neural networks use distributed representations and nonlinear transformations. Meaning is often encoded across patterns of many values. This is one reason interpretability is an active research area.

60. Parameters and Representations

A useful progression is:

The farther you move down that chain, the more meaningful the interpretation becomes. A single weight is usually difficult to interpret. A collection of activations or a learned representation can sometimes reveal useful structure.

A complete model can exhibit recognizable capabilities. This is why asking:

"Which parameter contains the knowledge of chemistry?"

is usually the wrong question.

61. Parameters and Embeddings Are Related but Not Identical

An embedding is a vector representation. The values in an embedding matrix are parameters. For example:

The vectors are learned. They are not static dictionary definitions. A token's embedding is one starting representation that the rest of the network transforms through many layers.

The meaning available to the model is not simply "stored inside the embedding vector."

The deeper layers build increasingly contextual representations.

62. Static Parameters, Dynamic Computation

This gives us another useful mental model. The learned parameters are relatively static during inference:

But the computations are dynamic:

The same model parameters can produce completely different outputs because the input and resulting activations change. This distinction is fundamental.

63. Training Changes Parameters

During training:

This cycle happens repeatedly. The parameter values gradually move toward configurations that reduce the training objective. That is what "learning" means at the mathematical level.

The model is not rewriting an English knowledge base. It is changing numerical values.

64. Inference Uses Parameters

During inference:

There is normally no parameter update. If you run a model locally and ask it:

"Explain photosynthesis."

the model is not normally permanently modifying itself after generating the answer. It uses the learned parameters to perform the computation. This is why:

and: Inference does not normally teach the model new permanent knowledge. That distinction connects directly to Lesson 5.

65. What Happens When You Load a Model?

Imagine you have a quantized model file. Loco conceptually needs to:

1. Locate the model file.
2. Read its metadata.
3. Identify its architecture.
4. Allocate memory.
5. Load model tensors.
6. Prepare the inference runtime.
7. Initialize the required computation structures.
8. Accept an input.
9. Tokenize the input.
10. Run inference.
11. Produce output tokens.

The important part for this lesson is step 5:

The learned numerical parameters must become accessible to the inference process.

The runtime then performs mathematical operations using those values.

66. A Model File Is Not Just "A Brain"

When you download a model, you are not downloading a magical executable intelligence. A model file generally contains some combination of:

• learned tensors
• numerical representations
• metadata
• architecture information
• tokenizer-related information or references
• quantization information
• other implementation-specific data

The runtime is responsible for interpreting these structures. This is another reason to distinguish:

Model

from:

Inference software

The model contains learned values. The runtime executes the computations.

67. Loco and Parameters

This concept is central to using Loco. When Loco presents a model for local use, the important question is not merely:

"How big is the download?"

You should also think about:

Parameter count + Quantization + Context requirements + Available RAM + Available VRAM + CPU/GPU capabilities + Expected performance

For example, two model files may both appear attractive. One might be:

8B, 4-bit

while another is:

14B, 4-bit

The second one generally requires more memory and computation. But the question is whether its additional capability is worth those resources for your particular workload. This is why model selection is an engineering decision rather than a simple "biggest model wins" contest.

68. Why Your Computer's Hardware Matters

Suppose you have limited RAM and no dedicated GPU. A model with billions of parameters can still be possible to run depending on:

• model size
• quantization
• context
• inference runtime
• CPU
• available system memory
• operating system overhead

But there is a practical limit. If the model requires more memory than your computer can provide comfortably, you may experience:

• inability to load
• heavy swapping
• extremely poor responsiveness
• low token generation speed
• system instability

This is why understanding parameters helps you make realistic decisions before downloading huge models.

69. RAM and VRAM

Let's revisit two pieces of hardware.

RAM

System memory. It is used by:

• the operating system
• applications
• local AI runtimes
• model data
• other processes

VRAM

Memory attached to a GPU. It is particularly useful for:

• model weights
• activations
• GPU computation
• other graphics/compute workloads

If a model is partially or fully loaded onto a GPU, the available VRAM becomes an important constraint.

If the model does not fit entirely into VRAM, some systems can keep part of it in system RAM and move data between the CPU and GPU.

This can work, but data movement can affect performance.

70. Memory Bandwidth Matters

It is easy to focus entirely on compute power. But moving billions of parameter values through memory is also important. Imagine an inference system that needs to access a huge amount of model data repeatedly.

The processor can only work with data once it receives it. Therefore:

The speed of that data movement matters.

This is one reason quantization can help local inference even when the raw arithmetic reduction is not the only benefit.

Smaller representations can reduce the amount of data that must be moved.

71. Parameter Count and Token Generation

When a language model generates text, it repeatedly performs mathematical transformations. Conceptually:

The model's parameters participate in those transformations. The larger and more complex the model, the more computation may be required per token. But the exact performance depends heavily on architecture and implementation.

Therefore:

Parameter count

is useful for understanding scale. It is not a reliable standalone measurement of:

tokens per second

72. Why the Same Parameter Count Can Behave Differently

Imagine two 8B models. They both have:

but one might have:

• a different architecture
• different tokenizer
• different training data
• different training objective
• different instruction tuning
• different context design
• different attention structure
• different quantization

The models can therefore have different:

• coding ability
• writing quality
• multilingual ability
• factual reliability
• reasoning behavior
• speed
• memory requirements

This is another reason parameter count is only one part of model evaluation.

73. A More Complete Model Description

Instead of saying:

"I have an 8B model."

a more useful description might be:

Model family: [model family] Parameter class: ~8B Architecture: Transformer-based Context: [supported context] Precision: 4-bit quantized Format: GGUF Runtime: [inference runtime] Hardware: CPU / GPU

This tells you far more. A model's parameter count is one coordinate in a larger technical description.

74. GGUF and Parameter Storage

For local AI, you will often encounter GGUF model files. GGUF is a model-file format associated with the llama.cpp ecosystem. It can store model tensors along with metadata needed to help an inference system interpret the model.

The important lesson here is not that GGUF changes the parameter count. It does not turn:

8B

into:

4B

Instead, a GGUF file can represent a model in a particular numerical format, including quantized variants. So:

8B model + 4-bit quantization + GGUF representation

can result in a model file that is much smaller than an FP16 representation of the same general model.

75. Why Model Files Can Have Different Sizes

Suppose the same model is available in several files:

model-f16.gguf model-q8.gguf model-q5.gguf model-q4.gguf

The underlying model family may be the same. The parameter count may be the same. The files can nevertheless have different sizes because the parameter representation differs.

Conceptually:

The exact quality and performance tradeoffs depend on the quantization method and workload.

76. A Parameter Calculation Exercise

Let's calculate a simple dense layer. Suppose:

The weight matrix contains:

1,024 × 2,048

Calculate:

weights. If there is one bias per output:

Total:

2,099,200 parameters

So one ordinary dense layer contains more than two million parameters. And this is tiny compared with modern language models.

77. Storage for That Layer

Suppose those:

2,099,200 parameters

are stored in FP16. Each parameter uses:

2 bytes

Therefore:

2,099,200 × 2

4,198,400 bytes

Approximately:

4.2 MB

At 4 bits:

2,099,200 × 0.5

1,049,600 bytes

Approximately:

1.05 MB

This simple exercise demonstrates the relationship between:

parameter count

and:

storage per parameter

78. An 8B Example

Let's estimate raw parameter storage for an 8B model.

At FP32:

8B × 4 bytes = 32,000,000,000 bytes ≈ 32 GB decimal

At FP16:

8B × 2 bytes = 16,000,000,000 bytes ≈ 16 GB

At 8-bit:

8B × 1 byte = 8,000,000,000 bytes ≈ 8 GB

At 4-bit:

8B × 0.5 bytes = 4,000,000,000 bytes ≈ 4 GB

Again:

These are raw parameter-storage estimates, not guaranteed final file sizes or exact runtime-memory requirements.

79. Why "0.5 Bytes per Parameter" Is Not Literally a Normal Byte

This can initially sound strange. A byte contains eight bits. A 4-bit representation uses half of one byte's worth of information per parameter.

Computers therefore pack multiple low-bit values into bytes or larger blocks. For example, two 4-bit values can fit into one byte:

[ 4 bits ][ 4 bits ]

This is one reason quantized formats need packing schemes and metadata.

You are not creating a physical object called "half a byte."

You are packing information efficiently.

80. Quantization Overhead

The simple equation:

parameters × bits / 8

assumes perfectly efficient storage. Real quantization formats can require additional information. For example, a block of values might use:

quantized values + scale + possibly zero point + other metadata

Therefore:

Actual file size > theoretical parameter payload

in many cases. This is completely normal. The simplified formula is still useful for estimating scale.

81. Parameter Count vs Checkpoint Size

Another term you may encounter is checkpoint. A checkpoint is a saved state of a model during or after training. Depending on the training context, a checkpoint may contain more than just the final inference weights.

Training checkpoints can include things such as:

• model parameters
• optimizer state
• scheduler state
• training metadata
• other information needed to resume training

This means a training checkpoint can be substantially larger than the inference model that you eventually download. For local inference, the distributed model file is usually much more relevant than a full training checkpoint.

82. Why Training State Can Be Much Larger

Suppose your model parameters are stored in one numerical format. The optimizer may also need additional state associated with those parameters. For some optimizers, that can require multiple extra numerical tensors.

Therefore:

Inference model

and:

Training checkpoint

can have very different storage requirements. This helps explain why training enormous models is so demanding. The training system is not simply storing one copy of the model.

It may be maintaining additional numerical state required for optimization.

83. Parameters and Gradients

During training, a parameter can have an associated gradient. For example:

Parameter: w = 0.52

Gradient: dw = -0.13

The gradient tells the optimizer how the loss changes with respect to that parameter. The optimizer may then update:

w

using that information. During ordinary inference, you generally do not need to maintain gradients. This is another reason inference is much less memory-intensive than training.

84. Parameters and Backpropagation

Recall from Lesson 7:

The backward pass calculates derivatives for parameters. If a model has billions of parameters, that means an enormous number of values may need to be involved in training. This is one of the fundamental reasons training large models requires substantial compute and memory infrastructure.

Inference has a much simpler job: Use the learned parameters.

85. Why Parameters Start Out Random-ish

At the beginning of training, neural-network parameters are typically initialized according to carefully designed initialization methods.

They are not usually initialized as a giant database of useful knowledge. The network begins with parameter values that are not yet useful for the target task. Training transforms them.

Conceptually:

This is another reason it is misleading to think of a model as a database that is simply filled with facts.

86. Parameter Count and Model Capacity

A larger number of parameters generally gives a model more capacity to represent complex functions. That does not mean:

It means: More parameters can provide more representational capacity. Whether that capacity becomes useful depends on:

• training data
• optimization
• architecture
• training objective
• model design
• post-training
• data quality

A huge model trained badly can be less useful than a smaller model trained well.

87. Capacity Is Not the Same as Knowledge

Imagine two models:

The 70B model has much greater parameter capacity. But that does not mean it necessarily contains:

There is no simple conversion like that. Parameter count is not a direct knowledge meter. It is better understood as a rough measure of the scale of the learned numerical system.

88. Why Bigger Models Can Be Better

Larger models can sometimes learn more complex patterns. They may have more capacity to represent:

• language structure
• long-range relationships
• code patterns
• mathematical relationships
• multilingual patterns
• domain-specific information

But the benefit depends on the quality and quantity of training. A model cannot magically learn useful information that was never represented in its training process. And more capacity can also increase computational requirements.

89. Why Smaller Models Matter

Small models are extremely important for local AI. A smaller model can provide:

• lower memory requirements
• faster loading
• lower hardware requirements
• easier deployment
• better suitability for laptops
• lower energy consumption
• easier edge deployment

For many tasks, a compact model can be entirely sufficient. This is particularly important for the local-AI ecosystem because local AI is not only about enormous models. It is about matching:

model capability

to:

available hardware

and:

actual task

90. The Engineering Tradeoff

Local AI can be viewed as a resource optimization problem. You are balancing:

Capability ↕ Model size ↕ Memory ↕ Speed ↕ Quality ↕ Hardware

For example:

versus:

The correct choice depends on the application.

91. A Practical Loco Mental Model

When looking at a model inside a local-AI platform such as Loco, you can mentally parse the model information in layers.

Layer 1: What is the model?

Model family

Layer 2: How large is it?

Parameter count

Layer 3: How is it stored?

Precision / quantization

Layer 4: How much context can it process?

Context window

Layer 5: What hardware can run it?

CPU GPU RAM VRAM

Layer 6: What can it do?

Chat Coding Reasoning Vision Tool use RAG etc.

Layer 7: What is the practical performance?

Speed latency quality memory usage

This is a much better way to evaluate models than looking only at the "B" number.

92. A Worked Example: Choosing Between Two Models

Imagine two hypothetical models:

Model A 8B parameters 4-bit

and:

Model B 14B parameters 4-bit

The rough raw parameter-storage estimates are:

Model A: 8B × 0.5 bytes ≈ 4 GB

Model B: 14B × 0.5 bytes ≈ 7 GB

Model B therefore has a significantly larger raw parameter footprint. But you should not stop there. You would also ask:

What is the context length? What is the architecture? How much RAM do I have? How much VRAM do I have? What runtime am I using? What task am I solving? What speed do I need?

Only after considering those factors can you determine whether the larger model is practical.

93. Why Model Metadata Matters

A good model library should expose useful technical information. At minimum, model information can include:

Model name Parameter count Architecture Context length Quantization File format Approximate file size Supported modalities License Developer Base/instruction status

This is especially important for local AI because users need to make hardware decisions. A model name alone is not enough.

94. Why Hardware Recommendations Should Be Approximate

It is tempting to say:

"An 8B 4-bit model requires exactly 6 GB of RAM."

That is usually too simplistic. Actual requirements depend on:

• model architecture
• quantization
• context length
• runtime
• offloading
• batch size
• operating system
• other applications
• implementation details

A better approach is to provide an estimate and explain the variables. For example:

Raw parameter storage: ≈ 4 GB

Expected runtime memory: Higher than the raw parameter size because of context, buffers, and other overhead.

This is more technically honest.

95. Why the "B" Number Is Still Useful

Despite all these caveats, parameter count remains extremely useful. If someone says:

3B

versus:

70B

you immediately understand that you are dealing with dramatically different model scales. It gives you a first approximation for:

• storage
• memory
• compute
• deployment difficulty

It is simply not the only factor. Think of parameter count as:

A scale indicator, not a quality score.

96. Common Misconception: "8B Means Eight Billion Words"

False. 8B means approximately:

8 billion learned parameters

It does not mean:

8 billion words

The training dataset can contain vastly more or fewer tokens than the parameter count. Parameter count and training-token count are separate quantities.

97. Common Misconception: "One Parameter Stores One Fact"

False. Modern neural networks use distributed representations. A fact or capability can be reflected in patterns across many parameters and layers.

There is generally no clean mapping:

one fact → one parameter

98. Common Misconception: "A 70B Model Is Ten Times Smarter Than a 7B Model"

Not necessarily. A 70B model has roughly ten times the parameter count. That does not imply ten times the intelligence or ten times the useful knowledge.

Performance is multidimensional.

99. Common Misconception: "A 4-Bit Model Is a 4B Model"

False. These describe different things.

8B

describes parameter scale.

4-bit

describes numerical representation. So:

8B 4-bit

means approximately:

8 billion parameters represented using a roughly 4-bit quantized format

100. Common Misconception: "A 5 GB Model Only Needs 5 GB of RAM"

Not necessarily. Runtime memory can include:

model parameters + KV cache + activations + runtime buffers + other processes

Therefore actual memory requirements can exceed file size.

101. Common Misconception: "Parameters Are the Model's Code"

Not exactly. The model architecture and inference runtime define how the numerical values are used. The parameters are the learned numerical state.

Think:

Together they produce the functioning model.

102. Common Misconception: "The Model Changes Its Parameters Every Time I Chat With It"

Normally, no. Ordinary inference uses the existing parameters.

Your conversation can affect the model's temporary context and activations, but that is different from permanently updating the learned weights.

A system can be designed to perform learning or fine-tuning, but ordinary prompting is not equivalent to retraining the model.

103. Common Misconception: "More Parameters Always Means Slower"

Not in a simple absolute sense. Larger models generally require more computation, but actual speed depends on:

• hardware
• architecture
• quantization
• runtime
• memory bandwidth
• context
• batching
• active parameters in MoE systems

So parameter count is an important factor, not a complete speed equation.

104. Common Misconception: "Quantization Changes the Model Into a Smaller Model"

Usually, that is not the right way to describe it. A better description is:

Same general learned model + different numerical representation

Quantization reduces storage and memory requirements while attempting to preserve useful behavior.

105. A Deep Mental Model: Parameters as the Learned Configuration

If you remember only one conceptual definition from this lesson, remember this:

The parameters are the learned numerical configuration of the neural network.

The architecture says:

"What computations should exist?"

The parameters say:

"What numerical configuration should those computations use?"

The input provides:

"What problem are we processing right now?"

The inference process produces:

"What does the model output?"

This gives us:

That is the core of a neural-network model.

106. Another Deep Mental Model: The Model as a Learned Function

Mathematically, you can think of a neural network as:

where:

• x is the input.
• y is the output.
• f is the neural-network computation.
• θ represents the learned parameters.

During training:

Find θ

such that:

f(x; θ)

produces useful outputs. During inference:

Given θ and x, calculate f(x; θ).

This notation is incredibly powerful because it separates the two ideas:

When you prompt a local model, you are providing x. You are not normally changing θ.

107. Parameters and the Entire Local AI Stack

You can now connect almost everything we have learned so far.

This is the journey from training data to the model running on your computer. Loco sits at the practical local-AI layer where users manage and use these models.

108. Parameters and the Loco User Experience

A beginner should not have to understand every tensor inside a model before running it. That is one of the useful roles a platform can play. Instead of forcing users to interpret raw technical details, a model library can expose meaningful information such as:

Model 8B

Quantization 4-bit

File size ~X GB

Context X tokens

Recommended memory Approximate range

Capabilities Chat / Code / Vision / etc.

The underlying complexity remains. The interface simply makes the complexity understandable. The more you understand parameters, the more sense these model-library fields will make.

109. Why Model Libraries Need Accurate Parameter Data

Because parameter count affects practical deployment decisions, it should not be treated as decorative metadata. A model page that claims:

8B

should ideally be based on reliable model metadata or documentation. Similarly, if a model is an MoE architecture, the page should distinguish between:

total parameters

and:

active parameters

when those values are relevant and verified. This is especially important when users are deciding whether a model can run on their machine.

110. Parameters and Model Search

Imagine a user searching for:

"Models that can run on a computer with limited memory."

Parameter count becomes one useful filter. They might select:

Parameter range: 1B–8B

Then:

Quantization: 4-bit

Then:

Modality: Text

Then:

Task: General assistant

This is far more useful than showing a giant list of model names. Understanding parameters helps explain why model-library filtering matters.

111. Practical Exercise 1: Dense Layer

Calculate the number of parameters in:

Assume one bias per output. Formula:

Therefore:

Biases:

512

Total:

131,584 parameters

112. Practical Exercise 2: Storage

Take the previous:

131,584 parameters

At FP16:

131,584 × 2 = 263,168 bytes

Approximately:

0.26 MB

At 4-bit:

131,584 × 0.5 = 65,792 bytes

Approximately:

0.066 MB

Again, real storage depends on the actual representation and metadata.

113. Practical Exercise 3: 14B Model

Suppose you have a:

14B model

Estimate raw storage at 4 bits.

14,000,000,000 × 0.5 bytes

7,000,000,000 bytes

Approximately:

7 GB

This does not mean the model will necessarily require exactly 7 GB of RAM. It is a first-order parameter-storage estimate.

114. Practical Exercise 4: Compare 8B and 32B

At 4 bits:

8B: ≈ 4 GB raw parameter storage

32B: ≈ 16 GB raw parameter storage

So the 32B model has approximately four times as many parameters. Its raw parameter-storage requirement at the same nominal bit width is also approximately four times larger. That gives you an immediate intuition for model scale.

115. Practical Exercise 5: Why Context Matters Separately

Imagine an 8B model. Its parameter count does not change when you increase the context length. You still have:

8B parameters

But runtime memory can increase because the model has to maintain more contextual state. Therefore:

This distinction becomes very important when we study context windows.

116. Practical Exercise 6: Identify the Parameter

Which of these is a parameter?

A

User prompt: "Explain gravity."

B

Weight: 0.0231

C

Token: "gravity"

D

Context length: 8192

The correct answer is:

B

The weight is a learned numerical value. The prompt and token are inputs. The context length is a configuration/model capability rather than an individual learned weight.

117. Practical Exercise 7: Parameter or Hyperparameter?

Classify each:

Learning rate Weight matrix Bias vector Batch size Embedding matrix Number of layers

Parameters:

Weight matrix Bias vector Embedding matrix

Hyperparameters/configuration:

Learning rate Batch size Number of layers

The distinction is simple but foundational.

118. Knowledge Check

Before moving on, try answering these without looking back.

Question 1

What is a parameter?

Question 2

What is the difference between a weight and a parameter?

Question 3

What does 8B mean?

Question 4

Does 8B mean eight billion words?

Question 5

Why can an 8B model have a file size of only a few gigabytes?

Question 6

What is the basic raw storage equation?

Question 7

Why is model file size not the same as runtime RAM usage?

Question 8

What are activations?

Question 9

What is the KV cache?

Question 10

Does a larger parameter count automatically mean a better model?

Question 11

What is an embedding matrix?

Question 12

Why can Transformer MLP layers contain huge numbers of parameters?

Question 13

What is the difference between total and active parameters in an MoE model?

Question 14

Does quantizing an 8B model turn it into a 4B model?

Question 15

Why can you not usually point to one parameter and say, "That parameter contains the fact about Paris"?

119. Knowledge Check Answers

Answer 1

A parameter is a learned numerical value inside a machine-learning model.

Answer 2

A weight is a type of learned parameter used as a coefficient in a mathematical transformation. Parameters can also include biases and other learned values.

Answer 3

Approximately eight billion parameters.

Answer 4

No. It refers to learned numerical values, not words.

Answer 5

Because the parameters can be represented using lower-precision formats or quantized representations.

Answer 6

A useful approximation is:

Answer 7

Runtime memory also includes things such as activations, KV cache, buffers, runtime overhead, and other application memory.

Answer 8

Activations are intermediate numerical values produced as the model processes an input.

Answer 9

The KV cache stores attention-related key/value information during autoregressive inference so previous context does not always need to be recomputed from scratch.

Answer 10

No. Parameter count is a scale indicator, not a direct intelligence or quality score.

Answer 11

An embedding matrix is a learned table of vectors used to represent tokens numerically.

Answer 12

Because the MLP contains large learned matrices, and matrix dimensions multiply together to produce very large numbers of parameters.

Answer 13

Total parameters describe the whole MoE parameter set, while active parameters describe the subset used for a particular token's computation.

Answer 14

No. An 8B model quantized to 4 bits still represents approximately eight billion parameters.

Answer 15

Because learned information and capabilities are distributed across many parameters and transformations rather than stored as one parameter per fact.

120. A Compact Parameter Cheat Sheet

Keep this nearby when working with local models.

121. The Parameter Formula Cheat Sheet

For a dense layer:

For raw parameter storage:

Examples:

8B FP32 ≈ 32 GB raw parameter storage

8B FP16 ≈ 16 GB

8B INT8 ≈ 8 GB

8B 4-bit ≈ 4 GB

Remember: These are approximations. Actual files and runtime memory can be larger or otherwise different.

122. The Three Most Important Distinctions

If you are starting to work seriously with local AI, memorize these three.

Distinction 1

because precision and quantization matter.

Distinction 2

because inference requires additional memory.

Distinction 3

because architecture, training, data, and many other factors determine behavior. These three distinctions will prevent a huge number of beginner mistakes.

123. Connecting Back to Lesson 7

Lesson 7 explained how neural networks learn. The central process was:

Lesson 8 answers:

What exactly is being updated?

The answer is: The model's parameters. During training:

w1 → w1' w2 → w2' w3 → w3' ...

Billions of values can be adjusted over the course of training. The final collection of learned values becomes the trained model's learned state.

124. Connecting Back to Local AI

Now consider what happens when you download a local model. You are effectively receiving a stored version of that learned state. You are not performing the original training process.

You are taking:

Learned parameters

and using them for:

Inference

on your own hardware. That is the fundamental reason local AI is possible. The expensive training process has already happened.

The user receives the trained model and performs inference locally.

125. Why This Matters for Loco

A local-AI platform has to make several technical decisions understandable to users. A beginner might see:

8B Q4

and have no idea what it means. After this lesson, you should read it approximately as:

8B ≈ roughly eight billion learned parameters

Q4 ≈ a roughly four-bit quantized representation

You can then immediately infer:

The model is relatively large. Its quantized representation reduces storage requirements. It will still require more than the raw file size in many runtime situations. Hardware and context length matter.

That is exactly the kind of technical literacy needed to use local AI intelligently.

126. A Better Way to Think About Model Size

Do not think:

"This model is 8B, so it is eight billion units of intelligence."

Instead think:

"This model contains approximately eight billion learned numerical parameters that collectively define a large parameterized neural network."

That sentence sounds less magical. It is also much closer to reality. And once you understand it, almost every model-size discussion becomes easier.

127. A Full Mental Picture

At this point, your mental model should look something like this:

That is the foundation.

128. What You Should Be Able to Explain Now

If someone asks:

"What is an 8B model?"

you should be able to answer:

An 8B model is a model containing approximately eight billion learned parameters. Those parameters are numerical values distributed across components such as embeddings, attention projections, feed-forward networks, normalization layers, and output projections. The parameters can be represented at different numerical precisions, so the model's file size is not necessarily eight gigabytes. Quantization can reduce the storage required to represent the parameters, but actual runtime memory also depends on activations, context, KV cache, runtime overhead, and hardware.

That is a technically useful answer.

129. Final Key Takeaways

1. Parameters are learned numerical values inside a model.

2. Weights are a type of parameter.

3. Biases can also be parameters.

4. Model labels such as 7B, 8B, and 70B refer approximately to parameter count.

5. Parameter count is not the number of words the model knows.

6. Parameter count is not a direct intelligence score.

7. A model's parameters are distributed across many tensors and layers.

8. Embeddings, attention projections, MLPs, normalization components, and output heads can all contain learned parameters.

9. Dense-layer parameter counts can be calculated from matrix dimensions.

10. The raw storage requirement depends strongly on numerical precision.

11. A useful estimate is:

1. Quantization reduces the number of bits used to represent parameters.

2. An 8B model quantized to 4 bits is still an approximately 8B model.

3. Model file size is not the same as runtime RAM usage.

4. Activations are different from parameters.

5. The KV cache is different from parameters.

6. Longer context can increase runtime memory even though parameter count stays unchanged.

7. MoE models can distinguish total parameters from active parameters.

8. Architecture, training data, optimization, and post-training all influence model behavior.

9. The parameters are best understood as the learned numerical configuration of the neural network.

130. Final Mental Model

If you remember one diagram from this lesson, remember this:

And remember what the parameters represent:

Not: one parameter = one word one parameter = one fact one parameter = one neuron one parameter = one capability

Instead: billions of learned numerical values working together through a neural-network architecture to transform inputs into outputs.

Once you understand that, the mysterious labels attached to local AI models become much less mysterious. "8B" is no longer just a number. It tells you roughly how large the learned numerical system is.

"4-bit" tells you something about how those numerical values are represented. "GGUF" tells you about the model-file representation. "Context length" tells you something about the amount of input the model can process.

"RAM and VRAM" tell you about the hardware available to perform the computation. And all of these pieces come together when you try to run a model locally.

131. Preview of Lesson 9

Lesson 9 — What Is a Large Language Model?

We now know:

• what AI is
• what machine learning is
• what deep learning is
• what generative AI is
• what an AI model is
• how training differs from inference
• what neural networks are
• how neural networks learn
• what parameters and weights are

The next question is:

What exactly makes a model a Large Language Model?

In Lesson 9, we will move from the general concept of neural networks into the specific architecture and behavior of language models.

We will examine:

• what "language model" actually means
• why language can be modeled statistically
• what makes a language model "large"
• how tokens become numerical representations
• how a model predicts the next token
• why next-token prediction can produce surprisingly complex behavior
• pretraining
• self-supervised learning
• language-model objectives
• instruction tuning
• chat models
• base models
• context
• vocabulary
• tokenization
• model scale
• why LLMs can write, code, summarize, translate, and answer questions
• why an LLM is not simply a database of text
• how LLMs relate to local AI
• why LLMs became the foundation of modern local generative AI

By the end of that lesson, the phrase Large Language Model should stop sounding like a mysterious technical label and become a precise description of a specific kind of learned computational system.

Sources and Further Reading

The following resources provide deeper technical background for the concepts covered in this lesson.

1. Google Machine Learning Glossary
   https://developers.google.com/machine-learning/glossary

2. Google Machine Learning Crash Course
   https://developers.google.com/machine-learning/crash-course

3. Attention Is All You Need — Vaswani et al. (2017)
   https://arxiv.org/abs/1706.03762

4. Deep Learning — Goodfellow, Bengio and Courville
   https://www.deeplearningbook.org/

5. Hugging Face Transformers Documentation
   https://huggingface.co/docs/transformers/

6. Hugging Face Quantization Documentation
   https://huggingface.co/docs/transformers/main/en/quantization/overview

7. llama.cpp — GitHub
   https://github.com/ggml-org/llama.cpp

8. NIST Artificial Intelligence Resource Center
   https://www.nist.gov/artificial-intelligence

These sources are useful for moving from the simplified explanations in this lesson toward formal technical documentation and research literature.

Lesson Complete

Completed: Lesson 8 — What Are Parameters and Weights?

Next: Lesson 9 — What Is a Large Language Model?

Course: Local AI 101

Loco Academy: Lesson Status

Completed: Lesson 8: What Are Parameters and Weights?

Next: What Is a Large Language Model?

Course: Local AI 101