“Probabilistic” Is an Ambiguous Pointer

Calling a system probabilistic often sounds like an explanation.

It usually is not.

Consider the following statements:

A Bloom filter is probabilistic.

HyperLogLog is probabilistic.

A large language model is probabilistic.

Sampling is probabilistic.

The result is probabilistic.

These statements all sound related. They all contain the same adjective. Yet they can describe fundamentally different properties.

Does the system consume randomness?

Does it deterministically calculate a probability distribution?

Can it return an incorrect answer?

Does it return an approximation?

Is its error statistically bounded?

Can repeated executions produce different outputs?

Does the system represent uncertainty about some fact?

The word probabilistic may point to any of these.

That makes it an ambiguous pointer.

A.IsProbabilistic Is Not Enough

Suppose a system has some property:

A is probabilistic.

That statement assigns probabilistic to the entire system as though it were a simple Boolean attribute:

A.IsProbabilistic = true

But what does that value actually tell us?

Not much.

It does not tell us where probability enters the system, how probability affects the result, whether the execution is reproducible, or what guarantees the system provides.

The useful question is not:

Is the system probabilistic?

It is:

Which part of the system involves probability, and how?

The original statement needs to be dereferenced.

Perhaps the intended claim is:

A uses random values during execution.

Or:

A deterministically computes a probability distribution.

Or:

A returns an approximate estimate with a statistically characterized error.

Or:

A may return false positives but cannot return false negatives.

Or:

A can produce different outputs across repeated executions.

Each of these is a meaningful engineering statement.

“A is probabilistic” is an underspecified type.

The Bloom Filter Problem

Bloom filters are commonly introduced as probabilistic data structures.

That description is not wrong, but it is incomplete enough to create confusion.

A Bloom filter represents set membership using a bit array and several hash functions.

When inserting an item, the filter hashes the item to several positions and sets the corresponding bits to 1.

item A
  ├── hash 1 → bit 3
  ├── hash 2 → bit 11
  └── hash 3 → bit 19

After insertion:

bit 3  = 1
bit 11 = 1
bit 19 = 1

To query the filter, the same positions are calculated.

If any required bit is 0, the item was not inserted.

query positions: 3, 11, 19
stored bits:     1,  0,  1

The zero is conclusive. Had the item been inserted, every corresponding bit would have been set.

Therefore:

at least one required bit is 0
→ definitely not present

If every required bit is 1, however, the item may or may not have been inserted.

query positions: 3, 11, 19
stored bits:     1,  1,  1

Those bits may have been set by that item. They may also have been set independently by several other items whose hashes happened to overlap.

Therefore:

all required bits are 1
→ possibly present

The filter can prove non-membership. It cannot prove membership.

A more honest method name would be:

filter.MightContain(item)

rather than:

filter.Contains(item)

The contract is asymmetric:

false → definitely absent
true  → possibly present

A useful mnemonic is:

No means no. Yes means maybe.

Where Is the Probability?

At first, it is tempting to say that the negative result is deterministic and the positive result is probabilistic.

That still mixes two different dimensions.

Both results can be calculated deterministically.

Given:

• the same Bloom filter state,
• the same item,
• the same hash functions,
• and the same implementation,

the filter will inspect the same bits and return the same answer every time.

There is no coin flip during lookup.

An absent item that produces a false positive will continue producing the same false positive when queried again against the same filter state.

The lookup itself is deterministic.

The probabilistic property concerns the filter’s error behavior across possible queries.

A Bloom filter can be described using a classification table:

Reality	Filter says absent	Filter says possibly present
Item is absent	True negative	False positive
Item is present	Impossible	True positive

One error quadrant is possible: the false positive.

The relevant probability is:

$$ P(filter\ says\ possibly\ present \mid x \notin S) $$

This is the probability that an item not in the set happens to map entirely to bits that have already been set by other insertions.

That probability depends on properties such as:

• the size of the bit array,
• the number of inserted items,
• and the number of hash functions.

The filter is therefore not vaguely “making a pretty good try.”

It provides a precise one-sided guarantee:

A negative result is conclusive. A positive result has a quantifiable false-positive rate.

That is much more informative than saying the data structure is probabilistic.

Probability Is Not Randomness

The Bloom filter example exposes a common inference error.

When people hear probabilistic, they often infer random.

But probability and randomness are not the same property.

A deterministic program can calculate a probability:

Chance of rain: 70%

Running that program again with identical inputs may return exactly 70%.

The computation is deterministic. The result represents uncertainty.

Similarly, a deterministic classifier might return:

dog: 0.92
cat: 0.06
other: 0.02

The program did not necessarily behave randomly. It calculated values interpreted as probabilities.

A system can also execute deterministically while providing a probabilistic correctness guarantee.

That is what happens with a Bloom filter.

The implementation may repeatedly produce the same answer for the same input, while its false-positive behavior is analyzed statistically over a population of possible inputs.

So this combination is entirely coherent:

deterministic execution
+
probabilistic error guarantee

The words deterministic and probabilistic are not always opposites because they may be describing different layers.

HyperLogLog Is “Probabilistic” in Yet Another Way

HyperLogLog estimates the number of distinct values in a data stream.

The exact distinct-count problem asks:

How many unique items have appeared?

The straightforward solution is to retain enough information to identify every unique item, commonly by storing them in a set.

That can require substantial memory.

HyperLogLog instead stores a compact statistical summary derived from hashed values. It uses patterns in those hashes to estimate cardinality.

The result might be:

Estimated distinct values: 10,214,731

The exact value could be somewhat higher or lower.

HyperLogLog is therefore often called probabilistic. But once again, that adjective needs to be expanded.

A particular implementation can be deterministic:

same state
+
same algorithm
→
same estimate

The estimate does not need to fluctuate every time it is read.

The probability appears in the characterization of the estimator’s error. HyperLogLog is better described as:

A deterministic approximate cardinality estimator with statistically characterized error.

That statement tells us considerably more.

Large Language Models Complicate the Word Further

Large language models introduce several different probability-related properties at once.

Given some context, an autoregressive language model calculates scores for possible next tokens.

After normalization, these scores define a conditional probability distribution:

$$ P(tᵢ \mid t₁, t₂, \ldots, tᵢ₋₁) $$

A simplified result might look like this:

"the"   → 0.42
"a"     → 0.18
"this"  → 0.07
"some"  → 0.04
...

In this sense, the model is probabilistic because it models a probability distribution over possible continuations.

But the forward pass that calculates that distribution need not be random.

At an abstract level, the network performs a series of numerical operations using fixed weights and a fixed input:

$$ hᵢ₊₁ = fᵢ(Wᵢ hᵢ + bᵢ) $$

Fix the weights, input, arithmetic, and execution conditions, and the calculated values are determined.

The model can therefore be described as:

A deterministic computation of a conditional probability distribution.

That is already different from the probabilistic behavior of a Bloom filter.

The Bloom filter does not normally output an explicit probability distribution. Its use of probability appears in its false-positive guarantee.

The language model explicitly parameterizes a distribution over possible next tokens.

Calling both systems “probabilistic” conceals that distinction.

Decoding Introduces Another Property

Once the language model has produced a distribution, something must select the next token.

One option is greedy decoding:

$$ t = \arg\max_i P(i \mid context) $$

The decoder selects the token with the highest probability.

Given identical model outputs, greedy decoding selects the same token.

Another option is sampling.

The decoder randomly selects a token according to the model’s distribution, possibly modified using settings such as temperature, top-(k), or top-(p).

Now repeated generations can produce different continuations.

These two cases should not be compressed into the same statement.

The model calculates token probabilities.

and:

The decoder samples stochastically from those probabilities.

are separate claims.

An LLM system can use a probabilistic model while performing deterministic decoding.

It can also use a probabilistic model with stochastic decoding.

The bare statement “LLMs are probabilistic” does not tell us which one is happening.

Nondeterminism Is Not Necessarily Sampling

There is another complication.

Even when stochastic sampling is disabled, real inference systems may not always reproduce the same exact output.

The mathematical model may be deterministic, but the deployed computation can involve:

• finite-precision floating-point arithmetic,
• parallel reductions,
• different operation ordering,
• different batch shapes,
• different hardware,
• different inference kernels,
• or different levels of quantization.

Floating-point arithmetic is not perfectly associative:

$$ (a+b)+c \ne a+(b+c) $$

Changing the order of operations can introduce very small numerical differences.

Most small differences do not matter. But if two candidate tokens have nearly equal scores, a tiny change can reverse their order.

Execution 1:
token A → 12.847294
token B → 12.847291

Execution 2:
token A → 12.847291
token B → 12.847294

Greedy decoding now chooses a different token.

Because generation is autoregressive, that token becomes part of the next input:

different token
→ different context
→ different distribution
→ different continuation

The outputs can diverge substantially.

This is execution nondeterminism, but it is not the same as probabilistic sampling.

Again, saying “the LLM is probabilistic” tells us almost nothing about which mechanism caused the variation.

A useful description would instead say:

The model deterministically calculates a token distribution in theory, greedy decoding selects the maximum-probability token, but the inference implementation is not bitwise reproducible across these execution environments.

Longer, yes.

Also considerably more useful.

Exact Reproduction and Semantic Stability

Language-model evaluation often introduces another ambiguity: the difference between exact textual reproduction and semantic stability.

Suppose a model is run ten times with the same prompt.

It might produce ten different strings that all express essentially the same answer.

Someone may describe the model as stable because the meaning remains similar.

Another person may describe it as unstable because none of the byte sequences match.

Both may be correct according to different definitions.

“Stable” has the same underspecification problem as “probabilistic.”

A proper measurement must name the invariant:

Exact byte equality

Exact token equality

Same first divergence position

Same selected answer

Equivalent meaning

Semantic equivalence may be valuable, but it is inherently fuzzier than exact textual equality.

For a strict reproducibility test, the invariant can be brutally simple:

stable = all(output == outputs[0] for output in outputs)

Either every output matches or at least one does not.

That result still belongs to the complete execution tuple:

model checkpoint
+
tokenizer
+
inference backend
+
backend version
+
hardware
+
precision
+
quantization
+
batch configuration
+
decoding settings
+
random state

It is not necessarily a property of the model checkpoint alone.

A Taxonomy That Actually Says Something

Instead of assigning probabilistic to an entire system, describe the specific property.

Randomized execution

The algorithm deliberately consumes random values during execution.

same input
+
different random state
→
potentially different output

Stochastic decoding

A decoder samples from a probability distribution rather than always choosing the maximum-scoring option.

Probability-distribution model

The system represents or calculates a probability distribution over possible outcomes.

P(outcome | context)

One-sided error

Only one category of mistake is possible.

For a standard Bloom filter:

false positives: possible
false negatives: impossible

Bounded error probability

The system may return an incorrect result, but the probability of error is mathematically characterized.

Approximate estimation

The system returns an estimate rather than an exact answer.

Approximation does not automatically imply randomized execution.

Statistical error guarantee

The estimator’s expected error, variance, or confidence behavior is characterized over some input or hash distribution.

Nondeterministic execution

Repeated runs under apparently identical user-level inputs may produce different outputs.

This could result from randomness, concurrency, numerical behavior, hardware, or implementation details.

Uncertain claim

The output expresses less than complete confidence in some proposition.

The item may be present.

There is a 70% chance of rain.

The image probably contains a dog.

These properties can coexist, but they should not be treated as interchangeable.

The Axes Are Independent

A system can occupy several positions simultaneously.

System	Execution	Result type	Error behavior
Exact hash set	Deterministic	Exact membership	No approximation
Bloom filter	Deterministic	One-sided membership test	Bounded false positives
HyperLogLog	Deterministic	Approximate cardinality	Statistical estimation error
LLM with greedy decoding	Generally deterministic in the abstract	Token sequence	Computed from a probability model
LLM with sampling	Stochastic	Token sequence	Sampled from a token distribution
Parallel local LLM inference	Potentially nondeterministic in practice	Token sequence	Sensitive to numerical execution
Monte Carlo estimator	Randomized	Approximate estimate	Error decreases statistically with samples

The table makes the central problem visible.

There is no single probabilistic-versus-deterministic axis that cleanly separates these systems.

A system can be deterministic and still involve probabilities.

A system can be randomized and still return an exact answer.

A system can return an approximation without using fresh randomness during a query.

A system can model probabilities while selecting outputs deterministically.

A system can produce different outputs because of numerical nondeterminism rather than intentional sampling.

The adjective is trying to carry too much information.

Probability Must Have an Object

A probability is always a probability of something, under some set of assumptions.

For a Bloom filter:

$$ P(false\ positive \mid x \notin S) $$

For an approximate counter:

$$ P(|\hat n-n| < \epsilon) $$

For a language model:

$$ P(next\ token \mid context) $$

For stochastic decoding:

$$ P(selected\ token=t) $$

For execution stability:

$$ P(output_1=output_2 \mid environment) $$

These are different random variables, different events, and different conditional assumptions.

Saying only that the surrounding system is probabilistic removes the object of the probability.

It is similar to saying:

This function is nullable.

What is nullable?

The input?

The return value?

One field inside the return value?

A dependency?

A database column?

Without specifying the target, the annotation is incomplete.

“Probabilistic” often has the same problem.

Ask the Next Question

When someone calls a system probabilistic, the correct response is not necessarily to disagree.

It is to ask for the pointer to be dereferenced.

Which part?

What is the relevant probability?

Is the execution randomized?

Is the output a probability distribution?

Can the answer be wrong?

Which errors are possible?

Is the result approximate?

Can repeated runs differ?

What remains fixed between runs?

Over what population is the probability defined?

These questions convert a broad label into an engineering description.

Better Descriptions

Instead of:

A Bloom filter is probabilistic.

Say:

A Bloom filter performs deterministic lookups, proves non-membership when any required bit is unset, and admits false-positive membership results at a quantifiable rate.

Instead of:

HyperLogLog is probabilistic.

Say:

HyperLogLog is an approximate distinct-count estimator whose error is statistically characterized.

Instead of:

An LLM is probabilistic.

Say:

An LLM calculates a conditional probability distribution over tokens.

Then specify the decoder:

Generation uses greedy token selection.

or:

Generation samples stochastically from the distribution.

Then specify reproducibility separately:

The inference stack is bitwise reproducible under the fixed environment.

or:

Repeated inference can diverge because execution order is not fully reproducible.

These descriptions take more words because they contain more information.

That is not unnecessary verbosity. It is type resolution.

The Word Is Not Wrong

The argument is not that the word probabilistic should disappear.

It is useful as a category label.

Bloom filters, HyperLogLog, randomized algorithms, Bayesian models, Markov chains, Monte Carlo methods, and language models all meaningfully involve probability.

The problem begins when the category label is mistaken for a complete explanation.

“Probabilistic” tells us that probability appears somewhere in the system.

It does not tell us where.

It does not tell us whether execution is random.

It does not tell us whether repeated runs will match.

It does not tell us whether the output is approximate.

It does not tell us which claims are conclusive.

It does not tell us which errors are possible.

It does not tell us what event the probability measures.

Without that information, the word points in several possible directions at once.

Dereference the Pointer

“A is probabilistic” should be treated as an incomplete statement.

Not necessarily false.

Incomplete.

The next step is to identify the exact object to which probability applies:

A's execution is randomized.

A's decoder samples stochastically.

A's model represents a conditional probability distribution.

A's positive classifications have a bounded false-positive rate.

A's estimate has a statistical error guarantee.

A's output is not reproducible across repeated executions.

Only then do we know what property is actually being described.

Until the pointer is dereferenced, probabilistic is not an explanation.

It is an underspecified type.