Glossary

Tokens per Second

TPS

A performance metric measuring the rate at which an inference engine generates output tokens per second.

Explore more about Metrics

Related terms

Latency

The time delay between an AI system receiving a request and producing a response or completing an operation.

Throughput

The amount of data, tokens, or requests processed by a system within a specified period.

Time to First TokenTTFT

The duration of time it takes for a language model to produce its initial output token after receiving a request.

Token Usage

The total volume of prompt and completion tokens consumed during interactions with a language model.

Response Time

The total duration required for an AI system or service to process an input query and return the completed output.

Signal, not noise.

Focused newsletter for builders and knowledge workers tracking how AI is changing real work. We surface what matters in practice, not every headline. Curated for practitioners, not spectators.

Subscribe to our newsletter