Time to First Token

The time delay between an AI system receiving a request and producing a response or completing an operation.

The total duration required for an AI system or service to process an input query and return the completed output.

The amount of data, tokens, or requests processed by a system within a specified period.

A performance metric measuring the rate at which an inference engine generates output tokens per second.

A metric that measures the average monetary cost incurred to process a single API call, inference, or user request.