ITG NowHome › AI glossary › Latency
What is latency in AI?
Latency is the wait between sending a request to an AI and getting the first part of its answer.
- Category Benchmarks
- Also called time to first token
- Updated 20 Oct 2026
Latency, explained simply
Latency is the delay. You press send, and latency is how long you wait until the answer starts to appear.
It is often measured as time to first token. A token is a small piece of text the AI reads or writes.
This matters because low latency feels fast and smooth, especially in voice chats and live tools. Lower is better.
For example, a chatbot that starts replying right away feels quick, even if the full answer takes a while.
On ITG Now, the AI models page shows output speed next to quality scores, so you can weigh speed against quality.
Related terms
Tokens per secondInferenceTokenBenchmarkEdge AI
Where you'll see it on ITG Now
Questions people ask about Latency
Is lower latency better?
Yes. Lower latency means a shorter wait before the answer begins.
Latency vs tokens per second: what is the difference?
Latency is the wait at the start. Tokens per second is how fast the text flows after it starts.
What makes AI latency high?
Long inputs, big models, busy servers and slow internet can all add delay.