Call transcription and summaries for a contact centre on its own servers need two GPU stages: speech recognition for every minute of conversation, and one language-model pass per call. Both are sized from two numbers: how many conversations run at once in the peak hour, and how many call minutes accumulate per day. By our estimate, real-time agent assist for 1,000 agents takes 4 to 16 NVIDIA L40S cards for speech alone, while the call summaries for the same shift fit on a single RTX PRO 6000 Server Edition.

Recognition models, their memory and their speed are covered in our guide to a speech-to-text server for Whisper and Parakeet.

In brief

  • Separate channels for agent and customer remove the need for diarisation but double the streams: 1,000 simultaneous conversations become 2,000 streams.
  • NVIDIA quotes up to 190 live English streams per L40S for Parakeet 0.6B CTC with 160 ms chunks and up to 900 with 960 ms; by our estimate that means 4 to 16 L40S for 1,000 agents.
  • Eight RTX PRO 6000 Server Edition cards reached 50,965.3 tokens/s on Llama 3.1 8B offline in MLPerf Inference v6.1 (entry 6.1-0050); by our estimate one card handles the summaries for 1,000 agents.
  • NVIDIA's L40S streaming figures cover English, Spanish, Vietnamese and Mandarin; for Ukrainian and Russian, a test on your own recordings sets the stream count.
  • Call recordings are personal data: callers are told about the recording and its purpose at the start, and retention periods are fixed in advance.

Speech recognition and the language model: which models for each stage

Recognition turns each minute of conversation into text, and the language model reads the transcript once to write a summary covering why the customer called and what happens next. Real-time recognition needs a streaming model, for example Parakeet CTC or the multilingual Parakeet RNNT 1.1B, because NVIDIA ASR NIM runs Whisper in offline mode only. After the call, offline models will do, for instance Parakeet TDT 0.6B v3, which covers 25 European languages, Ukrainian and Russian among them.

If your telephony platform allows it, record the agent and the customer on separate channels: the channel then tells you who is speaking and no diarisation model is required. When both voices share one track, diarisation may consume more GPU time than recognition itself. NVIDIA's ASR NIM page of 7 October 2026 shows Parakeet 0.6B CTC on one L40S with 32 parallel requests at an offline RTFx of 3,638 without diarisation, but only 101.5 once it is switched on. For single-track recordings, Nemotron 3 Diarization (OpenMDW-1.1, 100M parameters, up to eight speakers) is worth testing; it is covered in our article on diarisation on your own server.

The language-model stage reads a long text and writes a short one. The Llama 3.1 8B test in MLPerf Inference has the same profile: according to MLCommons, the model summarises CNN/DailyMail news, averaging 778 tokens in and 73 tokens out. We use its results as the reference point for an 8B-class model.

Stage Model type Published figure Card
Real-time transcription Parakeet 0.6B CTC, streaming max. 190 English streams with 160 ms chunks or 900 with 960 ms, per L40S L40S
Post-call, English Parakeet 0.6B CTC, offline RTFx 3,638 with 32 requests in parallel, per L40S L40S
Call summary 8B-class LLM, batched 50,965.3 tokens/s on eight cards, offline scenario RTX PRO 6000 Server Edition
Agent assist 8B-class LLM, latency-capped 49,523.3 tokens/s, server scenario, eight cards RTX PRO 6000 Server Edition

Source: NVIDIA ASR NIM performance page, updated 07.10.2026 (L40S, English, maximum streams as stated by NVIDIA); MLPerf Inference v6.1, data centre, closed division, available systems, Llama 3.1 8B, entry 6.1-0050, HPE ProLiant Compute DL380a Gen12 with eight RTX PRO 6000 Blackwell Server Edition, retrieved from mlcommons.org on 16.10.2026, results verified by MLCommons Association; the server scenario caps time to first token at 2 s and time per output token at 100 ms. MLPerf is a trademark of MLCommons Association.

Real-time agent assist or post-call processing

After the call, recordings are transcribed and summarised from a queue. If the summary has to be ready before the agent's next call, the hardware is sized for the peak hour; if it can wait until morning, for the daily volume.

With real-time agent assist the system transcribes both channels during the conversation and, each time the customer stops talking, asks the language model to suggest a reply, so the hardware is sized for the peak. In NVIDIA's L40S data, 512 English streams using 960 ms chunks showed an average latency of 198.8 ms, with text arriving in steps of roughly one chunk. With 160 ms chunks, 64 streams showed 33.9 ms, but the maximum falls from 900 to 190 streams, so you need nearly five times as many cards.

Every suggestion request sends the whole conversation from the start. vLLM's automatic prefix caching keeps the KV cache of earlier requests and lets a new request reuse it when the beginning matches, so each request mostly computes the newest turn. Check time to first token and time per token under peak-hour load.

Ukrainian and Russian in streaming recognition

The L40S streaming results we could read on NVIDIA's page cover English, Spanish, Vietnamese and Mandarin. For Ukrainian or Russian, a test on your recordings decides how many streams one card carries.

An open streaming model with both languages does exist: Nemotron 3.5 ASR Streaming 0.6B (Hugging Face, 4 June 2026, OpenMDW-1.1, 40 locales, 32 of them without fine-tuning). On the Handy leaderboard (FLEURS) it scores a WER of 14.88% in Ukrainian and 12.61% in Russian, while the offline Parakeet TDT 0.6B v3 scores 6.84% and 6.54% on the same board. We found no measurements on surzhyk or on mixed-language conversations. Which open models handle both languages better is covered in our article on recognising Ukrainian and Russian locally.

How to size GPUs from the peak hour

  1. Take the number of simultaneous conversations in the peak hour, or, at most, the count of agents who are logged in.
  2. Double it for separate channels: the result is your peak count of live streams, and also of audio hours per hour.
  3. For agent assist, divide the streams by the planned streams per card; we plan at about 70% of NVIDIA's maximum.
  4. For post-call processing, recognition requires an RTFx no lower than the number of channels in the peak hour if summaries must keep up, or the daily channel-hours divided by the length of the processing window in hours, if they can wait.
  5. For the language model, multiply calls ending each hour by summary output tokens, then add the suggestion tokens and divide by the planned speed of one card.

An example for step 4: 1,000 agents talking 5 hours a day on two channels produce 10,000 channel-hours daily. In a 12-hour night window that is an RTFx of about 833, and keeping up with a peak hour in which every agent is on the line takes 2,000.

How many GPUs 200, 500 and 1,000 agents need

Agents and calls at peak Live streams Post-call, L40S Real-time, L40S LLM, RTX PRO 6000
200 agents on 200 calls 400 1 (English) 960 ms: 1; 160 ms: 4 1 either way
500 agents on 500 calls 1,000 1 (English) 960 ms: 2; 160 ms: 8 1 either way
1,000 agents on 1,000 calls 2,000 2 (English) 960 ms: 4; 160 ms: 16 1 for summaries; 2 with assist

Source: our estimates, not measurements, from the first table: live streams at 70% of NVIDIA's maxima (133 and 630 per L40S, English); post-call recognition at half the published speed (RTFx 1,819 per L40S, keeping up with the peak hour); the language model at half the per-card rate from MLPerf Inference v6.1 (3,185 tokens/s offline, 3,095 in the server scenario; our division, not an MLPerf metric).

The table assumes every agent is on a call in the peak hour, an average call of 6 minutes, summaries of 150 output tokens and, with agent assist, 80 tokens of suggestion per call every 20 seconds. When recognition runs live, summaries use the live transcript and the post-call column drops out. If your suggestions must be faster than the server scenario's 100 ms for each output token, expect fewer tokens/s per card.

With 1,000 agents, the peak hour sees 10,000 calls end, and their summaries amount to about 417 output tokens/s, roughly 13% of what we plan for one RTX PRO 6000. Suggestions during calls add about 4,000 tokens/s, which is why agent assist doubles the cards for the language model. For Parakeet TDT v3 (25 languages) we found no published speed on these cards, so a test on your recordings decides the card. ETE builds servers for this layout with L40S and RTX PRO 6000 Server Edition cards, and Vixen.UNO engineers can deploy Parakeet, vLLM and an 8B-class summary model where needed.

Recordings and transcripts: volume and retention

The ITU-T G.711 codec specifies 8,000 samples per second at eight bits each, so one channel takes 64 kbit/s, or by our calculation 28.8 MB per hour. The 10,000 channel-hours of 1,000 agents add up to 288 GB of audio a day, while transcripts and summaries take only a small fraction of that.

Recordings in which a customer or an agent can be recognised are personal data. Ukraine has no specific law on call recording by commercial companies, so you need one of the six grounds for processing in Article 11 of Law No. 2297-VI and prior notice under Article 12. In practice the caller hears at the start that the call is recorded and transcribed and why, and the recording is kept only as long as that purpose requires. In consumer lending, Law No. 1734-VIII (Article 25, part 4) explicitly requires lenders and collectors to record every direct interaction with a consumer on audio or video, warn about the recording and keep it for 3 years.

Audio, transcript and summary can have different retention periods, and deletion must remove the transcript from both the file store and the search index. Health information mentioned by callers belongs to the special categories under Article 7 of Law No. 2297-VI, so grant access to transcripts by role and log searches. This overview is not legal advice.

Agent emotions, employee monitoring and the EU AI Act

Ukraine has no specific rule on analysing employees' conversations with AI, so the general rules apply: employees are informed in advance, against signature, of their working conditions, including any monitoring (Article 29 of the Labour Code), and Article 8 (part 2, item 13) of Law No. 2297-VI gives people protection against automated decisions with legal effects. Bonuses or dismissals should therefore not rest on a model's score alone.

Ukraine has no AI law yet. The EU AI Act matters to a Ukrainian contact centre when the system is placed on the EU market or its output is used in the EU. Since 2 February 2025, Article 5(1)(f) has banned AI systems that infer the emotions of people at work or in education, with exceptions for medical and safety reasons. A CMS summary of the Commission's guidelines of 4 February 2025 lists tracking call centre staff's emotions by voice among the banned examples. Detecting customers' emotions by voice is outside the ban, but Article 50(3) has required informing those affected since 2 August 2026, and Annex III, point 1(c), classes such a system as high-risk. Rating agents from their transcripts may fall under point 4(b) of Annex III. The high-risk requirements for Annex III apply from 2 December 2027, and summaries, search and suggestions are not among the Annex III purposes.

Technically, the agent's channel should never be routed to a voice-based emotion model; with separate channels, this becomes a simple routing rule.

Two servers for resilience

Agent assist becomes part of an agent's working day, so a server outage has to be planned for. With two servers, each running both the speech models and the language model, one keeps working at half capacity during repair or updates of the other. Call recording runs independently of the GPU servers, and post-call jobs wait in the queue. Given the power outages in Ukraine, plan the servers together with a UPS and a generator.

L40S and RTX PRO 6000 Server Edition cards alike use passive cooling and rely on server chassis airflow. For production use, NVIDIA NIM containers require an NVIDIA AI Enterprise licence, whereas open engines such as vLLM run without one.

FAQ

Can contact centre AI run on your own servers?

Yes. Open speech models such as Whisper and Parakeet, together with open language models for summaries, run on the company's GPU servers, so recordings and transcripts never leave its network.

What server do I need to transcribe calls?

Count the simultaneous conversations in the peak hour and double them for separate channels. NVIDIA quotes up to 900 live English streams per L40S with 960 ms chunks and 190 with 160 ms, and we plan at about 70% of those maxima. NVIDIA's L40S streaming figures do not include Ukrainian or Russian, so test with your own recordings.

How much GPU do call summaries with a language model need?

Less than recognition. Eight RTX PRO 6000 Server Edition cards reached 50,965.3 tokens/s on Llama 3.1 8B in MLPerf Inference v6.1, and at half that speed one card is enough for summaries from 1,000 agents, by our estimate.

What does real-time agent assist on-premise require?

Streaming recognition of both channels of every conversation and a language model that answers within a target latency at the peak hour. By our estimate, 1,000 simultaneous calls need 4 L40S with 960 ms chunks or 16 with 160 ms, and on top of that two RTX PRO 6000 Server Edition cards for summaries and suggestions.

Is it allowed to analyse contact centre agents' emotions?

Ukraine has no direct ban, but the rules on employee monitoring (Article 29 of the Labour Code) and the protection against automated decisions under Law No. 2297-VI apply. If the system or its output is used in the EU, Article 5(1)(f) of the EU AI Act has banned inferring people's emotions at work since 2 February 2025. A lawyer should assess the specific case.

What ETE supplies for this

  • NVIDIA L40S 48 GB: the card for which NVIDIA publishes Parakeet streaming figures in ASR NIM.
  • NVIDIA RTX PRO 6000 Server Edition 96 GB: a single card handles summaries for 1,000 agents, a second adds real-time agent assist.
  • ETE AI Server RTX PRO: four 96 GB RTX PRO 6000 Server Edition cards in a 4U chassis; two of these servers form a resilient pair, each holding both speech and the language model.
  • ETE AI XL43: eight RTX PRO 6000 Server Edition cards, the same count as in MLPerf result 6.1-0050.

Need servers for AI in your contact centre?

Tell us the number of agents, the simultaneous conversations in the peak hour, the languages of the calls and what you need first: post-call summaries or real-time agent assist. ETE will work out the card and server configuration, and Vixen.UNO engineers can deploy the recognition and summary models if required.

Discuss your project

Author: ETE Engineer