Translated with AI from the Russian original.

Along with it, the company presented a set of Russian-language audio data for testing ASR models under identical conditions

MWS AI, part of MTS Web Services, has released RESON, an open-source tool for evaluating automatic speech recognition models. It allows comparing them across more than 10 metrics and generates an interactive report with analysis results.

To test a model in RESON, its transcripts and reference texts are uploaded. The system unifies the data format, calculates selected metrics, and helps identify typical errors. Available metrics include WER and CER, which evaluate errors at the word and character levels, as well as MWA and the MWS AI-developed WIS metric, which considers the significance of individual words for a specific scenario.

RESON is distributed as an open Python library. It can be run via the command line or API and used within one's own infrastructure, including closed circuits.

Simultaneously, MWS AI introduced the MWS RESON ASR OSD dataset for comparing models under identical conditions. It includes over 12,000 Russian-language audio recordings with a total duration of about 38 hours. The sample contains conversations, phone numbers, dates, amounts, company and service names, as well as recordings with noise and background speech.

Across four datasets, GigaAM v3 from Sber was the best among the tested models with an average WER of 7.42%. Vosk showed 11.07%, Nvidia Parakeet 15.83%, and Nvidia Nemotron 25.52%.

Now on the front page

News

“MediaKult – 2026”. From the attention economy to AI – important industry trends on a single platform

Submissions are open for “MediaKult – 2026” – one of the key industry events bringing together a professional award and a business forum for leaders in media, marketing, creative industries, and technology. The event will take place this autumn in Moscow and will bring together more than 500 participants – representatives of business, agencies, media, and government agencies.