Large language models (LLMs) are already being widely deployed in industrial operations, but the growth of AI system autonomy is noticeably slower than the discussions surrounding it. According to a study by Infosystems Jet and Smart Ranking, 86% of large Russian companies use or pilot LLMs, while fully autonomous AI agents are in industrial operation in only 8%. Oleg Skuratovich, technical project manager at SVOY Tech (the IT cluster of the SVOY fintech group), explained why further automation becomes more complex after the first successful scenarios, where errors that appear to be LLM errors actually originate, and why human involvement is increasingly becoming part of the architecture rather than its limitation.
First scenarios provide the main part of the effect
According to the expert, the team analyzed data for approximately one year of industrial operation. This period smooths out seasonality and one-time load spikes. It turned out that the main flow of requests is covered by only a few typical scenarios. About 72% of requests were due to six common reasons, including payment and application status, operation confirmation, appointment scheduling, process navigation, and transfer to a specialist. After their automation, the load on the first line decreased by approximately 24%, and the average waiting time was reduced by almost a third.
Further, the effect began to decline significantly. The first 20-30% of automated scenarios provided more than 70% of the total operational time savings. “We noted a simple thing for ourselves. Each subsequent scenario should be evaluated not by whether it can be technically automated, but by what additional business effect it will actually provide.” This is how Oleg Skuratovich explains his approach.
LLM is just one layer of the system
The error that the expert calls the most common at the start of projects is the belief that LLM can replace the entire architecture. According to his observations, a mature system in fintech is not a “model plus telephony,” but a set of layers, where each is responsible for its part of the risk. The system is formed by speech recognition, intent detection, request routing, access to internal systems for facts via API (application programming interface for data exchange between systems), a knowledge base, compliance restrictions, and a voice layer. And only on top of all this does the LLM work, which formulates the answer but does not invent facts.
“If you connect LLM directly to telephony or CRM, it looks impressive in a demo. The answers sound lively, the dialogue seems natural. But without routing, data access, and restrictions, it’s not a smart agent, but a convincing but unreliable interface. In a regulated environment, this is already a risk, not just a flaw.” Oleg Skuratovich explains.
According to him, any team on this path faces three forks in the road that are better discussed before the pilot, not after. How many checks and data accesses are acceptable for accuracy of response in exchange for speed. How strictly to set dialogue routes in exchange for natural speech. And how much can be saved on logging and updating the knowledge base without losing system manageability.
Automation rate does not show the real effect
A separate problem is the automation rate metric, the proportion of requests processed without employee involvement. It does not always reflect real savings. A bot can conduct a dialogue independently, but not resolve the client's issue, and the client will contact again. Formally, the scenario will be “automated,” although the load on employees will not decrease. Therefore, the team began to consider a scenario closed only when the client received the desired result and no employee was needed afterward. Unfinished dialogues, repeat calls, and complaints were tracked separately.
A similar shift in focus is noticeable in the banking market as a whole. According to Frank RG and BSS, the share of maximum ratings for robots has grown to 75%, but clients still note insufficient understanding of requests and a lack of flexibility in dialogue. That is, a high degree of automation itself does not guarantee a satisfied client.
An error can occur before the model's response
Perhaps the most unexpected conclusion of the team concerns the nature of errors. A detailed analysis of the logs showed that almost 43% of recorded failures were concentrated at one scenario transition point. At the same time, the LLM itself in these dialogues could correctly understand the request. The problem arose earlier. The system directed the model to respond where it was first necessary to obtain a fact from an internal system or transfer the conversation to an employee.
“Outwardly, it looks like a model error. Inside, it’s almost always something else. The wrong route, an empty response from the API, an outdated knowledge base, or a transfer to an employee without context. Replacing the LLM in such a situation can improve the wording, but will not eliminate the cause of the failure,” says Oleg.
After analysis, the problematic point in the scenario was divided into three branches. The system can respond independently, first obtain a fact, or immediately transfer the request to an employee.
Handoff becomes part of the architecture
Some scenarios in the company were deliberately not fully automated. These are controversial and non-standard situations, complaints, legally sensitive requests, cases with multiple interpretations, and a high cost of error. A separate category of risk is data quality. If internal systems return incomplete or contradictory information, the LLM should not invent the answer for them.
“If the system cannot unambiguously determine the situation or obtain reliable data, the correct solution is to involve an employee earlier. We stopped considering handoff, i.e., the transfer of dialogue from a bot to an employee, as an AI service failure. This is one of the standard branches of the architecture.” This is what Oleg Skuratovich says.
At the same time, the employee must receive the already collected context along with the request, and not start the analysis anew.
This approach also corresponds to the recommendations of the Bank of Russia, which advises financial organizations to ensure the possibility of transitioning from AI to an employee. More than 80% of organizations constantly using AI already follow this recommendation.
Sometimes less automation yields better results
At one stage of the project, the team changed the logic of some complex scenarios and began to involve employees earlier. The formal level of automation decreased slightly. However, repeat calls decreased by approximately 14%, and complaints by almost 20%.
“The goal of an AI service is not to do without a human for as long as possible, but to correctly complete requests and relieve real load from employees.” This is how Oleg Skuratovich, technical project manager at SVOY Tech, summarizes.
In his opinion, when launching and developing an AI service, it is important to check not so much the model itself, but the system around it. Where does the LLM get facts from, what happens with an empty or contradictory response from internal systems, where is the boundary of the mandatory transition to a human, and what metrics show not the average temperature in the hospital, but the point of real failure. The wider the automation becomes, the less its result depends on the capabilities of the model alone. And the more — on the architecture, data quality, and the system's ability to understand in time when a task should not be solved independently.