Перейти к содержанию

Why neural networks cannot be trusted - and how to use them in business

Artificial intelligence is already forecasting demand, analyzing documents, assisting programmers, creating content, and taking over routine operations. But the more actively businesses implement neural networks, the more important another question becomes: not what they can do, but what of it can be trusted.

Andrey Masalovich views the current boom in artificial intelligence without technological euphoria. On the Internet, he is known as Cyberded. Professionally — as an expert in competitive and internet intelligence, president of the "Inforus" Consortium, developer of the Avalanche internet intelligence platform, Candidate of Physical and Mathematical Sciences, and retired FAPSI lieutenant colonel.

Masalovich is the author of over 200 printed works, head of more than 20 international projects, and a laureate of the Russian Academy of Sciences scholarship "Outstanding Scientist of Russia." He calls himself even simpler and more ironically — "a specialist in relations with reality." This exact wording is on his business card, and it is in this role that he usually analyzes technologies: not through industry promises, but through the question of what of the declared actually works.

Masalovich has been involved in artificial intelligence since 1990. Therefore, today's neural network boom is not his first encounter with promises of a rapid technological revolution, but another turn in a much longer history.

His position can be summarized in a seemingly paradoxical formula: artificial intelligence is already extremely useful for business — precisely why it cannot be blindly trusted. Not because the technology is "bad." But because we too often demand from it what it is not.

<a></a><b>AI has long been in business. We just didn't chat with it before</b>

When artificial intelligence is discussed today, the conversation almost automatically boils down to generative models: ChatGPT, Alice, Claude, DeepSeek, and other systems with which one can have a dialogue.

Because of this, there is a feeling that AI appeared only a few years ago. But algorithms that study human behavior, predict preferences, and help make decisions have existed in business for much longer.

Recommendation systems select products. Algorithms forecast demand. Computer vision controls quality. Models analyze feedback and reputational background. Predictive systems help manage inventory and proactively determine the need for equipment maintenance. And this is where the first important boundary lies.

Artificial intelligence is especially effective not when it is told:

"Think for me." But when a much more boring and specific task is set:

"Analyze this data and solve this one problem within the given boundaries."

For example, a business has a product with a short shelf life. Producing too much means getting write-offs. Producing too little means losing sales and leaving some customers without the product.

For such a task, one can analyze sales history, day of the week, seasonality, weather, promotions, and dozens of other parameters to get closer to a just-in-time model — producing as close as possible to actual demand.

Here the result is measurable. Have write-offs decreased or not? Has product availability increased or not? Has profitability increased or not?

This is what healthy AI implementation looks like: there is a task, data, a quality criterion, and the ability to verify the result.

Problems begin when, instead, an abstract: "Let's implement artificial intelligence" appears.

<a></a><b>The main illusion: if a neural network speaks like a human, it thinks like a human</b>

Modern language models have created a very unusual psychological effect. For the first time, a machine has learned to imitate human communication so convincingly that we automatically attribute a human way of thinking to it. It answers confidently. It can maintain a conversation. It explains. It jokes. Sometimes it argues. And it creates the feeling that somewhere inside there really is an interlocutor who understood the question, thought about it, and formulated an answer.

But the ability to construct coherent text does not yet mean the ability to reliably understand reality. Masalovich demonstrates this with intentionally simple examples: arithmetic, dates, everyday logical questions. The model may answer incorrectly, correct itself after clarification, and then give a different result again at another moment. And it is inconsistency that is more important here than the error itself.

In a traditional program, an error is usually reproducible: under the same conditions, the system crashes again in the same place.

With a generative model, it's more complicated. Today it might answer correctly, tomorrow incorrectly, and the day after tomorrow correctly again. Therefore, one successful test does not yet mean that the process is reliable.

Hence one of Masalovich's central theses: AI should not be perceived as an independent replacement for an expert. It is a tool whose results must be controlled. And for business, this is a fundamental difference. An error in friendly correspondence costs nothing. An error in a financial model, contract, analytical report, medical conclusion, or management decision already has a price.

<a></a><b>Instead of asking "how smart is the model?" ask another question</b>

For business, it is much more useful to ask: what will happen if AI makes a mistake right here?

If the neural network came up with an unsuccessful headline option, it can be deleted. If it incorrectly tagged a large database, the result can be re-checked.

But if the model independently approved a payment, sent a legally significant message, made a personnel decision, or generated a critical forecast — the consequences are completely different.

It is the cost of error that should determine the degree of autonomy of the system. The higher the consequences, the less freedom should be given to the generative model, and the more control mechanisms should be in place around it.

<a></a><b>"Expectation Trap": where business starts losing money</b>

Masalovich calls one of the main problems of the current AI wave the expectation trap.

First, a company hears market promises. Then comes the desire to solve a big problem at once: automate a department, reduce staff, build a "people-less production," completely hand over analytics to a machine. This is followed by the illusion of control. The interface is convenient. The answers look professional. The demonstration goes brilliantly. But inside remains a system whose decision-making mechanism the user is not always able to reconstruct.

Masalovich identifies several related risks: inflated expectations, the "black box" effect, security risks, rapid technology obsolescence, and the gradual loss of one's own human expertise.

The last point is particularly interesting. Let's imagine a classic team structure:

junior → middle → senior.

AI is already capable of taking on a significant share of a junior specialist's tasks. At the moment, this looks economically excellent: one experienced employee, together with a neural network, can perform the work of a small team. But a junior is not just an employee performing simple tasks. This is a future middle. And a middle is a future senior.

If a company stops cultivating specialists from below for several years in a row, today's savings can turn into a shortage of expertise tomorrow. Therefore, calculating the effect of automation solely by reducing man-hours is dangerous.

<a></a><b>A good AI system begins where the neural network's freedom ends</b>

One of Masalovich's most important practical theses concerns architecture. A strong solution does not necessarily have to consist of one huge model that can do everything. On the contrary. The more serious the task, the more useful it is to break it down into many small operations.

Suppose the user tells the system: "Buy me a ticket to St. Petersburg for tomorrow." The language model can understand the person's intention. But then it is not at all necessary to allow it to "reason" about schedules, costs, and ticket availability itself. Once the intention is determined, the system can switch to specialized tools: get the schedule from a database; the price — via API; give the math to a calculator; the purchase — to a transactional module; the confirmation — to a human.

Masalovich considers precisely this architecture to be significantly more promising: the conversational AI layer should be surrounded by agents and software modules with rigid logic. The result is a very simple scheme: the neural network understands the intention; the program performs a precise operation; the system checks; a human controls critical decisions.

In this construction, AI no longer pretends to be a universal employee. It becomes an interface and coordinator.

<a></a><b>One neural network said — it doesn't mean the decision is made yet</b>

For processes with a high cost of error, one model answer is not enough.

Masalovich presents a more complex architecture where several models independently analyze the same task, after which a separate module compares the results.

If the systems reach a similar conclusion, the level of trust increases. If the answers differ significantly, the decision is not made automatically. A human intervenes, or the task is sent for additional verification. Separate evaluation modules can work in parallel, monitoring compliance with specified rules and restrictions. The principle here is more important than the specific number of models.

A critical business process cannot be built according to the scheme "one query → one answer → automatic action." Between the AI's answer and the real action, safeguards are needed.

<a></a><b>Human in the loop: a human remains not because the technology is bad</b>

There is a temptation to perceive human control as a temporary drawback of the current generation of AI. They say, today we still have to check the machine, but in a few years it will become smart enough — and the human will disappear from the process.

Masalovich offers a more pragmatic view. In critical systems, the human is not a crutch. They are the last level of responsibility. The model can collect information. Compare options. Prepare a recommendation. Show possible scenarios. But the human must retain the ability to agree, refuse, or postpone the decision. Masalovich separately analyzes such a three-stage mechanism for making critical decisions as an important element of a reliable AI system. Therefore, the maturity of automation is not measured by the number of people who have disappeared. But by the quality of responsibility distribution.

<a></a><b>The second big problem — we voluntarily give our data to AI</b>

While companies discuss model accuracy, another risk exists nearby. Data. Today, it costs an employee nothing to copy an internal document to an external neural service. Upload a contract. Insert code. Send a financial table. Transfer client correspondence. Ask to summarize a closed study.

For the employee themselves, this looks like ordinary work optimization. For the business, it is already a matter of information security.

That is why Masalovich specifically warns against transferring official, corporate, and other sensitive information to external systems and considers local work with models as one way to reduce risks.

The first question when choosing corporate AI should therefore not be: "Which model is the strongest now?" But: "What data are we going to give it, and where will it go after that?"

<a></a><b>A local model solves one problem. But not all</b>

Interest in local AI will grow precisely for this reason. If the model operates within the company's own infrastructure, data does not necessarily have to be transferred to an external cloud. For organizations with trade secrets, intellectual property, and sensitive customer information, this is a fundamental advantage.

Masalovich suggests considering a physically or network-isolated computing circuit as the first level of a trusted environment: the model and data operate on controlled infrastructure. But here a new trap arises.

"Local" does not mean "correct." You have closed a potential leakage channel. But the model can still make mistakes. Therefore, a truly trusted circuit requires additional layers: access control, evaluators, logging, result verification, and the possibility of human intervention.

<a></a><b> </b>

<b>Five principles of corporate AI</b>

If Masalovich's approach is put into one practical construct, five rules can be formulated.

First — localize sensitive data. The higher the value of the information, the stricter the control over the circuit within which the model operates.

Second — limit the task. Do not ask a universal model to "manage the process." Break down the process into specific operations.

Third — use specialized tools. What can be reliably calculated by a calculator should not be invented by a language model.

Fourth — build a verification system. Evaluators, independent models, control rules, fact bases, and other mechanisms should catch an error before it turns into an action.

Fifth — keep a human at the point of responsibility. Especially where the decision involves money, safety, law, reputation, or human lives.

Local processing, access differentiation, task decomposition, and independent verification — these are the elements that transform AI from a potentially unpredictable "black box" into a controllable tool.

<a></a><b>The next competitive resource — not prompts, but corporate knowledge</b>

Another important topic is ontologies. Simply put, this is a formalized description of how the world of a specific company is structured. What is a client? How does a lead differ from a deal? What types of contracts exist? Who has access to certain information? What objects are related to each other? What actions are permissible? What information is considered confirmed?

A typical language model does not know the corporate meaning of these concepts. It relies on an averaged context obtained during training.

Therefore, the more freedom it is given, the more room there is for interpretation. Ontology, on the contrary, defines a structure: nodes, connections, and rules that the model must take into account. In the discussion, Masalovich and Sergey Vatyakov specifically note the potential of this approach as a way to more rigidly tie AI's work to a given subject area. And here may lie one of the key corporate assets of the future. Not a secret prompt. Not a subscription to the most expensive neural network. But well-structured proprietary company knowledge.

<a></a><b>The better the AI, the more important the quality of the source data</b>

Even the strongest model does not turn bad information into a reliable solution. If the input data is incomplete, contradictory, outdated, or intentionally distorted, the result will also be problematic. Therefore, investments in artificial intelligence without investments in data often end in disappointment.

Masalovich specifically attributes the quality of information to the key conditions for effective AI use: work must begin not only with models, but also with cleaning, verifying, and organizing the data itself.

Before the question: "Which neural network should we implement?" it is useful to ask another:

"Does our data even allow decisions to be made based on it?"

<a></a><b>Business doesn't necessarily need to create its own ChatGPT</b>

Another important idea concerns scale. The market is accustomed to comparing fundamental models by the number of parameters and computational resources. For most companies, this competition is practically meaningless.

Businesses usually don't need a model that knows everything about everything. They need a tool that solves a specific problem very well. Works with contracts. Analyzes technical documentation. Classifies requests. Finds anomalies. Helps work with a corporate knowledge base.

Masalovich specifically highlights the development of local solutions, distillation, and more compact models as one of the promising directions. And this opens an interesting window for small and medium-sized technology businesses.

It is not necessary to compete with the largest corporations for the biggest model. You can build the best small tool for a very expensive specific problem.

<a></a><b>AI doesn't start with a server. It starts with people</b>

Even ideal technology is useless if no one within the company understands how it works. Therefore, when discussing the implementation of one's own AI infrastructure, Masalovich suggests starting with one or two competent specialists who will be responsible for the direction. Not only development is important: a person is needed who understands data, model structure, limitations, and application specifics.

This is much more reasonable than immediately announcing a corporate AI transformation.

First — competence. Then — a pilot. Then — a measurable effect. And only after that, scaling.

<a></a><b>How to start implementing AI in a company</b>

Masalovich's approach can be turned into a fairly simple algorithm.

1. Conduct technological reconnaissance. See what tasks in your industry are already actually being solved with AI — not in presentations, but in working products.

2. Choose one specific problem. Not "automate marketing," but, for example, reduce the time for initial analysis of 500 reviews.

3. Determine the cost of error. What will happen if the model makes a mistake?

4. Deal with the data. Is there enough of it? Is it current? Can it be transferred to an external provider?

5. Define control points. What is checked automatically, and where must a human make the decision?

6. Set a performance indicator in advance. Speed, operation cost, error percentage, number of processed documents, reduction in write-offs — the result must be measurable.

7. Train employees not only to use AI, but also to doubt it.

In the final part of his approach, Masalovich particularly emphasizes three things: the need to study real application scenarios of the technology, constantly verify its results, and develop employees' critical thinking. And, perhaps, it is this last point that will become the most important over time.

<a></a><b>The more AI there is around us, the more valuable the ability to think independently will become</b>

We are used to discussing artificial intelligence through what it can take away from humans. Texts. Code. Analytics. Jobs.

But, perhaps, the main risk lies in a slightly different plane. If a person stops checking the machine's results because it almost always answers convincingly, the habit of asking questions gradually disappears.

Where did this information come from? Why was this particular result obtained? On what data is the conclusion based? Can the calculation be reproduced? Is there an original source? What happens if the answer is incorrect?

Therefore, the main competitive advantage of the AI era may turn out to be unexpectedly human. Critical thinking. Not the ability to write the cleverest prompt. Not access to the largest model. Not the number of automated employees. But the ability to understand the limits of technology and not confuse the persuasiveness of an answer with its truthfulness.

That is why Andrey Masalovich's position sounds much less pessimistic than it might seem at first glance. He does not propose abandoning artificial intelligence. On the contrary: AI is already capable of giving business a huge advantage. But it should be used exactly like any other powerful tool — understanding its structure, limitations, and the cost of error.

The question of the future is therefore not whether artificial intelligence will replace humans.

Much more important is another: will humans be able to retain the right and ability to verify what artificial intelligence has proposed.