The experimental set does not fully reflect real-world document verification conditions
The MWS AI team, part of MTS Web Services, tested multimodal neural networks for their ability to detect signs of document forgery. To do this, researchers expanded the open MWS AI Vision Bench benchmark by adding an Anti-fraud task set.
The dataset included 430 images: 200 original documents, 130 with manual alterations, and 100 created by generative models. The neural networks had to classify images, find suspicious elements, identify changes in details, and distinguish originals from AI-generated images.
Models from the Claude, GPT, and GLM families were among the leaders in detecting forgeries. However, high results in document recognition and analysis do not guarantee equally effective detection of alterations. For example, GPT-6.1 Sol took first place in the overall evaluation but third in the Anti-fraud category. Claude Opus 5.5 showed the best result in the latter.
MWS AI noted that the experimental set does not fully reflect real-world document verification conditions. Therefore, Anti-fraud results are published separately from the overall ranking. According to the benchmark developers, specialized solutions are preferable for protection against fraud in real processes.
