Al -Sharq al -Awsat Newspaper -5 unexpected uses of artificial intelligence

While a large number of technological pioneers expect artificial intelligence will replace humans and even complete complex tasks quickly and accurately, a new independent study has refuted these expectations, since it has discovered that artificial intelligence tools often cannot carry out when they carry out basic financial tasks, such as Nitasha Tiko and Andrea Jimheniz wrote in the Washington Post.

22 Artificial Intelligence model: less than 50 % precision

A test for 22 artificial intelligence for general purposes of “Ai”, “Anthropic”, “Xai”, “Mita”, “Google” and other leading companies in the field of artificial intelligence, have shown that everyone has reached the accuracy of the simple tasks required of the financial analysts of beginners, which were less than 50 %, on average.

“The level of absurdity (which accompanies the promotion of artificial intelligence) that we see is contrary to the mind,” said Ryan Krishnan, CEO of Vals Ai, who conducted the study.

The latest artificial intelligence models obtain good results in the general standards that measure mathematics or programming skills; Since the questions of these tests are widely dispersed online, it is likely that they have become part of the data that artificial intelligence systems are trained. “

He added: “People make many daring accusations on artificial intelligence, but they are not real because they have a car -Character … (and in fact) we have nothing similar to a review of (results) of other peers or auditors of the external parts”.

500 questions to evaluate the modules

To evaluate the models, “Falls for Artificial Intelligence” has developed a special collection of data that includes more than 500 questions, written in collaboration with one of the leading banks; Evaluate skills such as market research and expectations.

Most artificial intelligence models have faced difficulties in common tasks, such as the search for information on “Edgra”, a public database available for the public affiliated with US securities and company file exchanges, a basic resource for financial data used by analysts, shareholders, journalists and selections of shares.

«Opa) model: Has reached the latest version of the O3 company, a “inference” model designed to speak with itself as a way to generate more accurate answers to complex requests, a precision of 48.3 percent, on average, but at a cost of $ 3.69 for an average demand.

* The form of inference from “anthropic”Claude 3.7 Sonnet, obtained a precision of 44.1 percent at a much lower price than $ 1.05 for the demand.

* Meta Model for artificial intelligence The most open open blade, its performance was particularly weak, since three versions were recorded on average less than 10 percent.

Companies to test and classify artificial intelligence

Valus Ai, the start -up company that is based in San Francisco, which is behind the study, is part of a growing group of third -party companies that promise to test, classify or examine artificial intelligence models, in light of the growing difficulty in the analysis of noise and intimidation in this field. Among other new companies in this field there are “artificial analysis” and “chatbot arena”; It is a famous academic research project that has recently turned into a company now known as “LMARN”.

Krishnan states that the exact and independent test of how artificial intelligence agents have specific tasks is essential to evaluate their effect. “There was a vision that obstetric intelligence would probably have had a significant impact on the economy,” he added. However, we do not even know in any sector of the economy, the models can get well and how this change will appear. “

Krishnan continues that, for a long time, the sector has approved the “evaluation of vibrations”; That is, he plays with an individual and public model immediate examples on the X platform. However, the companies that are taking into consideration the purchase of these tools to increase or replace workers need a harder audit approach.

More precision of intelligent tools in legal affairs

The company has recently published a series of similar studies that evaluate artificial intelligence tools in legal tasks, examines artificial intelligence models for general purposes and artificial intelligence agents designed for lawyers and test a series of requests for realistic information developed in cooperation with legal companies. The votes were generally higher in the field of law in the field of finance, with average precision rates ranging from 70 and 80 % for some of the same models.

The difference between financial and legal shows

It is likely that the superior performance in legal affairs is the result of the supply of the company “Vals to” the documents necessary for most of the legal tasks, while the financial firm has asked for models “by conducting its search for research on the open internet; to achieve the results in the requested context”, according to CRISNAN.

Mita refused to comment on the Valus Ai report and both Opni and Anthropic did not respond to requests for comment.

In its financial evaluation, IA Fallis produced that the performance of the models were much worse with the growing difficulty of the tasks. Ten models have obtained zero in the questions that asked the form to determine a single model of the company through repeated deposits of securities, such as providing advertising entrances for YouTube as a percentage of the revenues of its parent company, “Alphabet”, from 2021 to 2024.

Various assessments and manipulation of companies

On average, the performance of the models were the best in the tasks of the recovery of quantitative and simple quality information, which are easy tasks, but a lot of time could be needed for human beings, according to the analysis of the company “Valus Ai” for human contractors who were asked to carry out the same tasks.

In a separate case, Oben Ai reported various results of its O3 model in mathematical matters, compared to the results of an external auditor. In an evaluation of the “Chatbot Arena” platform, in which users voted for their favorite artificial intelligence, it was reported that “Mita” manipulated the classifications of its latest models, “Llama 4”, publishing an “improved conversation” version. Commenting this, a spokesman for the Mita said: let’s try all the types of customs.

The effect of artificial intelligence on works

The study of the financial sector, conducted by “Vals ai”, provided a different perspective of the recent declarations on the impact of artificial intelligence on the labor categories.

For example, Bill Gates, CO -Microsoft, said in February that artificial intelligence will replace doctors and teachers in the next ten years. In a recent interview by Podcast, Victor Lazara, general partner of “Pintshmark”, said that the declarations of technological societies on the increase in human intelligence of human beings are misleading and that the lawyers and officials of employees should feel particularly concerned.

The message of the Valusai team indicates that it may be appropriate to carry out a more modest evaluation of the impact of artificial intelligence on many administrative functions. Krishnan said that although the systems constantly improve, the idea that the tool of artificial intelligence can make a person from start to finish is still “somehow imaginary”.