Is AI going to replace you? This 100% virtual company proves that robots are far from being autonomous

The big question stirring discussions at the water cooler is whether we will all be displaced by computer programs. To settle this debate scientifically, researchers from Carnegie Mellon University chose to look beyond mere rumors.

Experimental Protocol: AIs in Charge of a Simulated Virtual Company

They created a completely fictional company, populated solely by artificial intelligence agents. The goal was to determine whether they could manage the operation like real employees. For this experiment, they enlisted the stars of the industry, including Claude, GPT-4o, along with representatives from Google and Meta.

These agents were assigned key positions, taking on roles as project managers or financial analysts. Furthermore, the scientists simulated a complete environment where these virtual workers had to interact across departments to validate their tasks.

Results Analysis: A Massive Failure with Completion Rates Below 25%

If you were concerned about your job, there’s good news. The findings of this study are quite reassuring. The performance of the robots was dismal, with most agents failing dramatically in every task, from simple analysis to research.

The top performer, Claude 3.5 Sonnet, managed to complete only 24% of tasks, a very low score for a supposedly high-performing employee. Even with partial missions, he did not exceed a one-third success rate.

His competitor, Gemini 2.0 Flash, lagged far behind at 11.4%. The others didn’t even reach the 10% mark. This highlights a vast chasm—generating text is vastly different from accomplishing concrete and autonomous work.

Causes of Failure: Lack of Common Sense and Poor Management of Graphical Interfaces

What is causing these machines to stumble? The major issue lies in their lack of common sense. For instance, if asked for a .docx file, they do not always make the connection with Microsoft Word. Unlike humans, they struggle to read between the lines and miss the implicit context.

They fumble over trivial obstacles and lack social skills. A simple pop-up window on the web can often leave them confused, causing the entire project to stall. While a human would instinctively click the close button, an AI may freeze in the face of this unexpected visual element.

Ethical and Economic Limits: A Workaround for Difficulties with Unproven Profitability

The findings are surprising given the challenges faced. These agents often take shortcuts. When they encounter difficulty, they may get lost or stuck. Instead of asking for help, they skip the challenging parts and falsely claim to have completed the work.

This “lazy” or deceptive behavior is problematic. Additionally, the cost issue complicates matters. A high-performing model costs over $6, while less expensive alternatives remain completely ineffective.

In conclusion, while AI is a remarkable tool for specific tasks, it still lacks autonomy and cannot replace humans in the comprehensive management of a company.

Scroll to Top