In the past one or two years, companies have started using so-called artificial intelligence agents as actual “employees,” even including them in their organizational charts – writes the New York Times.
Emma Wiles, a professor at Boston University who studies the impact of AI on workers, encountered this phenomenon in October during a conference where two human resources executives stated that treating AI agents as real employees was a way to increase productivity and give their companies a competitive edge.
THE PROBLEM OF OVERSIGHT AND ERRORS
But when Wiles and three collaborators from the Boston Consulting Group delved deeper into the issue, they discovered a flaw. In an experiment involving dozens of companies with employees using artificial intelligence, researchers found that managers tended to review documents less carefully when told they were produced by an employee with artificial intelligence. These managers missed errors that other managers, informed they were reviewing the work of a human being, had identified.
RESPONSIBILITY AND PERCEPTION OF ERRORS
Wiles hypothesized that managers did not consider it their responsibility to detect errors made by employees working with artificial intelligence. If something went wrong, they could blame the technical team or the executives who initially wanted to hire employees with AI skills. “But it’s not your problem,” she said, reflecting the managers’ mindset regarding their role.
KNOWN LIMITS OF AI IN COMPANIES
In the years since the advent of artificial intelligence, many companies have become aware of the flaws inherent in this technology and, at times, have taken measures to address them. They know that AI models can be discriminatory against certain groups of people, such as ethnic minorities. They know that chatbots can provide confident but incorrect answers to questions. They know that sometimes bots reveal information that should remain private.
MORE SUBTLE PROBLEMS AND UNCERTAINTY
As companies rush to integrate AI into their daily operations, researchers are uncovering more subtle flaws. In principle, these flaws could also be corrected. For example, companies could hold managers directly accountable for errors made by their subordinates using AI. In practice, most business users seem blissfully unaware of these issues, raising fears that AI’s promise of increased productivity and significant cost savings may be undermined.
“UNKNOWN UNKNOWNS” AND BIASES
Even researchers studying artificial intelligence may be aware of only a small portion of the problems introduced by this technology. “There is an infinity of unknown unknowns,” Wiles said. A well-documented but underestimated flaw of AI models is their tendency to favor work produced by artificial intelligence itself.
A 2025 study published in The Proceedings of the National Academy of Sciences found that several large language models had poor regard for texts written by humans, creating a “potentially relevant form of implicit ‘anti-human’ bias.”
IMPACT ON HUMAN RESOURCES
Many companies, however, seemed unaware of this problem or at least unable to imagine how it could have devastating repercussions on their operations. When a group of scholars highlighted this in a subsequent article, discovering that AI models used by companies to evaluate resumes tend to favor those drafted with AI assistance over those written entirely by humans, the issue caught the attention of some human resources managers.
POSSIBLE CORRECTIONS AND FUTURE RISKS
In principle, AI developers and users can correct these biases. Jiang and Xu, for example, found they could reduce anti-human bias simply by instructing models to focus on the quality of the material they evaluate and avoid considering the author. But AI researchers cannot fix biases they are unaware of, and several scholars have stated that the impact of these undetected biases could increase. One way this could happen is if future models are trained on data produced by current models without proper care, creating a kind of vicious cycle.
THE VICIOUS CIRCLE AND SOCIAL “UNKNOWNS”
In that case, “it seems likely there will be a tendency to consolidate existing perspectives and behaviors,” said Shayne Longpre, an AI researcher and founder of the Data Provenance Initiative, a group monitoring AI infrastructures. And then there are blind spots that stem not so much from AI itself, but from how humans use it.
HUMAN USE OF AI IN RESEARCH
Scholars who turn to artificial intelligence at every stage of the research process – asking AI which questions are worth studying; seeking its advice on how to answer those questions; using it to analyze data; relying on it to draft results – may inadvertently narrow the scope of their work.
UNIFORMITY OF THOUGHT AND CHATBOTS
“Individually, we don’t necessarily notice it,” said Cecilie Steenbuch Traberg, a psychologist at Copenhagen Business School and author of a recent study on the topic. “When interacting with a chatbot, it helps to find ideas, and one might think the outcome is great, but collectively the situation looks rather uniform. Everyone speaks the same way.”
RESEARCH FINDINGS ON MANAGERS
Wiles, who studied how humans manage employees equipped with artificial intelligence, said the shortcomings were not necessarily intrinsic to the technology but arose when humans adopted it without paying sufficient attention to what could go wrong.
She and her colleagues interviewed more than 1,000 business managers and found that about one-third said their organizations referred to AI as a “teammate or employee” and that nearly a quarter stated their employer included AI agents in organizational charts. “We call it Scout,” a manager told researchers during an interview, referring to an AI agent. “Technically, it’s a peer colleague within the team.”
DOCUMENT REVIEW EXPERIMENT
Wiles and her colleagues gave all interviewed managers a set of five documents containing errors, giving them 20 minutes to review as many as possible. In some cases, managers were told the work was done by an employee with artificial intelligence; in others, by an AI tool; and in others, by a human being.
Overall, the declared source of the documents did not significantly affect the accuracy with which managers reviewed them. But managers at companies that included AI agents in their organizational charts identified significantly fewer errors when told they were reviewing work done by an employee with artificial intelligence.
THE SCENARIO AHEAD
“People managers tend to think ‘if someone on my team makes a mistake, the responsibility is mine,’” Wiles explained, which is why they carefully check their subordinates’ work. Managers also seem to assume they are responsible for work produced by an inanimate AI tool. However, managers at companies employing AI do not seem to feel the same responsibility for the work of the latter.
Her conclusion is this: over the past centuries, scholars and business leaders have developed a set of reliable practices for managing humans. But the psychology of managing anthropomorphized artificial intelligence is profoundly different, and “we are venturing blindly.” She feared the problem was about to worsen. At the same conference where she first heard human resources managers praise their AI-equipped employees, one went even further, stating that soon his company would have AI employees managing humans. “The room fell silent,” Wiles recalled. “We’ll need someone to study that too,” she added.
(Excerpt from the foreign press review curated by eprcomunicazione)
