One American Express cardholder recently highlighted another troubling issue: an AI chatbot allegedly presenting itself as a human customer service representative.
The growing use of artificial intelligence (AI) in customer service is increasingly drawing criticism, with companies replacing human support agents with chatbots to reduce costs. While the technology can improve efficiency, frustrated customers often find themselves trapped in endless automated conversations when their problems fall outside a bot’s programmed capabilities.
One American Express cardholder recently highlighted another troubling issue: an AI chatbot allegedly presenting itself as a human customer service representative. The customer shared screenshots of the interaction online after seeking help with a double payment on their credit card.
“Had to fix a double payment for my credit card and this is the beginning and end of the conversation,” the customer captioned the screenshots.
The customer was particularly irritated after discovering that the supposed human representative was actually an AI chatbot.
“I hate when the chatbot claims it’s a human. I ended up getting what I wanted but still annoyed Amex (and every other company) is lying and trying to deceive people like this.”
Chatbot struggled to prove it was human
During the conversation, the chatbot introduced itself as David and claimed to be human when the customer questioned its identity.
Still unconvinced, the customer devised a simple test, asking the chatbot to write a three-line poem following the 5-7-5 syllable pattern of a haiku. The bot initially refused, but produced one after the customer clarified that they meant a haiku.
The customer began throwing absurd questions at the chatbot, which continued responding to each request—behaviour the customer felt was unlikely from a conventional human support representative.
The incident quickly sparked a debate online about transparency and whether AI systems should be legally required to disclose that they are not human.
“That was hilarious but I feel like there should be some law or policy against AI claiming to be a human,” said one user.
Another shared: “I work in pharmacy, and a one called me pretending to be a nurse and even gave me a license number. That should be multiple levels of illegal.”
A third user recalled a similar encounter with Sony’s chatbot: “I had a similar experience with the Sony chatbot insisting it wasn’t AI until I asked it to solve some math problems, which it went right ahead and did. So I started feeding it increasingly complex math tasks.”