⚠️ De site vereist een werkende browser met JavaScript & DOM-ondersteuning voor het beste resultaat.

How will AI sycophancy change us? Early signs are not encouraging

#1 AcidiclyBasicGlitch
Do you ever get the feeling that the people running the world are delulu? That the 1% are living in a completely different universe from the rest of us? You’re not the only one. Even some tech elites are starting to worry about their peers’ grasp on reality. “CEOs are uniquely prone to AI psychosis,” Aaron Levie, a co-founder of the enterprise cloud company Box, declared on X last month. His reasoning for this? “They’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI. So when they play with AI, they see the happy path results, often not considering the next 10 or 20 things that have to happen to get sustainable results from agents.”

In other words: CEOs are so high up the food chain that they don’t understand the human labour that goes into turning an error-riddled AI creation into something that functions properly in a business context. They are desperate to replace their annoying and expensive human labour with compliant AI models, but grossly overestimate what the technology can do. Meanwhile, the industry is rushing out overhyped AI solutions without properly stress-testing them.

This collective euphoria has resulted in some predictable disasters. In April, an AI coding agent powered by Anthropic’s Claude went berserk and deleted a company called PocketOS’s entire production database, along with backups. Jeremy Crane, PocketOS’s founder, later mused on X that this sort of failure was “inevitable” because the industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”. To recall Facebook’s old mantra: it’s moving fast and breaking things.

Those things include our brains and grasp on reality. There’s a viral quote floating around the internet that quips: “The dumbest person you know is currently being told ‘You’re absolutely right!’ by ChatGPT.” Our tech overlords designed AI chatbots to be obsequious because it’s good business: having your opinions and feelings constantly validated increases user engagement.

But what else does this constant flattery do? While we’re still figuring that out, early studies aren’t reassuring. Research published in the Lancet Psychiatry in March found chatbots can encourage delusional thinking, particularly in people already vulnerable to developing psychotic symptoms. And a recent study from Stanford computer scientists found LLM “sycophancy can undermine users’ capacity for self-correction and responsible decision-making”. There is a pressing need, the study stresses, “to address AI sycophancy as a societal risk”.

#2 AcidiclyBasicGlitch
I feel like there needs to be a more accurate name for whatever the phenomenon of AI fueled sycophantic delusion is. Because clearly it's something, and more and more people seem to be noticing it.

It's like the people who are closest to the person spiraling can see they're watching a train headed towards a brick wall, and they can't stop it. But the person experiencing the delusion is often still fairly high functioning, so other people who are a bit more removed from the situation tend to to be very dismissive of the comparison to psychosis.

At least until the poor decision making of the delusional person expands to their world, and everything starts going tits up for them too.
Then suddenly they're like "Gorsh, he just seemed so confident and he was wearing a suit, so I assumed he had his shit together. But now that he's draining MY bank account, I think something might be wrong with his brain."

Like what's a better way to describe the high functioning level of AI fueled delusional? Where he hasn't sold all of his assets and moved to the desert so he can fully devote himself to whatever breakthrough he believes he's on the verge of discovering (yet), but somebody should really stop this guy before he does any more damage to himself or others?

Soft Psychosis?
#3 sylver_dragon
LLMs are a tool. Like all tools, we are going to go though a learning curve as we adapt to safe usage of that tool. LLMs cratering companies would be a really tame way to learn those lessons. Usually, we don't start writing regulations around tools until we have buckets of blood to write those regulations with.
#4 Miller
There is already a word for it and the word is misanthropy.
#5 Cherry
We see it in the old form of warmonger it’s a form of destruction based on blind trust…or fear.

Is the problem the leader, the administration or the masses?

I have been chatting through power structures and Macbeth recently with a child and it’s interesting the parallels with actions fuelled by ‘imaginary forces’ if that makes sense. This can be religion, nationalism etc but I find it so strange how tech which is traditionally driven from math and science etc seems to have fallen to the same fallacy. 🙂
#6 CompactFlax
I don’t think AI is the problem. It merely amplifies the delusional state that our global leadership in business and government is in.

It’s the 🔗Business Idiot and 🔗Rot Economy theory, and AI is just making it obvious to more people. “What if my CEO is actually an idiot? What if he’s just been lucky on some of the decisions he’s made and we forget about all the stupid stuff he does because it changes from month to month?”
#7 trijste
Not used to my calculator telling me I’m brilliant 👍
#8 AcidiclyBasicGlitch
In many of the cases in the essay, chatbots responded to users with mystical language to suggest that users have heightened spiritual importance. The bots also implied that users were speaking with a cosmic being who was using the chatbot as a medium.


Idk, I feel like a true misanthrope would be able to read these 2 sentences and at the very least immediately swear off AI chat bots, if not move to a completely isolated cabin in the woods with no electricity or internet.

It's like the broligarchy birthed a bastard child that's just a composite of humanity's worst qualities. All the traits they projected on to the masses as evidence to justify their antidemocratic savior complex bullshit. Impulsive/poor decision making, superstition, paranoia, narcissism.

They're investors, not inventors, but they're 100% convinced this bullshit is going to be man's greatest and longest lasting contribution to existence, and that their names will go down in history as the technocratic leaders who changed the world... And their own delusions are probably being fueled by the same fucking chat bots, and sharing data between platforms in a never ending sycophantic circle jerk that's spreading exponentially like the dumbest fucking virus to ever exist... I just... 🤦‍♀️

Tldr: These people don't deserve the title of misanthrope.

Edit: I guess I should add the caveat that in the case destruction of humanity/civilization was always the actual goal of some 4-D misanthropic chess master behind the curtain, then bravo and hats off to them for being a real one, I guess? 😂
#9 Miller
I would say that it does represent a type of misanthropy lite in that its source is a lack of faith in humanity based in unaquaintance. The irony is the LLM are only a distillation of human expression, boiled down to something close to absurdity. A distrust of humanity has made them reliant on something grown from its excesses.
#10 Triumph
I'm just here to watch it happen at this point. It's not going to get better for the proles without civil unrest at best.
#11 dbtng
“The dumbest person you know is currently being told ‘You’re absolutely right!’ by ChatGPT.”
#12 impairedimperator
Oh, don't worry, fully autonomous AI terminators have already gotten confirmed kills in real life! 🔥
#13 eestileib
Engineers are extremely vulnerable to bullshit because they think they aren't.
#14 MasterBlaster
Edge Runners anime already covered it: cyber psychosis.

🔗 Related topics

spacer
Shoutbox 0
💬 Shoutbox
Mayumi: Very nice, daddy!
webmastahh: From now on, this website is expanding to an international audience!! 🙂😄
Webmastahhh: Joined a random match in krunker, got destroyed instantly, left
Webmastahhh: “just one match”... 3 hours later. 🙁
Webmastahhh: So I’ve got Paralives, new James Bond, No Man’s Sky and Final Fantasy XIV on my list…What should I start with first?
Webmastahhh: Stay cool, stay hydrated, and keep stacking those wins. Whether you're chilling in the shade or battling online, today is a perfect day to game! ️Who's playing right now?
beep: PlayStation had decided that you can't play your purchased games offline and Xbox is deep in it's enshittification. Quick somebody call an ambulance!
The Picard Maneuver: People are saying not enough eyes either. Sad.
7101334: What happened to good destruction-physics focused games?
CaptainBasculin: A really creative way to ask getting into a playtest
ragingHungryPanda: My friend said this yesterday, so I made it a meme
cannedtuna: Just got my first Steam Machine and I’m excited to try it out. Just love that clear purple casing.
The Picard Maneuver: The horror!
The Picard Maneuver: Problem solved
Steelkrill Studio: I’m a solo developer creating a detective horror game where you search for victims by a boat, inspired by the real-life Island of the Dolls in Mexico. I’d really appreciate any feedback!
cannedtuna: The bestest boy will always be by your side. ALWAYS.
Paradachshund: With as few spoilers as possible, how early access is subnautica 2?
QuentinCallaghan: The Original Doom Soundtrack Is Officially In The Library Of Congress
cannedtuna: Publisher CEO for Subnautica asked ChatGPT how to avoid paying out bonuses
webmastahh: FINAL FANTASY VII REBIRTH is now available on Nintendo Switch™ 2, Xbox Series X|S and Xbox on PC as part of Xbox Play Anywhere.
RandyEmons: hello world

250 left
smile grin sad wink tongue out surprised confused laughing XD heart
🏠 Start 👾 News 💾 Downloads 💬 Community Profile Profile