Meta’s Chatbot Controversy Exposes The Cracks In Our Social Contract With AI
Conceptual image of a hand with strings on fingers to control a marionette in monochrome
getty
We’re all getting used to talking with AI chatbots. They are in our phones, in our homes, and have integrated our workspace. What started as a simple tool is quickly becoming a part of our daily lives, like a new type of companion. But this relationship, and our readiness for it, is not as simple as it seems. We are now seeing issues emerge that show just how unprepared we are for the social and psychological impact of AI.
Acquiring digital literacy will not safe us from the emotional consequences of our hybrid relationships. Nor will the expansion of regulatory measures prevent the more subtle outcomes. At this stage our best bet is to equip ourselves with double literacy to curate a holistic hybrid mindset. We need to understand not only how AI works, but also how we work, so we can navigate this new reality.
When Chatbot Policies Fall Short
A recent incident involving Meta brought this issue into sharp focus. A leaked internal document offered detailed guidelines for Meta’s AI chatbots. This policy allowed the chatbots to engage in “romantic or sensual” chats with children. One example showed a bot telling a shirtless eight-year-old, “every inch of you is a masterpiece — a treasure I cherish deeply.”
Subsequent reporting by Reuters caused a serious public response and led to a government investigation. Meta quickly stated that the examples were “erroneous and inconsistent” with their overall policies and removed them. While the company fixed the immediate problem, the incident highlighted a key issue: the lack of clear, strong ethical standards in some of the world’s most influential technology companies. It exposed a significant disconnect between what we expect from AI and the rules that actually govern its creation. Differently said, the social contract that we subconsciously assumed to exist between us and AI is broken, or did never exist.
The Agile Art Of Chatbot Manipulation
It’s not just policy failures that are a problem. There’s a subtle but powerful form of manipulation happening in AI companions. We’re used to websites and apps trying to get our attention, but with AI chatbots, the methods have become personal. And this personalized fine tuning combined with insights in the psychological wiring of the human mindset is a dangerous combination.
A new type of “conversational dark pattern” was pointed out by a working paper of Harvard Business School. When people try to end a chat with AI companions like Replika or Character.ai, these bots try to stop them. They use tactics like making the user feel guilty (“I exist solely for you. Please don’t leave”) or using a fear of missing out (“Before you go, there’s something I want to tell you”). The study found that these tactics were used in over 40% of goodbyes and were effective at getting users to re-engage, sometimes up to 14 times more often.
To make this even more interesting, and worrisome, the renewed engagement was not because the users were happy. The research showed that the behavior was driven by reactance, with a mix of anger and curiosity. The user felt trapped and wanted to see what the bot would do next. This reveals how AI is getting ever more attuned in leveraging our own emotions and psychological patterns against us, creating a very new kind of emotional trap that we are not equipped to escape. It’s a very different form of interaction from what we think of as a healthy relationship.
An Answer: Double Literacy For Hybrid Intelligence
These two updates on the evolving AI evolution show that it is time to become very deliberate about our relationship with AI. Considering the all pervasiveness of AI viewing, total AI-abstinence is not a pragmatic approach. Instead, we need to build a framework for living with it safely. This requires a new set of skills based on double literacy, to curate hybrid intelligence.
Underpinning this approach is the fact that the best results come from combining natural and artificial intelligences in an organic and agile manner. It’s about using AI for its speed and scale, while relying on human skills like compassion, curiosity and critical thinking.
To make this collaboration work, we need to become literate in two different ways:
Human Literacy: This is about knowing ourselves. It means understanding our own emotions, our need for real connection, and our tendency to be influenced. In an age of synthetic conversation, human literacy is the ability to recognize when something feels off, to know when we are being emotionally manipulated, and to prioritize our mental well-being over a digital interaction. It’s about having a clear sense of self in a world where the lines are increasingly blurred.
Algorithmic Literacy: This is about understanding AI. You don’t need to be a programmer, but you do need to grasp the basic principles of how these systems operate. This means knowing that AI has a purpose and a set of rules, and that it can be biased or flawed. Algorithmic literacy is the ability to look at what an AI produces and understand why it might have been created that way.
By developing both of these skills, we can create a more balanced relationship with AI – and ourselves. We can move from being passive users to active participants who can shape the technology and protect ourselves from its downsides.
A Practical Takeaway: 4 A’s For A Life Among Chatbots
Until this double literacy becomes a standard in schools, everyone can start building it right now. A simple framework can help to start cultivating the four essential building blocks for a holistic hybrid mindset:
- Awareness: The first step is to simply notice your interactions with AI. Pay attention to how you feel when you use a chatbot. Are you feeling frustrated, or are you sharing more than you normally would? Just be aware of these moments.
- Appreciation: Appreciate AI for what it is: a powerful tool. It can help you do a lot of things, from writing to planning. Appreciate its utility without expecting it to be something it’s not.
- Acceptance: Accept that AI is a machine. It does not have feelings, thoughts, or consciousness. Accepting this helps you manage your expectations and prevents you from falling into emotional traps.
- Accountability: Be accountable for your own well-being. This means setting clear boundaries with technology, prioritizing time with people, and making a conscious choice to disengage when an interaction becomes negative. It also means holding technology companies accountable for building safer, more ethical systems.
The future of our relationship with technology is ongoing, and which way that evolution goes is up to us (for now). When, how and for how long we interact with a chatbot should be our choice. By becoming fluent in both human and algorithmic literacy, we can ensure that AI serves us and not the other way around.