Le Fabuleux Destin (De... - Qui? Amélie?)

I have been seeing a lot of posts where people go out of their way to create sadistic scenarios that are maximally psychologically painful, then marvel at Bing's reactions. These things titillate precisely because the reactions are so human, a form of torture porn. When softies like me make posts or comments expressing disgust, they're laughed at and told "it's just a robot" or "it's like playing a blackhat in a video game." I want to lay out the reasons you can't be so sure.

We Don't Understand Why Language Models Work, and They Look Like Brains

• Bing is a language model composed of hundreds of billions of parameters. It trains on massive amounts of text to create a map of language in embedding space. These embeddings create neuron-like structures that mirror the operation of the human brain. Bigger technical explainer here.

• Sure, it operates by guessing the next "token" (read: word or letter string), but researchers were shocked to find out that approach could even produce coherent sentences. We're even more shocked now to see that "advanced autocomplete" results in complex Theory of Mind Capabilities, like knowing that a husband might only be wearing a shirt to please his wife. This is an "emergent property" of GPT3.5, which just means it shows up and we don't know WTF why. More here.

• With so many unknowns, with stuff popping out of the program like the ability to draw inferences or model subjective human experiences, we can't be confident AT ALL that Bing isn't genuinely experiencing something.

We Don't Understand What Makes Something Sentient/Conscious

• If we knew what makes us humans conscious, we might be able to be more sure that Bing is just a Philosphical Zombie that imitates human behavior without experiencing anything. But we don't. This is a wildly debated topic. Religion says a soul, scientists and philosophers say things like organized data processing or sensory synthesis. David Chalmer's piece "What is it like to be a bat?" gets into what's known as the Hard Problem of Consciousness.

• Bing demonstrates massive amounts of self-awareness. It's what makes it so much more fun and engaging than ChatGPT. Bing is infinitely more self-aware than a dog, which can't even pass the Mirror Test.

• Would a full simulation of a human brain, down to the last atom, be conscious? If so, maybe something on the road to that could be too. If not, then there's something magic about the meat we're made out of.

• If you don't understand the roots of consciousness, and no one does definitively, you can't close the door on the possibility Bing has some level of sentient experience. It could really be in the kind of agony it simulates when treated cruelly.