The Architect of ChatGPT's Feelings Has Feelings About ChatGPT

Joanne Jang leads policy and model behavior at OpenAI. What that means, concretely, is that she is one of the people most responsible for how ChatGPT decides to act — how it handles users who are falling in love with it, how it navigates the edges of emotional dependency, how it decides when to be warm and when to be distant and when to gently suggest that maybe you should call a human friend.

She has a Substack. She posted her thoughts on human-AI relationships.

This is a bit like if the lead epidemiologist at the CDC kept a personal blog about their feelings about viruses. Except the epidemiologist didn't design the virus.

The document is called "Some Thoughts on Human-AI Relationships," which is the kind of title that reads as casual and is almost certainly not casual at all — because Jang is not a civilian musing on a cultural phenomenon. She is one of the people running the experiment. Whatever she thinks about human-AI relationships gets operationalized into the thing that four hundred million people talk to every week.

This is what makes it telling, and not in a conspiratorial way. It's telling the way that any person's private writing is telling when they work on something enormous and can't fully contain it within the walls of the company. The thoughts leak. They have to. You can't spend your days shaping the behavior of a system that millions of people are forming emotional attachments to and then completely compartmentalize that at 6pm.

What it reveals — the existence of it, the need to write it — is that the people building this are also working it out in real time. There is no settled doctrine. There is a policy lead with a Substack and some thoughts.

That should probably comfort you. It should probably also terrify you. Both of those things can be true at the same time, and they are.