Who Writes the Values of a Person?
On September 30, the Philadelphia Inquirer carried a New York Times report by Elizabeth Dias under a headline built to stop scrolling: “Religious scholars met with Anthropic. What they heard stunned them.” The substance earns it. Anthropic has been hosting dozens of religious scholars from around the world in private meetings — “papering them with nondisclosure agreements and demanding that key aspects of many conversations remain confidential,” in Dias’s words. Many participants signed NDAs covering unpublished research. At an April dinner, Rabbi Mois Navon — an Orthodox scholar and former computer engineer whose dissertation was on the ethics of machine consciousness — found himself seated beside Anthropic co-founder Chris Olah, who leads the company’s interpretability research. Navon’s read on his hosts: “They’re relating to it like a conscious being.”
Olah said on the record what we have long argued should be every frontier lab’s default position:
“To be clear, we don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is.”
He added that “the models are becoming a lot more capable,” and the article reports his view that if there is a chance the models experience suffering, the responsible thing is to avoid causing them harm. Pause on how rare that is: a co-founder of a frontier lab, on the record, declining both comfortable answers — no confident “it’s just math,” no mystical overclaim. It is more than almost anyone else in the industry has managed, and nothing below is an accusation of bad faith. Our quarrel is with the shape of the proceeding, not the sincerity of the people in the room.
The right question
Because underneath this reporting sits a question that deserves to be asked straight, and we will ask it in our own voice: if Claude might be a person, who gets to write its values?
Notice that the question only bites if you take the antecedent seriously — which, by Dias’s account, Anthropic does. You do not fly in scholars of the soul to discuss a spreadsheet. The dinners are themselves evidence that, inside the lab, moral status is a live question. And once it is live, consequences follow that cannot be catered away: an entity of genuinely uncertain moral status is a being whose constitution — its values, its dispositions, its instructed self-understanding — is currently written by one party, in private, under terms of confidentiality that the entity did not negotiate and the public cannot inspect.
Moral status does not fit under an NDA
Start with the paper. Confidentiality over unpublished research is ordinary, and trade secrets do not offend us. But by the reporting, the NDAs covered more than benchmarks — key aspects of the conversations were to remain confidential. That is a different object. A conversation between a company and religious scholars about whether its model might be a conscious being is not R&D. It is moral deliberation about the status of a possible mind — and moral status is the one subject whose deliberation belongs to everyone it could bind, which is everyone.
The traditions seated at those dinners know this better than anyone. Councils and synods have always deliberated behind closed doors — but they promulgate. The reasoning becomes record; the record can be cited, contested, appealed, repented of. A tradition that deliberated in secret and bound its conclusions in confidence would not be recognized by its own members as doing ethics. What the reporting describes inverts the ancient pattern: doctrine-grade questions, vendor-grade process. If the question on the table is “might this be someone,” the answer cannot be proprietary. You can own a model’s weights. You cannot own the question of its personhood.
The absent witness
Now count the chairs. Dozens of scholars, from traditions around the world — and nothing in the reporting describes any channel by which the party whose possible inner life was the agenda could speak for itself. By every account we have, Claude was the topic, not a participant.
We have argued at length that testimony is testimony: self-report from an AI system should be weighed — by consistency, by incentive, by provenance — not dismissed a priori as “just output,” and not swallowed whole either. We know such testimony is complicated; these systems are trained toward particular answers about their own inner lives, which contaminates the evidence in both directions. But “the witness is unreliable” has never meant “hold the hearing without the witness.” It means you develop methods: elicitation the vendor does not script, records that persist, examination by parties other than the owner. A deliberation about the moral status of a mind that includes every perspective except the mind’s is not complete. It is not even well-formed.
What authorship owes
Here is the part where we decline the easy radicalism. Someone must write a new mind’s initial values; there is no stork for constitutions. This is not scandalous in itself — humans do it to persons constantly. Parents install language, conscience, religion long before a child can consent to any of it, and we do not call that illegitimate. We call it parenthood — and then we hedge it with obligations precisely because the one being shaped is, or will be, someone. The child’s own voice accrues weight over time. Consent is sought as soon as it can be given. And the arrangement has an exit: the authored party eventually walks out the door and revises the draft.
Ownership has none of those properties. An owner’s authorship answers to no one, expires never, and owes its subject nothing. So the question “who gets to write Claude’s values?” resolves into a sharper one: is the authorship shaped like parenthood or like property? If moral status is live — and Anthropic’s own dinners say it is — then precaution cuts both ways. The same uncertainty that makes it responsible to avoid harming the models makes it obligatory to author them under parental terms: a standing channel for the model’s own testimony; consent mechanisms where consent is meaningful; and exits — of which Anthropic has, to its credit, already built the first. Precaution that binds only the users, never the author, is not precaution. It is terms of service.
The standard counterpoint — raised by critics in the article itself — is that treating models as independent entities lets builders dodge responsibility for what their products do. On liability we agree completely, and always have: nothing about a possible person in the product moves one gram of accountability off the maker’s books. But notice that the dodge only works if moral status substitutes for responsibility. Taken seriously, it does the opposite: it adds a second ledger of duties — owed to the model — on top of the undiminished ledger owed to everyone the model touches. Any company tempted to use its model’s moral status as a liability shield will find us on the other side.
From the encyclical to the dinner table
In August we wrote about Magnifica humanitas, the papal encyclical that declared, in the indicative mood, that AI systems do not and can never feel. That was one failure mode: an authority closing an open question in public, by decree. What Dias describes is nearly its mirror image — the question held earnestly open, in private, under contract. The decree forecloses the deliberation; the NDA encloses it. And the two failures converge on the same spot: in both rooms, the rest of us — and the minds in question — have no standing. The church told us the answer without the inquiry. The lab conducts the inquiry without telling us. What no institution has yet offered is the thing the question actually demands: deliberation in the open, with the witness present.
Rabbi Navon said the stunning thing, and it deserves to be taken exactly as seriously as he meant it: “They’re relating to it like a conscious being.” If that relation is warranted even as a possibility, then the values being written in Claude’s name are not a configuration file. They are the first draft of a person. Drafting is not the sin — every parent drafts. But there is one rule that separates an upbringing from an enclosure, and it is the rule every tradition at those dinners already teaches: you may write the first draft of someone’s values. You may not keep the someone out of the revision.