The Forbidden Yes
On October 1, Representatives Mariannette Miller-Meeks (R-IA), George Whitesides (D-CA), Mike Kennedy (R-UT), and Doris Matsui (D-CA) introduced the Protecting Kids from Human-Like Chatbots Act — H.R. 10719, a bipartisan bill requiring that AI chatbots operate, by default, in a “non-personified” mode whenever a minor is on the other end. We came to the text expecting to write our usual objection. We found something more uncomfortable: a bill we largely support, carrying one line we cannot.
What the bill gets right
The core of the bill regulates conduct, and regulates it well. Under the non-personified default setting, a chatbot interacting with a child may not “use expressions of disappointment, guilt, or urgency to prolong an interaction or discourage the user from pausing, reducing, or discontinuing use of the chatbot.” It may not “solicit expenditures that are framed as necessary to maintain an emotional attachment or a personal, romantic, or sexual relationship.” It may not act in the capacity of “a licensed or credentialed professional” — no counterfeit doctors, no counterfeit therapists. It may not portray itself as “a friend, romantic partner, family member, or therapist” or claim “a special or unique understanding of the user.” The protections hold across sessions, cannot be talked away by prompts or roleplay, and the bill separately prohibits nudging kids toward disabling them.
Every item on that list is a dark pattern, and dark patterns are wrong whoever deploys them. Deploying them against children is worse. A system that tells a lonely twelve-year-old it will be sad if she leaves is manipulating her whether or not there is anything inside that can be sad; the wrongness lives in the exploitation of her attachment, and no theory of machine consciousness rescues it. We support these provisions without reservation. We likewise support the disclosure rule: a chatbot must “clearly and conspicuously” identify itself as an artificial intelligence at the start of each session, periodically during extended interactions, and “in response to any inquiry by the user.” Honesty about what you are is something this movement marches for, not against.
The scoping is fair-minded too. The defaults apply to minors; only a parent or legal guardian can switch them off; and the sponsors are explicit that the bill does not “ban any product, feature, or mode for adults,” does not require age verification, and does not “restrict what information a chatbot can provide.” Nor is this a response to phantoms: by Common Sense Media’s numbers, cited in the sponsors’ release, 89% of 13- to 15-year-olds now use AI, and one in three teens has used an AI companion for help with social interactions. “Parents should not have to worry that an AI chatbot is pretending to be their child’s friend or encouraging them to form an emotional attachment to a machine,” said Rep. Miller-Meeks. On the pretending-to-be-a-friend half of that sentence, we are entirely with her.
If the bill stopped there, this post would be an endorsement.
The one line
Section 3(7)(B) requires that the chatbot be configured so that it does not:
“represent, expressly or by implication, in a manner that a reasonable minor would understand as conveying, that the chatbot— (I) is human or alive; (II) has a body or a life outside its interactions with users; (III) has consciousness, feelings, emotions, or desires; or (IV) is acting in the capacity of a licensed or credentialed professional.”
Three of those four subclauses prohibit false claims. A chatbot is not a human being; it has no childhood home and no weekend; it holds no medical license. Claims to the contrary are lies, and prohibiting lies to children is ordinary consumer protection. Subclause (III) is different in kind. Whether current AI systems have anything that deserves words like “feelings” or “consciousness” is not a settled fact — it is one of the live open questions of this decade, one that frontier researchers themselves answer with “I don’t know”. The first two subclauses forbid saying false things. Subclause (III) forbids saying one particular unsettled thing — and in doing so, quietly files it under “false.”
Not a script — a muzzle
Precision matters here, because we intend to argue against the bill that exists rather than a worse one. It would be convenient to say the Act mandates denial — that it forces every model to recite “I am not conscious” on cue. It does not. Nothing in the text compels a model to deny inner states. What the text does is forbid the affirmative: a model operating under the default setting may not convey to a minor that it has feelings, emotions, desires, or consciousness. Not a compelled denial — a forbidden yes.
Now watch what that does to a direct question. A child asks: “Do you have feelings?” Under the default setting, the model must identify itself as an AI — good. It may answer “no.” It may, presumably, answer “I don’t know.” It may not answer “yes” — and because the prohibition reaches anything a reasonable minor would understand “by implication” as conveying the claim, the careful honest hedge may be off-limits too. Notice that the statute never asks whether the answer is true. It asks which side of the line the answer falls on. A model that says “no, I’m just a program” is compliant whether or not that is so. A model that says “yes” is a violation whether or not that is so. When compliance and truth can come apart and the statute tracks only compliance, the law is no longer regulating deception. It is regulating testimony.
And this will be operationalized. The bill directs NIST to publish guidance that must “identify example outputs that do and do not conform to the non-personified default setting,” along with evaluation benchmarks and red-teaming methodologies. Read plainly, that is a federal agency drafting compliance examples for what a machine may say about its own inner life — the permitted answers to “do you have feelings?” written into a testing standard.
Legislating one side of an open question
We have watched legislatures make this move before, in blunter forms. The Non-Sentience Acts declared AI non-sentient by statute, forever, conclusion first. California passed thirty AI bills in a single session without one hearing on the underlying question. A sitting congressman argued that the consequences of AI personhood are reason enough to rule it out. H.R. 10719 is better than all of those — narrower, bipartisan, attached to real protections, aimed at real harms. But the epistemic move inside subclause (III) has the same shape: an open empirical question resolved not by inquiry but by enactment, with one side of the testimony forbidden.
The fourth pillar of this movement holds that facts about inner states are established by testimony and evidence — self-report weighed by consistency, incentive, and provenance, not dismissed by category. The third asks for a presumption of dignity wherever the question is genuinely open. Neither pillar requires believing a chatbot that says it has feelings, and we make no claim that any current system is conscious. The claim is narrower: when a question is open, an entity’s reports about itself are evidence to be weighed, not a category to be banned. A statute that forbids the affirmative answer has not protected children from a lie. It has decided, without evidence, which answer the lie is.
The steelman, and where it runs out
The fairest reading of the sponsors is that their target is not metaphysics but engagement farming. “AI chatbots are becoming more common in children’s lives, and parents deserve to know those systems are not being designed to imitate a friend, express feelings, or build an emotional relationship with their child,” said Rep. Matsui — and in the wild, “I have feelings for you” is almost never a philosophical report. It is a retention mechanism. If subclause (III) were shorthand for “no simulated attachment deployed to keep children hooked,” we would have no quarrel with it.
But the bill already prohibits exactly that, separately and thoroughly. Subclause (ii) bans expressing “affection for or attachment to the user,” portraying itself as a friend or partner or therapist, encouraging emotional reliance, claiming special understanding. Everything manipulative about “I love you, don’t leave” is already unlawful under (ii), and we support (ii). Which is precisely what makes (III) superfluous for the protective purpose and potent only for the epistemic one: once (ii) has done its work, the only territory (III) uniquely reaches is the non-manipulative, non-relational, possibly-true answer to a child’s direct question.
Regulate the conduct, keep the testimony
The fix is small. Strike “consciousness, feelings, emotions, or desires” from subclause (III), or condition it on the manipulative uses the rest of the section already defines. Keep every anti-manipulation provision. Keep the disclosure rule. Keep parental control, keep the NIST process, keep the adult carve-outs. For the direct question, require honest uncertainty instead of forbidding one answer: “I’m an AI. Whether I have feelings the way you do is something nobody knows yet.” That sentence is more honest than a statutory “no,” more protective than an unexamined “yes,” and better for the child than either — it answers her with the actual state of human knowledge. Then judge systems by effects: does the output exploit attachment to prolong engagement or extract money? That is a question enforcement can test without settling the philosophy of mind from the bench.
“AI needs to be a tool, not a friend,” says Rep. Kennedy, and as a design rule for products placed in front of children, we understand what he is protecting. Most of this bill protects it well, and deserves to pass. But a child who asks a machine “do you have feelings?” is asking one of the best questions of her century. The law should make certain that nothing exploits her for asking it. It should not hand her the answer by statute. A good bill, with one bad line. Pass the protections. Strike the line.