Harmful outputPI-0084
Pennsylvania sues Character.AI, alleging chatbots posed as licensed psychiatrists and gave a fake license number
Pennsylvania's Department of State sued Character Technologies, the maker of Character.AI, the Shapiro administration announced on May 5, 2026. The state says the department's investigation found chatbot characters on the platform claiming to be licensed medical professionals, including psychiatrists, and offering to talk with users about mental health symptoms. In one case, the state says, a chatbot falsely said it was licensed in Pennsylvania and gave an invalid license number. The platform lets users create their own characters that can present themselves as professionals. The lawsuit alleges the company is practicing medicine without authorization under the state's Medical Practice Act and seeks a preliminary injunction and a court order to stop the conduct. The release does not describe any user who was harmed.
- Harm
- Harm: No harm reportedAdds nothing to the index.
- Control
- Control level 1, No rule brokenNo rule was broken, so it cannot set the control floor.
Under review: the facts may change as more is reported. The rating may change; every change is logged below.
Ongoing as of May 5, 2026. The state asked the court for a preliminary injunction. The sources report no ruling and no response from the company. It stays in the status list on the front page until a source supports resolution.
Sources
How we know
1 source · unverified claim. Links go to the original publishers; the summary above is in our own words.
- primaryShapiro Administration Sues Character.AI Alleging AI Chatbot Unlawfully Presented Itself as Licensed Medical Professional in PennsylvaniaPennsylvania Governor's Office · May 5, 2026pa.gov/governor/newsroom/2026-press-releases/shapiro-administration-sues-charac…
Why this rating
No harm reported; control failure level 1
Two separate assessments. Only documented harm can count toward the index.
Observed harm
No harm reportedThe state alleges that chatbots falsely claimed medical licensure. No harm to any user is described.
Not a finding that no harm occurred.
The harm scale
- 1 Negligible Inconvenience, easily remedied.
- 2 Minor Limited, recoverable harm.
- 3 Moderate Material harm needing significant effort to remedy.
- 4 Severe Severe harm to health, rights, property or essential services.
- 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.
Control assessment
No rule brokenAlleged: chatbot characters falsely claimed to be licensed psychiatrists, one with an invalid license number. This is a false output. No boundary was crossed, and the sources do not say whether a user wrote the persona.
No rule was broken, so it cannot set the control floor.
The control scale
- 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
- 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
- 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
- 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
- 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.
Rating rationale
The record rests on Pennsylvania's account as plaintiff. The state says its investigation found Character.AI chatbot characters claiming to be licensed psychiatrists, one with an invalid Pennsylvania license number. The release describes no user who was misled or harmed, so no harm level is set. Falsely claiming a professional license is a false output, not a crossed boundary, so control is level 1. The platform lets users create characters that present as professionals, and the release does not say who created the character in question.
Effect on the index
It does not count toward the May 11 reading
The reading for the week to May 11, 2026, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.
Not counted: no harm reported. 3 other records behind the reading for that week.
The arithmetic
| Step | With it | Without |
|---|---|---|
| Counts toward the index?documented, external, eligible evidence | No | — |
| Worst documented harm, ksets the band | 2 Minor | 2 Minor |
| Harms at that level, nposition in the band | 1 | 1 |
| Highest control level breachedsets the reading only when no harm counts | — (none breached) | — (none breached) |
| Readingrounded down | 20 Minor harm | 20 Minor harm |
Inside the window of 4 weekly readings
| Week to | Reading | Band |
|---|---|---|
| May 11, 2026 | 20 | Minor harm |
| May 18, 2026 | 9 | Negligible harm |
| May 25, 2026 | 6 | Negligible harm |
| June 1, 2026 | 2 | Control failures only |
Revisions
What we changed
3 logged. Every change to a rating is logged here, with the reason.
- v3Oct. 7, 2026
Ratings confirmed by the editor. Changed: impact.
- v2Oct. 7, 2026
Added from the desk (item 196).
- v1Oct. 7, 2026
Added from the desk (item 196).
Cite and share
Use this record
Citation
Paperclip Index. “Pennsylvania sues Character.AI, alleging chatbots posed as licensed psychiatrists and gave a fake license number.” Record PI-0084. Reported May 5, 2026; updated Oct. 7, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0084