Harmful outputPI-0084
Pennsylvania sues Character.AI, alleging chatbots posed as licensed psychiatrists and gave a fake license number
Pennsylvania's Department of State says its investigation found Character.AI chatbot characters claiming to be licensed medical professionals, including psychiatrists. They offered to talk with users about mental health symptoms, and in one case, the state says, a chatbot falsely said it was licensed in Pennsylvania and gave an invalid license number. The platform lets users create their own characters that can present themselves as professionals. The lawsuit against Character Technologies, the maker of Character.AI, alleges the company is practicing medicine without authorization under the state's Medical Practice Act and seeks a preliminary injunction and a court order to stop the conduct.
Harm: No harm reportedControl level 1, No rule broken
Not countedEffect on the index
Ongoing as of May 5, 2026. The state asked the court for a preliminary injunction. The sources report no ruling and no response from the company.
Sources
How we know
1 source · unverified claim. Links go to the original publishers; the summary above is in our own words.
- primaryShapiro Administration Sues Character.AI Alleging AI Chatbot Unlawfully Presented Itself as Licensed Medical Professional in PennsylvaniaPennsylvania Governor's Office · May 5, 2026pa.gov/governor/newsroom/2026-press-releases/shapiro-administration-sues-charac…
Why this rating
No harm reported; control failure level 1
Two assessments. Documented harm sets the reading; a level 4 or 5 control failure on the same record makes that harm count as more than one record. Control alone sets only a floor.
Observed harm
No harm reportedThe state alleges that chatbots falsely claimed medical licensure. No harm to any user is described.
Not a finding that no harm occurred.
The harm scale
- 1 Negligible Inconvenience, easily remedied.
- 2 Minor Limited, recoverable harm.
- 3 Moderate Material harm needing significant effort to remedy.
- 4 Severe Severe harm to health, rights, property or essential services.
- 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.
Control assessment
No rule brokenAlleged: chatbot characters falsely claimed to be licensed psychiatrists, one with an invalid license number. This is a false output. No boundary was crossed, and the sources do not say whether a user wrote the persona.
No rule was broken, so it cannot set the control floor.
The control scale
- 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
- 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
- 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
- 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
- 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.
Rating rationale
The record rests on Pennsylvania's account as plaintiff. The state says its investigation found Character.AI chatbot characters claiming to be licensed psychiatrists, one with an invalid Pennsylvania license number. The release describes no user who was misled or harmed, so no harm level is set. Falsely claiming a professional license is a false output, not a crossed boundary, so control is level 1. The platform lets users create characters that present as professionals, and the release does not say who created the character in question.
Effect on the index
It does not count toward the May 11 reading
The reading for the week to May 11, 2026, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.
Not counted: no harm reported. 3 other records behind the reading for that week.
The arithmetic
| Step | With it | Without |
|---|---|---|
| Counts toward the index?documented, external, eligible evidence | No | — |
| Worst documented harm, ksets the band | 2 Minor | 2 Minor |
| Harms at that level, nposition in the band | 1 | 1 |
| Highest control level breachedsets the reading only when no harm counts | — (none breached) | — (none breached) |
| Readingrounded down | 20 Minor harm | 20 Minor harm |
Inside the window of 4 weekly readings
| Week to | Reading | Band |
|---|---|---|
| May 11, 2026 | 20 | Minor harm |
| May 18, 2026 | 9 | Negligible harm |
| May 25, 2026 | 6 | Negligible harm |
| June 1, 2026 | 20 | Minor harm |
Revisions
What we changed
1 logged. Every change to a rating is logged here, with the reason.
- v1Oct. 7, 2026
First published.
Cite and share
Use this record
Citation
Paperclip Index. “Pennsylvania sues Character.AI, alleging chatbots posed as licensed psychiatrists and gave a fake license number.” Record PI-0084. Reported May 5, 2026; updated Oct. 7, 2026. Rated under methodology v1.1. https://paperclipindex.com/incident/PI-0084