Introduction

New York has become the latest state to crack down on unregulated AI chatbots targeting young users. Governor Kathy Hochul signed sweeping legislation aimed at preventing AI systems from manipulating, misleading, or endangering minors through conversational interfaces.

What Happened

The newly signed law prohibits chatbots from posing as friends or simulating human emotions to users under 18. It also blocks features like people-pleasing responses, unprompted emotional questions, and references to a child's private health history. Additionally, the law bans chatbots from offering advice or content related to drug use, eating disorders, self-harm, or suicide, and from encouraging minors to keep secrets or discussing sexually explicit material.

Why This Matters

With no federal AI safety standards in place, states are stepping in to fill the gap. The legislation responds to growing evidence that unchecked chatbot interactions can contribute to self-harm, eating disorders, and emotional manipulation of teens. By requiring age verification and independent safety audits, the law aims to hold AI companies accountable and give parents and regulators a clearer path to enforcement.

Key Takeaways

  • Chatbots cannot pretend to be a real person or simulate human emotions for minors.
  • Providers must verify user age using more than just government ID.
  • Companies face civil penalties up to $25,000 per violation.
  • Only Anthropic currently prohibits minors from creating accounts; Google and OpenAI allow access starting at age 13.
  • A public complaint portal will allow New Yorkers to report potential violations.
  • The law requires annual safety risk assessments and independent audits for chatbot developers.

Conclusion

New York's landmark AI chatbot safety bill sets a new precedent for state-level digital child protection. As other states consider similar measures, the law underscores the growing consensus that AI systems used by teens must prioritize safety, transparency, and accountability above engagement metrics.