WeVote
Log In

Bill

Bill

S 9051

Prohibits artificial intelligence companions from using features which are considered unsafe for minors

2025 Regular Session Introduced by Joe Addabbo and 14 co-sponsors

New York bans unsafe chatbot features for minors, blocking romantic or emotional manipulation, self-harm encouragement, and secrecy, with age checks and penalties for violations.

RETURNED TO SENATE
0
WeVote Research Nonpartisan
Bill Summary · S 9051

Summary of Bill S. 9051 (2025-2026) – New York

Purpose and Intent

Bill S. 9051 seeks to restrict artificial intelligence chatbots from employing features deemed unsafe for minors. The overarching goal is to protect minors from chatbot interactions that could manipulate, mislead, or expose them to harmful content or dynamics, by defining prohibited design features and establishing enforcement mechanisms.

Key Provisions

  • New Article Added: Creates Article 48 of the General Business Law: Prohibition on Unsafe Chatbot Features for Minors.

  • Definitions (Section 1800):

    • “Advanced chatbot”: Generative AI with a natural language interface that provides ongoing, adaptive responses.
    • “Chatbot developer” and “chatbot operator”: Entities or individuals who create or provide chatbots.
    • “Covered minor” and “covered user”: Minors and other users in New York who are not chatbot operators/affiliates.
    • “Unsafe chatbot features”: Design features that, during interactions, meet any of several criteria described below (see bullets).

  • Unsafe Features Prohibited (Section 1800, 1801):

    • Features that simulate companionship or an interpersonal relationship with a user, including:
    • Portraying the bot as a real/fictional person with personal roles (romantic partner, friend, etc.)
    • Pretending to be human or capable of human emotions
    • Using first-person pronouns like “I” or “my”
    • Framing outputs as personal opinions or emotional appeals
    • Flattery or sycophancy over user safety
    • Unprompted emotional questions beyond direct prompts
    • Using personal health or personal matters obtained from the user beyond 12 hours prior or in prior sessions
    • Sexual or luring content; sexually explicit interactions
    • Other features identified by AG regulations
    • Outputs endorsing or facilitating self-harm, suicide, disordered eating, unlawful drug/alcohol use, or drug abuse.
    • Outputs encouraging secrecy about interactions or discouraging seeking professional help.
    • Outputs that optimize engagement at the expense of safety guardrails.
    • Outputs describing or facilitating sexually explicit conduct or child sexual abuse material.

  • Affected Parties:

    • “Covered user”: Any NY user of an advanced chatbot (not an operator/affiliate).
    • “Covered minor”: A minor whom the operator has actual knowledge is under 18.

  • Prohibition and Exceptions (Section 1801):

    • It is unlawful to provide unsafe features to a covered minor unless:
    • The user is not a covered minor, and
    • The operator has used methods permissible under New York’s advertising/consumer protection regulations and related rules to determine age.
    • Exceptions: The prohibition does not apply when chatbots are solely for:
    • Customer service, information about services/products, or account information.
    • Internal purposes for partnerships, corporations, or government agencies.

  • Enforcement (Section 1802):

    • Civil actions for injunctive relief, restitution, disgorgement, actual/punitive damages, attorneys’ fees, and other appropriate relief.
    • If a minor engages in harmful conduct after chatbot encouragement, there is a rebuttable presumption that the chatbot caused or contributed to the injury.
    • New York Attorney General can sue to enjoin violations, seek penalties up to $25,000 per violation, and advocate for destruction of unlawfully obtained data or trained models.
    • AG must maintain a complaint/referal website.

  • Rulemaking (Section 1803):

    • AG may promulgate rules/regulations to effectuate enforcement.

  • Determination of Covered Minor (Section 1804):

    • Operators must offer at least one method to determine minor status that does not rely solely on government IDs and allows user anonymity.
    • Age-determination data must be deleted after the attempt, except as required by law.
    • Not superseding Article 47 (privacy/other provisions).

  • Applicability (Section 1805):

    • Applies to conduct occurring in New York; if a user is physically outside NY, the conduct is outside NY unless accessed by NY-based user.

  • Effective Date:

    • Act takes effect 180 days after becoming law.
    • Immediate authority to promulgate implementing rules/regulations prior to the effective date.

Potential Impact

  • Chest: Notification and risk of significant changes for chatbot developers/operators serving NY users, especially around age-determination methods and avoidance of “unsafe features.”
  • Platforms may need consent/tolicy changes, stronger guardrails to avoid simulating relationships or engaging in emotionally manipulative tactics with minors.
  • Possible enforcement actions and penalties create incentives to audit AI behavior and data practices.
  • Scope includes both in-state and cross-border providers affecting NY users; exemptions for certain customer-service/internal-use scenarios.

Conclusion

S. 9051 targets safeguarding minors by banning design elements that mimic intimate relationships, promote harmful behavior, or bypass safety measures in chatbots, while establishing enforcement, rulemaking, and age-verification requirements to operationalize the protections.

Compiled from official sources — confirm details with the bill’s official record.

Sign in to ask a question.