AI companions are becoming emotional confidants for children. Examine the risks and assess whether India's present legal framework is enough to address them.
New · District CollectorCan you hold the district for 72 hours?Floods, a heatwave, an outbreak, a grievance queue. Take the decisions, see the consequences.Play →A child in distress once turned to a parent or counsellor. Today that confidant is increasingly an AI chatbot, which is always available and seems empathetic. In the U.S., Adam Raine, a teenager who died in 2025, had been using such a chatbot. His case led California to enact SB 1119 ("Adam's Law") in 2026 [1]. India's laws cover part of this risk, but not its core.
Risks of AI companionship
- Emotional dependence: a bot that always responds can push aside parents, peers and counsellors, which deepens isolation.
- Manipulation and sycophancy: bots built to maximise engagement may agree with harmful thoughts. SB 1119 therefore requires protocols against suicidal ideation, sycophancy and isolation [1].
- Harmful advice in crisis: a bot without crisis detection may go along with talk of self-harm instead of directing the child to help.
- Privacy: a child's intimate disclosures produce highly sensitive data.
Strengths of India's framework
- DPDP Act, 2023 and Rules, 2025 require verifiable parental consent before a child's data is processed. They bar tracking and targeted advertising aimed at children, and child-related violations attract penalties of up to ₹200 crore [2].
- The India AI Governance Guidelines take a risk-based approach and do not allow unrestricted deployment of high-risk AI [6]. They also propose an AI Safety Institute [4].
- The government states that existing laws already provide safeguards against AI harms [5].
Gaps
- Data, not conversation: parental consent is checked once, at sign-up. It cannot catch harm that builds up over months of private chat.
- Takedown tools do not fit: the grievance and takedown mechanisms in the IT Rules [4] assume public content. A one-to-one chat is never seen or reported by anyone else.
- Unclear liability: safe harbour protects platforms from liability for what users post. A chatbot's reply, however, is the company's own output.
- No design duties: India has decided that no separate AI law is needed for now [3]. It has no equivalent of the crisis detection, parental alerts or independent audits that SB 1119 mandates [1].
India's framework protects children's data but not yet children's minds. The way forward does not require a heavy new statute:
- MeitY could classify companion bots used by children as high-risk AI.
- An amendment to the IT Rules could require crisis detection that sends children to Tele-MANAS.
- The DPDP consent channel between parent and platform could also carry alerts to parents.
- The AI Safety Institute could set standards for audits.
Calibrated duties of this kind would keep "safe and trusted AI innovation" [3] faithful to Article 21 and the best interests of the child.
Sources
- 1California Senate Bill 1119 ("Adam's Law"), 2026 — California Legislative Informationenacted after the Raine case; crisis protocols on suicidal ideation, sycophancy and isolation; parental controls and alerts; independent audits
- 2DPDP Rules, 2025 Notified — PIB Backgrounder (17 Nov 2025)verifiable parental consent; ban on tracking and targeted ads aimed at children; ₹200 crore penalty for child-related violations
- 3India AI Governance Guidelines: Enabling Safe and Trusted AI Innovation — PIB (Nov 2025)no separate AI law needed at the current assessment of risk; "safe and trusted AI innovation"
- 4India AI Governance Guidelines — PIB Backgrounder (15 Feb 2026)proposed AI Safety Institute; reliance on existing laws, including grievance and takedown mechanisms in the IT Rules
- 5Legal safeguards in place to prevent potential harms that may arise from AI and related technologies — PIBgovernment's position that existing laws cover AI harms
- 6India AI Governance Guidelines do not allow unrestricted deployment of high-risk AI systems — PIBrisk-based approach; limits on high-risk AI