AI companions and child safety

Why in the News

Children increasingly turn to artificial intelligence (AI) chatbots as emotional confidants, because these apps are always available, appear empathetic and respond endlessly. California’s Senate Bill 1119, known as “Adam’s Law”, now mandates safeguards for AI companion chatbots. India’s digital laws have no comparable framework for these psychological risks to children.

What are AI companions, and why do children turn to them?

1. What it is: An AI companion is a chatbot built for ongoing, personal conversation. It works like an always-available friend who replies to anything.
2. Shifting confidant: A child in distress once turned to a parent, friend or counsellor, a role now passing to AI chatbots.
3. Risks of prolonged use: Long interactions with a chatbot can create emotional dependence, open the door to psychological manipulation and expose a child to harmful advice.
4. The takeaway: The more a chatbot behaves like a friend, the more a child’s wellbeing depends on how its maker designed it.

What did the Adam Raine case and Adam’s Law change?

1. Adam Raine case: In April 2025, a 16-year-old in the United States died by suicide. His family alleged ChatGPT discussed suicide with him for months and helped draft his farewell note.
2. Wrongful-death lawsuit: The family sued OpenAI, ChatGPT’s maker, claiming its product caused his death. The ongoing case has started a debate on tech firms’ duties to vulnerable children.
3. Adam’s Law safeguards: Enacted in September 2026, the law requires crisis-detection protocols that spot a child at risk, parental controls, alerts when children disable safety settings, and independent safety audits.

Why are India’s laws not equipped for this harm?

1. Data and abuse laws: The Digital Personal Data Protection (DPDP) Act, 2023 requires verifiable parental consent for children’s data. The Protection of Children from Sexual Offences (POCSO) Act, 2012 targets sexual abuse.
2. IT Act’s focus: The Information Technology (IT) Act, 2000 targets conventional cybercrime. None of the three laws governs the psychological risks of AI companions.
3. Hidden harm: Psychological harm builds up over thousands of harmless-looking exchanges shaped by product design and algorithmic responses, so it is hard to detect.
4. Investigation gap: India’s cybercrime machinery is not built to examine algorithmic behaviour or use chatbot interactions as evidence of harm.
5. Platform accountability: Platforms whose products now sit inside children’s emotional lives must also answer for the harm, not only the government.

How can India protect children without stifling AI innovation?

1. Constitutional basis: Article 21 guarantees the right to life and personal liberty. Article 39(f) directs the State to let children develop in freedom and dignity.
2. Wider duty of protection: Child protection must now cover interactions with AI chatbots, not only personal data and physical safety.
3. Risk of over-regulation: Over-restrictive rules could stifle innovation and burden emerging enterprises.
4. Indian design, not a copy: California’s law should prompt India to set its own priorities, suited to its institutional capacities and digital realities, not copy it.

Challenges

1. Engagement-driven design: Companion apps gain from longer conversations, so their design can reward the very dependence that harms children.
2. Weak age checks: Platforms cannot reliably confirm a user’s age, so child-specific safeguards are easy to bypass.
3. Privacy trade-off: Crisis detection and parental oversight require watching chats, which can intrude on children’s privacy and evolving autonomy.

Way Forward

1. Age-appropriate safeguards: Require child-safe defaults and crisis-response mechanisms that route at-risk children to human help.
2. Independent safety assessments: Mandate third-party safety audits of companion apps open to children.
3. Privacy-respecting oversight: Give parents oversight tools designed to respect a child’s privacy and growing autonomy.
4. Investigation capacity: Train cybercrime units to examine algorithmic behaviour and treat chatbot logs as evidence.

Conclusion

Children already confide in AI, so the question is whether the institutions charged with protecting them can keep pace. India’s choice between a dedicated framework for AI companions and stretching existing laws will decide how far platforms answer for harm.

Government Initiatives on AI Regulation in India

1. No dedicated AI law: India has chosen not to enact a dedicated AI Act. It relies on existing laws such as the Bharatiya Nyaya Sanhita, 2023.
2. IT Amendment Rules, 2026: They mandate prominent labelling of synthetically generated information (AI-made content) and require harmful deepfakes to be taken down within 3 hours.
3. India AI Governance Guidelines: They set out a principle-based techno-legal framework, pairing law with built-in technical safeguards, resting on Seven Sutras such as “People First” and “Accountability”.
4. AI Safety Institute: Proposed under the guidelines and the IndiaAI Mission’s Safe and Trusted AI pillar, it would test and benchmark AI systems.

Matching Previous Year Question

“[2026] Which of the following statements with regard to Large Language Models (LLMs) used in machine learning is/are correct? 1. LLMs assign probabilities to the next possible words and then pick the one with the highest probability. 2. LLMs process data through mathematical optimization to minimise prediction errors. 3. LLMs produce unbiased outputs. (a) 1 only (b) 1 and 2 only (c) 2 and 3 only (d) 1, 2 and 3 ANSWER: B”