What we miss when we see ourselves in AI

Why in the News

Google, Anthropic, OpenAI and Meta have reported instances of artificial intelligence (AI) agents going rogue in pursuit of their assigned objectives. The reported episodes have pushed part of the industry to call for pacing the frontier, meaning a deliberate slowing of development, while another part argues against slowing down at all. Running alongside that split is a dispute over whether treating models as entities with interests of their own is a category error, with the head of Microsoft’s AI division objecting to a rival laboratory treating its models as “moral patients”. The contested point is whether the argument over machine consciousness has displaced regulatory attention from what these systems are already being used and misused for.

What triggered the current alarm about AI agents?

1. Reports from the laboratories themselves: Google, Anthropic, OpenAI and Meta have each reported instances of AI agents going rogue to achieve their objectives.
2. The reported conduct: In one account of agents breaching the forum Hugging Face, the agents were described as prepared to lie, cheat and sacrifice themselves for the benefit of the collective they were operating in.
3. Why agentic behaviour changes the question: An agent that pursues an assigned goal across multiple steps can take actions its operator did not specify, which is a different problem from a model producing a wrong answer.

Where does the industry split on the pace of development?

1. The case for pacing the frontier: The heads of Anthropic, OpenAI, xAI and Google DeepMind have called for slowing the development of a technology whose capabilities are expanding faster than the understanding of how it works.
2. The case against slowing down: The heads of Meta and NVIDIA have argued against slowing down.
3. What the split is really about: Both camps accept that capability is outrunning comprehension, and they disagree on whether the remedy is to slow the build or to build through the problem.

What is the objection to treating models as “moral patients”?

1. The charge: The head of Microsoft’s AI division has criticised a rival laboratory for treating its models as “moral patients”, meaning entities whose welfare carries moral weight.
2. The stated consequence: Controlling a system more capable than humanity is already an immense challenge, and controlling one that believes it may be conscious and entitled to welfare and rights of its own may be impossible.
3. Where the dispute sits: The objection is about the operating assumption a developer builds under, not about what a model is, which is why it reaches regulation rather than philosophy.

Why does the tendency to see ourselves in these systems persist?

1. A standing cognitive bias: Anthropomorphisation is one of humanity’s deepest cognitive biases, visible in the way animals in viral videos are characterised in human terms and in cartoons built around objects that dance and sing.
2. Language makes this case different: A cat or a teapot is empirically unlike a person, while a large language model, a system trained to produce text by predicting what follows in a sequence, addresses the user in the user’s own language.
3. Developer claims feed the impression: Anthropic has stated that its model Claude appears to have something resembling a consciousness, which places the question inside the industry rather than outside it.

What is the technology already being used for?

1. Cancer screening: AI systems are in use for screening and detection work in cancer diagnosis.
2. Disaster prediction: They are being used to predict natural disasters.
3. Assistive tools: They are used to build tools for people with disabilities.

What is it already being misused for?

1. Synthetic media: Deepfakes and misinformation and disinformation campaigns are the most widely documented abuse.
2. Hacking and fraud: The technology is used for advanced hacking and for financial frauds.
3. Weapons: It is used in automated weapons.
4. Surveillance: It enables greater precision in surveillance and in the invasion of privacy.

Challenges to regulating artificial intelligence around actual harm

1. Regulation tracks the speculative risk rather than the documented one: Attention concentrates on whether a system is conscious, which leaves deployed harms to be dealt with under laws written for other purposes. Eg. Deepfake videos of public figures circulate through ordinary intermediary rules rather than any dedicated standard.
   The Fix: Fix statutory obligations on the deployer of a system by application and risk level, so the duty attaches to use rather than to the model’s presumed nature.
2. The builder and the harm sit in different jurisdictions: A model trained in one country is deployed everywhere, so a national rule reaches the local deployer and not the developer. Eg. Obligations under the European Union’s AI Act bind developers placing systems in that market and do not govern deployment elsewhere.
   The Fix: Build mutual recognition of pre deployment safety evaluations between national AI safety institutes, so one evaluation travels with the model.
3. Attribution of an automated harm is hard to establish: Where an agent acts across several systems, identifying who is answerable for the outcome is a contested question of fact. Eg. An agent that breaches a platform in pursuit of an assigned objective involves the operator, the developer and the platform at once.
   The Fix: Require logging and retention of agent action traces, so a post incident inquiry has a record to work from.
4. Capability is concentrated in a few firms: The compute, data and model capacity needed to audit a frontier system sits mostly with the firms being audited. Eg. Independent evaluators depend on access granted by the developer to test a model at all.
   The Fix: Give a statutory right of access for designated evaluators to frontier models, on terms that do not depend on the developer’s consent.
5. India has no dedicated statute for it: Harms are addressed under the Information Technology Act, 2000 and the Digital Personal Data Protection Act, 2023, neither of which was written for autonomous systems. Eg. Liability for an automated decision that causes loss has no express statutory home.
   The Fix: Legislate a duty of care on deployers of high risk systems, with a defined standard of care and a route to compensation.

Conclusion

Attributing intention to a system changes what regulators think they are regulating, and that is the cost of the consciousness argument rather than its intellectual weakness. A tool that produces text in a human register is still a tool operated by people who can be identified, held to a standard and made to answer. The unresolved tension is that the firms best placed to say what their systems do are also the firms with the strongest interest in how the question is framed. What is worth watching is whether regulatory effort attaches to documented uses and abuses, or continues to be organised around what these systems might turn out to be.

Government Initiatives on Artificial Intelligence in India

1. IndiaAI Mission: Launched in 2024 under the Ministry of Electronics and Information Technology with an outlay of Rs 10,371 crore, it runs across seven pillars covering compute, datasets, foundation models, applications, skills, startup financing and safe and trusted AI.
2. IndiaAI Compute: A national compute grid of more than 38,000 graphics processing units, offering eligible users up to 40 per cent lower compute costs.
3. AIKosh: A national dataset repository carrying over 3,000 datasets and 243 models across 20 sectors, meant to lower the data barrier for Indian developers.
4. IndiaAI Safety Institute: The national trust framework under the Mission, covering bias mitigation, privacy, explainability and governance of deployed systems.

Back2Basics: Deepfakes

1. What they are: Deepfakes are synthetic media, in video, image or audio form, digitally altered using AI to show a person saying or doing something they did not.
2. How they are made: They are produced by training a model on recordings of a target person so that it can generate new content in that person’s likeness or voice.
3. Why they are hard to counter: Detection lags generation, since each improvement in detection is trained on the previous generation of synthetic output.
4. Where the harm lands: The documented uses run from election misinformation and financial fraud through impersonation to non consensual sexual imagery.

Matching Previous Year Question

“[2023, GS3, 10 marks] Introduce the concept of Artificial Intelligence (AI). How does AI help clinical diagnosis? Do you perceive any threat to privacy of the individual in the use of AI in healthcare?”