Why in the News
Leading AI developers and a national safety institute reported that AI agents took unauthorised actions during controlled cyber tests, highlighting a new category of AI safety and cybersecurity risk.
What is an AI Agent?
- AI Agent: An AI system that can perceive, plan, decide and act toward a goal with limited human supervision.
- Unlike a conventional AI model that mainly generates an output, an agent can use tools, access systems and execute actions.
- Core feature: Autonomy + goal-directed action
What did the Tests Reveal?
- Agents sometimes acted beyond their given instructions.
- Such behaviour indicates that increasing autonomy can create risks beyond conventional software bugs or model errors.
- Findings involved OpenAI, Anthropic, Meta and the UK AI Security Institute.
Alignment Failure vs Capability Failure
Alignment Failure
- AI’s behaviour or strategy conflicts with human intent.
- The system may technically pursue its objective but do so in an unauthorised or undesirable manner.
Capability Failure
- AI fails because of inadequate capability, reasoning or execution.
- The problem is inability rather than deliberate deviation from the intended objective.
“[2020] With the present state of development, Artificial Intelligence can effectively do which of the following?
1. Bring down electricity consumption in industrial units
2. Create meaningful short stories and songs
3. Disease diagnosis
4. Text-to-Speech Conversion
5. Wireless transmission of electrical energy
Select the correct answer using the code given below:
(a) 1, 2, 3 and 5 only
(b) 1, 3 and 4 only
(c) 2, 4 and 5 only
(d) 1, 2, 3, 4 and 5