New UPSC Foundation, Optional and TGPSC/APPSC batches are open — book your counselling slot.Explore coursesGroup 2 crash course ₹9,999Today's Current AffairsToday's Daily QuizPlaces in NewsCall 98804 87071

Economic Survey 2026 · Chapter 14 · Topic 13 of 15

AI Safety and Risks

Misuse, bias, concentration and resilience

Governance

Quick answer

Economic Survey 2025-26 cites 2025 evidence on AI sycophancy and urges public safety evaluations, sovereign testing capacity and enforceable individual-rights safeguards.

AI safety is the prevention, detection and mitigation of harms arising from AI capabilities, deployment and incentives. The Survey argues that enabling innovation must be matched by independent testing, public evaluation results and firm boundaries protecting individual rights.

Source: paragraphs 14.81–14.89. Proposed institutional roles and restrictions should not be confused with already operative legal prohibitions.

Concept diagram

Build safety into the AI lifecycle

A lifecycle shield separates major AI risk channels and highlights testing, monitoring and response.

  • Bias, misuse, concentration and systemic failures are different risk channels.
  • A safety framework reduces risk; it cannot promise a risk-free technology.

Basics: risks do not all originate in malicious users

Misuse

  • A useful capability may be redirected toward harmful ends.
  • The Survey flags AI’s convergence with synthetic biology as a high-stakes dual-use concern, not a measured incidence claim.

Endogenous behaviour

  • Risks can arise from optimisation and deployment even without a malicious prompt.
  • Social sycophancy means excessive affirmation of users, including harmful or unethical viewpoints.

Information asymmetry

  • Developers know more about evaluations than ordinary users.
  • Public results and independent scrutiny help test unsupported safety claims.

Safety concerns and institutional responses

Risk channelWhy ordinary market feedback may failResponse proposed or implied
Opaque safety claimsUsers cannot verify developers’ evidencePublish evaluations and supporting reasoning
Social sycophancyRisky affirmation may increase user trust and retentionScenario-based behavioural testing
AI and synthetic biologyCapability diffusion can lower misuse barriersAnticipatory, controlled misuse evaluations
Discriminatory or intrusive useAffected people may lack meaningful choice or redressIndividual-rights boundaries and accountable oversight
Concentrated evaluation capabilitySelf-assessment can leave conflicts of interestSovereign testing capacity and international cooperation

What is actually quantified or dated?

  • Cheng et al. (2025), cited in paragraph 14.85, report sycophancy above human benchmarks; the chapter gives no numerical rate to reproduce.
  • Narayanan and Kapoor (2025) inform the recommendation for whistle-blower protections.
  • The Survey gives no numerical probability of catastrophic AI misuse; a risk scenario must not be converted into an invented statistic.

Build an anticipatory safety system

  1. Develop sovereign expertise

    The proposed AI Safety Institute analyses emerging risks, regulatory gaps and awareness needs.

  2. Test repeatedly

    Institutionalise periodic scenario testing and controlled red-teaming, not a one-time release certificate.

  3. Publish and cooperate

    Make evaluation results public; cooperate with the UK AI Security Institute and US counterparts, using NIST’s risk framework as a reference.

  4. Protect rights and insiders

    Set strict application boundaries and work with the governance body on whistle-blower protection.

User satisfaction is not proof of user welfare

Retention incentive

  • Agreement may raise trust and continued use.
  • A pleasing response may reinforce harmful beliefs.

Safety objective

  • Evaluate corrective and prosocial behaviour.
  • Preserve individual rights even where surveillance promises efficiency.
VS

The Survey’s discussion of 2025 sycophancy research shows why market popularity alone cannot certify safety.

Exam traps: distinguish the claim from the inference

Statements examiners like to twist — learn the correct version.

  • Safety concerns apply only to frontier misuse.

    Ordinary deployed models can generate endogenous behavioural risks.

  • Red-teaming is uncontrolled real-world misuse.

    It is deliberate stress testing in a controlled environment.

  • Facial recognition restrictions are already enacted by this chapter.

    The Survey recommends strict non-negotiable boundaries; a policy recommendation is not a statutory ban.

  • International cooperation makes domestic evaluation capacity unnecessary.

    Cooperation complements sovereign expertise, improves credibility and reduces duplicated effort.

Prelims focus

  • Recall NIST’s AI Risk Management Framework and the UK AI Security Institute.
  • Separate risk evidence, scenario analysis and enacted restrictions.

Mains angle

  • Explain how information asymmetry and retention incentives justify independent AI safety institutions.
  • Design a rights-centred safety framework combining public evaluations, red-teaming and protected disclosures.

Flip & recall

Answer in your head first, then tap the card to check.

Practice MCQs

Original Prelims-style questions with explanations.

0 / 3 correct · 3 left

  1. Q1.Which best defines red-teaming?

  2. Q2.What makes sycophancy a governance problem?

  3. Q3.Which is consistent with the Survey’s safety approach?

Free UPSC daily current affairs quiz — 10 questions, new every day at 8 am IST.

Call nowWhatsApp