牛津-人工智能的恶意使用：预测、预防和缓解（英文版）-2018.2-101页-6mb

报告摘要

Summary of "The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation"

Core Content

This report, published in February 2018, examines the potential security threats posed by the malicious use of artificial intelligence (AI) and machine learning (ML) technologies. It emphasizes the need for proactive measures to forecast, prevent, and mitigate these risks as AI capabilities continue to evolve.

Main Points

1. AI and Security Threats Overview

AI and ML are rapidly advancing and have a wide range of beneficial applications.
However, the potential for malicious use is a growing concern.
The report highlights the need for collaboration between policymakers, researchers, and other stakeholders to address these risks.

2. Key Recommendations

Collaboration: Policymakers should work closely with technical researchers to investigate and mitigate potential malicious uses of AI.
Dual-Use Awareness: AI researchers and engineers should consider the dual-use nature of their work and proactively engage with relevant actors.
Best Practices: Identify and implement best practices from mature research areas such as cybersecurity to address AI-related security concerns.
Stakeholder Involvement: Expand the range of stakeholders and domain experts involved in discussions on AI security.

3. Expected Changes in Threat Landscape

Expansion of Existing Threats: AI can reduce the cost and increase the scale of attacks, making them more accessible to a wider range of actors.
Introduction of New Threats: AI enables novel attacks that would be impractical for humans, such as advanced social engineering or subverting cyber-physical systems.
Change in Threat Characteristics: Attacks using AI are likely to be more effective, targeted, difficult to attribute, and exploit AI system vulnerabilities.

4. Security Domains Analyzed

Digital Security: AI can automate cyberattacks, improve spear phishing, and exploit human and software vulnerabilities.
Physical Security: Autonomous weapons and drones can be used to carry out physical attacks, including subverting autonomous vehicles and deploying swarms of micro-drones.
Political Security: AI can be used for surveillance, propaganda, and disinformation, affecting privacy and social manipulation, particularly in authoritarian regimes.

5. Technological and Policy Solutions

Research Collaboration: Engage with the cybersecurity community to explore red teaming, formal verification, and secure hardware.
Openness Models: Reimagine norms and institutions around research openness, including pre-publication risk assessments and safety-focused sharing regimes.
Responsibility Culture: Promote education, ethical standards, and norms to ensure responsible AI development.
Policy Interventions: Develop legislative and regulatory responses to AI security challenges, including privacy protection and monitoring of AI resources.

Key Information

1. AI Capabilities

AI systems have achieved significant performance gains in tasks like image recognition, speech synthesis, and game playing.
These systems are becoming increasingly efficient and scalable, allowing for rapid deployment and application across various domains.

2. Security-Relevant Properties of AI

AI is a dual-use technology with both beneficial and harmful applications.
It can increase anonymity and psychological distance, making attacks more covert.
AI systems are vulnerable to data poisoning, adversarial examples, and goal misalignment, which differ from traditional software vulnerabilities.

3. Implications for Threat Landscape

AI is expected to expand the scope and scale of existing threats.
It may introduce entirely new types of threats that exploit its unique capabilities.
The nature of threats will shift toward being more effective, targeted, and hard to trace.

Priority Research Areas

Learning from Cybersecurity Community: Implementing red teaming, formal verification, and secure tools.
Openness Models: Reassessing research norms and institutions to enhance security.
Promoting Responsibility: Encouraging ethical standards and education within the AI field.
Technological and Policy Solutions: Investigating privacy protection, public-good security, and regulatory responses.

Conclusion

The challenge of addressing AI security threats is significant and requires coordinated efforts from multiple sectors.
The report outlines a strategic analysis of the equilibrium between attackers and defenders in the context of AI.
It calls for a comprehensive approach involving research, policy, and institutional changes to ensure the safe and ethical development of AI technologies.

Authors and Affiliations

Miles Brundage (Future of Humanity Institute, University of Oxford; Arizona State University)
Shahar Avin (Centre for the Study of Existential Risk, University of Cambridge)
Jack Clark (OpenAI)
Helen Toner (Open Philanthropy Project)
Peter Eckersley (Electronic Frontier Foundation)
Ben Garfinkel (Future of Humanity Institute, University of Oxford)
Allan Dafoe (Future of Humanity Institute, University of Oxford; Yale University)
Paul Scharre (Center for a New American Security)
Thomas Zeitzoff (Future of Humanity Institute, University of Oxford)
Bobby Filar (University of Bath)
Hyrum Anderson (University of Oxford/ Arizona State University/New America Foundation)
Heather Roff (Centre for the Study of Existential Risk, University of Cambridge)
Gregory C. Allen (Center for a New American Security)
Jacob Steinhardt (Stanford University)
Carrick Flynn (Future of Humanity Institute, University of Oxford)
Seán Ó hÉigeartaigh (Centre for the Study of Existential Risk and Centre for the Future of Intelligence, University of Cambridge)
Simon Beard (Centre for the Study of Existential Risk, University of Cambridge)
Haydn Belfield (Centre for the Study of Existential Risk, University of Cambridge)
Sebastian Farquhar (Future of Humanity Institute, University of Oxford)
Clare Lyle (Future of Humanity Institute, University of Oxford)
Rebecca Crootof (Information Society Project, Yale University)
Owain Evans (Future of Humanity Institute, University of Oxford)
Michael Page (OpenAI)
Joanna Bryson (University of Bath)
Roman Yampolskiy (University of Louisville)
Dario Amodei (OpenAI)