《安全机器学习算法报告》-70页_2mb

Summary of "Securing Machine Learning Algorithms" by ENISA

Core Content

This report by the European Union Agency for Cybersecurity (ENISA) provides a comprehensive analysis of the cybersecurity challenges associated with machine learning (ML) algorithms. It outlines a taxonomy of ML algorithms, identifies relevant threats and vulnerabilities, and proposes security controls tailored to these specific risks.

The report is structured into three main sections:

1. ML Algorithms Taxonomy – This section categorizes ML algorithms based on their core functionalities, learning paradigms, and data types.
2. ML Threats and Vulnerabilities – A detailed list of threats and sub-threats is identified, mapped to the ML lifecycle, and linked to the algorithm taxonomy.
3. Security Controls – A set of security controls is proposed, mapped to the ML taxonomy and associated vulnerabilities, to enhance the cybersecurity posture of ML systems.

The report emphasizes the need for a combination of traditional and ML-specific security controls to protect against both general and AI/ML-specific threats.

Main Objectives

• To produce a taxonomy of ML techniques and core functionalities to establish a logical link between threats and security controls.
• To identify the threats targeting ML algorithms and the associated vulnerabilities.
• To propose recommendations for future steps to enhance cybersecurity in systems that rely on ML techniques.

Target Audience

• Public and governmental sector: EU institutions, Member States’ regulatory bodies, data protection authorities, military and intelligence agencies, law enforcement, and national cybersecurity authorities.
• Industry: Including SMEs, operators of essential services, and organizations using AI or cybersecurity solutions.
• AI technical community: Developers, data scientists, and cybersecurity experts working on secure AI and ML solutions.
• Cybersecurity community: Professionals seeking to understand threats and controls specific to ML algorithms.
• Academia and research community: Researchers interested in the security of ML algorithms.
• Standardisation bodies: Entities looking to define key aspects for securing ML systems.

Methodology

The report was developed through a three-step process:

1. Literature Review: A systematic review of over 200 documents, including 100 related to security, was conducted to build the taxonomy and identify threats and vulnerabilities.
2. Expert Interviews: Insights from the ENISA Ad-Hoc Working Group on Artificial Intelligence Cybersecurity were incorporated to refine the taxonomy and validate the findings.
3. Mapping and Analysis: Threats and security controls were mapped to the ML algorithm taxonomy and lifecycle to highlight their relevance and impact.

ML Algorithms Taxonomy

The report introduces a non-exhaustive taxonomy of ML algorithms, focusing on the following key dimensions:

• Main Domain: Algorithms are categorized based on the application domain, such as Computer Vision, NLP & Speech Processing, and Classic Data Science.
• Data Types: Algorithms are grouped by the type of data they process, including images, text, time series, and structured data.
• Learning Paradigms: Three primary learning paradigms are identified:
  • Supervised Learning: Uses labeled data for classification and regression.
  • Unsupervised Learning: Analyzes unlabeled data for clustering and dimensionality reduction.
  • Reinforcement Learning: Involves learning through interaction and feedback.

The taxonomy includes 40 commonly used ML algorithms, and algorithms are grouped into families or clusters based on shared principles, using nested boxes for clarity.

Key Threats and Vulnerabilities

The report identifies six high-level threats and seven sub-threats, mapping them to specific stages of the ML lifecycle:

• Evasion: Attackers manipulate inputs to mislead the algorithm, often using adversarial examples.
• Oracle: Attackers probe the model by providing inputs and observing outputs.
• Poisoning: Attackers corrupt training data to influence the model’s behavior.
• Model Inversion: Attackers attempt to reconstruct sensitive data from the model.
• Membership Inference: Attackers determine if a particular data point was used to train the model.
• Data Exfiltration: Attackers extract data from the model or its environment.

These threats are associated with specific vulnerabilities and are mapped to the algorithm taxonomy to better understand their impact.

Security Controls

Security controls are proposed to address the identified threats and vulnerabilities. These include:

• Inclusion of adversarial examples in training datasets.
• Implementation of access controls.
• Regular model testing and validation.
• Data anonymization and encryption.
• Monitoring and logging of model behavior.
• Model obfuscation and watermarking.

The report emphasizes that while these controls are specific to ML, they must be complemented by conventional security standards such as ISO 27001 and NIST Cybersecurity Framework to ensure comprehensive protection.

Conclusion

The report highlights that securing ML algorithms requires a tailored approach, as traditional security controls are insufficient. It recommends further research into benchmarking security controls for ML systems, conducting targeted risk assessments, and raising awareness among stakeholders, especially data scientists, about the risks and security practices. Governments and institutions are encouraged to engage with ML experts to develop innovative security solutions and mitigate emerging threats.

Key Recommendations

• Tailored Security Controls: Develop and implement security controls specifically designed for ML systems.
• Benchmarking: Create benchmarks to evaluate the effectiveness of security controls in ML contexts.
• Risk Assessment: Conduct targeted risk assessments to understand trade-offs between security and performance.
• Awareness and Education: Educate data scientists and ML practitioners on security and privacy by design.
• Collaboration: Foster collaboration between cybersecurity and AI communities to enhance ML security.

References

• The report references a wide range of literature and standards, including:
  • ENISA AI Threat Landscape Report
  • ISO 27001/2
  • NIST Cybersecurity Framework
  • GDPR (General Data Protection Regulation)
  • FGSM (Fast Gradient Sign Method) and other adversarial attack techniques

The report serves as a foundational resource for understanding and securing ML systems, particularly in the context of AI and cybersecurity.