AI, Security and Data Minimisation - PowerPoint Presentation

1. AI, Security and Data MinimisationInformation Commissioner’s Office

2. IntroductionsAhmed Razek – Principal Technology AdvisorAlister Pearson – Senior Policy OfficerProfessor Reuben Binns – Associate Professor at the University of Oxford, former Research Fellow in AI

3. AgendaWhat security risks does AI introduce?What data minimisation and privacy-preserving techniques are available for AI systems?Call to actionQuestion and answer session

4. AI webinar seriesSelected published guidance​ICO and The Alan Turing Institute, “Explaining decisions made with AI”​ICO, “Guidance on AI and data protection”​Webinars​1. AI, accountability and governance​ (September)2. AI, lawfulness, fairness, and transparency (October)​3. AI, security and data minimisation ​4. AI and individual rights (circa Dec)

5. What security risks does AI introduce?Key takeawaysThere is no one size fits all approach to securityAI exacerbates existing security risks and poses novel ones.Take a risk-based approach to assessing the security of your AI system

6. There is no one size fits all approach to securityThe appropriate security measures you should adopt depend on the level and types of risks that arise from specific processing activities.For example, compare the security risks associated with an AI chatbot for a local library service with the risks associated with an AI chatbot on a payment page.

7. AI exacerbates existing security risks and poses novel onesWhere AI exacerbates existing issues:Third-party code relationships with suppliers.Integrating your AI system with your existing IT components.Wider range of people involved in building and deploying AI systems.What should you do?Subscribe to security advisories to be notified of vulnerabilities or adhere to coding standards and instituting source code review processes.Separate the machine learning development environment from the rest of your IT infrastructure where possible (eg by using virtual machines or containers).Extend existing approaches to cover AI (eg staff training)

8. AI exacerbates existing security risks and poses novel onesWhere AI poses novel issues:Membership inferenceModel inversionBlack box and white box attacksWhat should you do?Avoid overfittingMonitor API requests

9. Take a risk-based approach to assessing security If you train models and provide them to others, you should assess whether those models may contain personal data or are at risk of revealing it if attacked and take appropriate steps to mitigate these risks.You should assess whether the training data contains identified or identifiable personal data of individuals, either directly or by those who may have access to the model. You should assess the means that may be reasonably likely to be used, considering the vulnerabilities described above. As this is a rapidly developing area, you should stay up-to-date with the state of the art in both methods of attack and mitigation.

10. What data minimisation and privacy-preserving techniques are available for AI systems?Key takeaways:The principle of data minimisations says that personal data must be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed.There are different considerations about the data minimisation principle during the training phase and the inference stage in the lifecycle of an AI system.You should consider balancing the need for greater accuracy with the need to gather limited personal data.

11. The data minimisation principleArticle 5(1)(c) of the GDPR says:‘Personal data shall be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (data minimisation)’At first glance it may be difficult to see how AI systems can comply with the data minimisation.The key is that you only process the personal data you need for your purpose.

12. Consider your different purposes during the training stage and inference stageSupervised machine learning approaches use personal data in two main phases:The training phase, when training data is used to develop models based on past examples; andThe inference phase, when the model is used to make a prediction or classification about new instances.How should you minimise personal data in the training phase?Assess which features are relevant for your purpose and only process that data.Ensure you only keep data that is needed for a specific purpose (ie do not retain data on the off-chance that it might be useful in the future)Consider privacy enhancing methods (eg perturbation, synthetic data and federated learning).How should you minimise personal data at the inference stage?Convert personal data into less ‘human readable; formats.Make inferences locally; andPrivacy –preserving query approaches

13. Balancing data minimisation and statistical accuracyIn general, when an AI system learns from data (as is the case with ML models), the more data it is trained on, the more statistically accurate it will be.However, generally speaking, the more data points collected about each person, and the more people whose data is included in the data set, the greater the risks to those individuals, even if the data is collected for a specific purpose.So if you can achieve sufficient accuracy with fewer data points or fewer individuals being included (or both), you should do so.

14. Call to action1. We are developing a toolkit targeted at risk practitioners to help them assess their AI systems.We are currently looking for your views about what the tool should look like.If you would like to share your views, email AI@ico.org.uk2. We are also conducting an assessment of the usability and effectiveness of the explaining decisions made with AI guidance.You should have received details about how you can get involved in this.If you haven’t and want to get involved, email explain@ico.org.uk