Responsible Artificial Intelligence for Defence

slide1. Responsible Artificial Intelligence for Defence (RAID) Toolkit A Trusted Autonomous Systems
Ethics Uplift Project for Australian Defence industry Version 1.1
Consultation 2023<br>
slide2. Contents<br>
slide3. Foreword Defence industry plays a crucial role in the development and transfer of AI capability to Defence.
Defence and the ADF are responsible to the Australian government and the Australian people to conduct their activities in a lawful and ethical manner. This responsibility includes ensuring appropriate consideration of the risks arising from the design, development, acquisition and use of AI capability by Defence.
The novel nature of AI means that Defence will be required to develop and implement new control measures to ensure AI capabilities remain lawful, ethical, and safe. Control measures implemented in the design phase will be critical to ensuring AI capabilities are capable of performing their functions lawfully, ethically and safely. In short, AI capabilities will need to be legal and ethical by design.
This Toolkit provides an opportunity for Defence industry to enhance their AI literacy in relation to the expectations of Defence when acquiring AI capabilities.<br>
slide4. Why Responsible AI for Defence (RAID)? The Commonwealth Government Blueprint and Action Plan for Critical Technologies* identifies the need for Government to promote and protect new and emerging technology, such as AI, that are fundamental to Australia’s national security. It pledges Government will work to determine where policy gaps exist or may emerge to ensure policy actions remain fit for purpose as new challenges materialise. These strategic framework documents enable enhanced cooperation with Australia’s closest allies on critical technologies. There is a need for any Australian approach to legal and ethical AI to complement, or preferably, enhance our interoperability with partners who share common principles for ethical AI.
Government initiatives and industry developments coincide with the release of the ADF's philosophical doctrine ‘Military Ethics’. As foundational doctrine, it reinforces the centrality of ethical conduct to the ADF’s moral authority. Technology can enable responsible decision making and mitigate ethical risks. It recognises that evolving technologies, including AI, raise ‘deeply challenging ethical dimensions’.^
Contemporary national and international best practice highlights the importance of addressing ethical and legal risk early in the design of AI capabilities. Australia’s 2021 AI Action Plan** aims to progress the implementation of Australia’s AI Ethics Principles. These Principles are designed to ensure AI is safe, secure and reliable. Similarly, our closest allies, including the US, UK and Canada, are implementing plans for responsible military AI.<br>
slide5. Commitment to the law and to ethics Laws give practical effect to ethical principles and create mechanisms for enforceability and accountability.
Ethics refers to moral principles or standards of acceptable behaviour by which any particular person is guided. Ethics compels us to ask, ‘Is it the right thing to do?’.
Defence and the ADF are responsible to the Australian government and the Australian people to conduct their activities in a lawful and ethical manner. This responsibility includes ensuring appropriate consideration of lawful, ethical and safety risks arising from the design, development, acquisition and use of AI capability by Defence and the ADF.
AI is lawful when it is capable of performing its functions in compliance with its user’s legal obligations.
Ethical AI is where the humans designing, developing or using AI, or the AI itself, are guided in their behaviour by moral principles or standards of acceptable behaviour.<br>
slide6. AI risks factors and mitigation The risk of adverse outcomes from unlawful or unethical AI applications must be considered early in the development of AI capabilities, rigorously assessed during Defence procurement and monitored during AI capabilities’ in-service life.
A legal and ethical assurance framework applied to AI capability throughout its lifecycle can mitigate risk at the earliest point, providing maximum efficiency in their design process and reducing the need to adjust their capability for Defence industry. This capitalises upon the ability to adopt the capability at the earliest possible time, maximising Australia’s technological edge.<br>
slide7. Before you start Defence industry plays a crucial role in the development and transfer of AI technology into Defence. AI is identified as one of the Commonwealth Government’s critical technologies in the Sovereign Industrial Capability Priorities (SICP)* and is recognised in the Defence Industrial Capability Plan (the Plan).**
The Plan identifies the need to engage and support Australian small and medium-sized enterprises (SMEs) to develop critical industrial capabilities, such as AI, for supply to Defence.
This Toolkit is intended to enhance Defence industry’s creation of ethical and legal AI – AI that is responsible – by empowering them to design and develop their AI with Defence’s ethical and legal considerations in mind throughout the design and development process. This supports streamlined acquisition, and eventual certification for adoption into service. When you do not need to apply this toolkit
You do not need to assess your product or service if:
you are using an AI system that has already been introduced into service by the ADF, and
you are not customising this AI system in any way or using it in a way other than as intended.

Examples: personal digital assistant, smart phones, smart watches, laptops, QR code reader, satnav system, smart card reader, smoke detector, digital thermometer.

AI systems developed on commercially available software platforms are not exempt. Before you start
In addition to the Responsible AI for Defence Toolkit, there are two documents that will assist in understanding Defence’s overarching approach to AI:

Australian AI Action Plan
Australian Defence Force Concept for Robotic and Autonomous Systems

If you are designing a service-specific capability, also read the Service-specific AI plan, for example:
Navy’s 2040 RAS-AI Strategy
Army’s RAS-AI Strategy, v2.0 Aug 2022
The RAAF’s Plan Jericho<br>
slide8. About the RAID What is it?
A Toolkit that fuses government, industry and international best practice to identify, mitigate and record legal and ethical risk associated with military AI capabilities.

Who should use it?
Anyone designing, developing, creating or a capability for sale to Defence that incorporates AI.

When should I use it?
As early as possible during the design process.
The earlier the process is adopted, the less likely you will have to review, redesign or revise your capability when Defence agrees to an acquisition pathway.<br>
slide9. How does it fit together<br>
slide10. How to conduct a RAID assessment After answering some basic questions about your capability in the RAID Checklist, you will be required to record the risk mitigation action taken in the RAID Risk Register.
If the Checklist identifies particular high risks associated with your capability, it will direct you to complete certain parts of a LEAPP.
The Risk Register and LEAPP are updated as the development of your capability progresses.<br>
slide11. When to engage Defence The Checklist should be completed when you are designing or developing any AI capability – either in conjunction with Defence or that you propose to sell to Defence.  
It applies to any proposal that entails the use of an AI capability by Defence; it is not limited to those proposals involving the use of AI functionality in weapons, weapons systems or other means or methods of warfare.
It should be used as early as possible during the design and development phase, for any AI capability that is intended to be sold to Defence.  It is recognised that this decision will be made at various stages during the design cycle, however, the earlier anticipated Defence governance and assurance considerations are identified, and risks mitigated, the greater the competitive advantage and the less onerous certification requirements will be when introducing an AI capability into Defence service. 
The RAID Checklist and Toolkit are not Defence products; rather they are documents that are intended to identify for you what to prepare for to aid in eventual certification of your AI capability for Defence acquisition.
The content of the Toolkit will in some cases replicate what is found in other Defence acquisition processes but will highlight the specific legal and ethical assurance tasks you are anticipated to need to undertake mitigate the legal and ethical risks associated with your capability.<br>
slide12. The Responsible AI for Defence (RAID) Toolkit The Toolkit comprises three Tools:
RAID Checklist
RAID Risk Register
Legal and Ethical Assurance Program Plan (LEAPP)
The RAID Toolkit comprising templates on guidance material will be posted and maintained on the Trusted Autonomous Systems website.<br>
slide13. RAID Checklist Completion of the RAID Checklist identifies which parts of the LEAPP require completion.
The RAID Checklist is the entry level to the Toolkit and risk identification process.<br>
slide14. RAID Risk Register Record the identified risks and the steps taken to resolve or mitigate them in the project RAID Risk Register. Depending on the stage of the acquisition, this may require revising as the capability design and development progresses and the risk mitigation or resolution process can be more refined.
Create Risk Register (see Resource E. RAID Risk Register) with detail for each project activity: 
Define the activity you are undertaking.
Indicate the LEAPP output and topic the activity is intended to address. 
Estimate the risk to the project objectives if issue is not addressed.
Define specific actions undertaken to support the activity. 
Provide a timeline for the activity.
Define action and activity outcomes. 
Identify the responsible party(ies). 
Provide the status of the activity.<br>
slide15. Legal and Ethical Assurance Program Plan (LEAPP) The LEAPP provides a framework for addressing the specific risk identified by articulating what measurable elements need to be addressed to mitigate or resolve the identified risk; and provides methods for providing that quantifiable output. It comprises three operative Parts:
Part 1. Introductory information
General information about your project
This facilitates an understanding and context for the actions taken in the LEAPP
Part 2. Checklist LEAPP requirements 
General information about your project
This facilitates an understanding and context for the actions taken in the LEAPP
Part 3. LEAPP
Further questions relating to risks identified in the Checklist
Measurable elements analysis
Identified further information required to understand particular legal or ethical risk
Identifies framework used to measure the element
Provides information and references about each risk and its treatment
Space for input by multidisciplinary experts and feedback from stakeholders – requirements for change 
Links to records of the risk mitigation tracked in Risk Register.
Part 4 of the LEAPP suggests risk mitigation methodologies that can be applies to the LEAPP component elements. 

The LEAPP is an iterative document and should be updated as the project passes through the ODCS Gates or reaches specified or significant project milestones.*<br>
slide16. Legal and Ethical Assurance Program Plan (LEAPP) For AI programs where the Checklist risks identified are above a certain threshold, a more comprehensive legal and ethical program plan should be provided. The Legal and Ethical Assurance Program Plan (LEAPP) describes a contractor's plan for assuring that software acquired under the contract meets the Commonwealth’s legal and ethical assurance (LEA) requirements. 
It provides guidance to contractors developing legal and ethical assurance programs for complex Defence AI systems. The LEAPP provides Defence with visibility into the contractor's legal and ethical planning, supports progress and risk assessment and provides input into Defence’s internal planning, including Australia’s weapons review obligation under Article 36 of Additional Protocol 1 to the Geneva Conventions.<br>
slide17. Responsible AI Framework By answering key questions about the AI components, you will be addressing the fundamental facets of responsible AI.
Any high risks identified will then require you to complete a LEAPP, which will provide measurable elements to be addressed for each component with an identified high-risk. This will require updating as your project progresses.
You keep a record the process of risk mitigation in the RAID Risk Register.<br>
slide18. The components of an AI systems being used by Defence RAS-AI systems in Defence are comprised of a number of components, which are the parts of the AI system that are required for it to function within a specific Defence environment.
See Resource 1.A for a description of the components.<br>
slide19. The components of an AI systems being used by Defence A. AI
B. Development inputs
C. HMI
D. AI Use Inputs
E. AI Use Outputs
F. Object of AI Action
G. Use case environment
H. System of control / system integration<br>
slide20. The components of an AI systems being used by Defence Each component forms part of the whole AI system, used within a particular Defence use-case and the Defence system as a whole.

This picture represents those components and how they fit together.

By addressing the entire system, those legal and ethical risks that attach to the use of the AI can be meaningfully addressed during its design, development and use.<br>
slide21. The elements of responsible AI The 12 elements are the measurable parts that ensure that the facets (overarching values and principles) of responsible AI are addressed.
They operationalise the ethical and legal issues identified within the components to be assessed and relate to the constituent qualities of the facets.
They allow the legal and ethical risks to be addressed in a measurable, repeatable and recordable way.

See Resource 1.B for a description of the elements<br>
slide22. Elements of Responsible AI measure the facets of Ethical AI in Defence: 2. Accountable 4. Explainable 9. Controllable 3. Understandable 12. Secure 11. Safe 8. Compliant 7. Predictable 6. Reliable 10. Able to integrate 5. Reviewable 1. Responsible 3. Understandable 3. Understandable<br>
slide23. The elements* of responsible AI 1. Responsible
2. Accountable 
3. Understandable
4. Explainable 
5. Reviewable  
6. Reliable 7. Predictable
8. Compliant 
9. Controllable
10. Integrated
11. Safe
12. Secure *See Resource A.2. for a description of these elements<br>
slide24. Putting them together Components + Elements = LEAPP

Risks identified about each AI component or action/interaction is translated into a measurable element in the LEAPP that requires resolution of mitigation.<br>
slide25. The RAID and other AI Frameworks The RAID is designed to assist industry prepare for the information and risk management requirements of Defence in relation to acquisition of military AI.
It is not intended to displace other risk management requirements; and in some cases can sit alongside other risk management frameworks. For example, see Resources B and C for other Policies, Guides, Frameworks and Processes that may assist in this regard.<br>
slide26. Example RAID Checklist The following slides provide the self-assessment undertaken when you complete a RAID Checklist.
Answering these questions will identify if you are required to conduct a LEAPP, and if so, which parts of the LEAPP require completion.
On completion of the Checklist, complete the RAID Risk Register with the identified risk mitigation requirements; and, if applicable, complete the sections of the LEAPP as directed in the Checklist.<br>
slide27. RAID Checklist – A. AI (1)<br>
slide28. RAID Checklist – A. AI (2)<br>
slide29. RAID Checklist – B. Design inputs<br>
slide30. RAID Checklist – C. HMI<br>
slide31. RAID Checklist – D. AI Use Inputs<br>
slide32. RAID Checklist – E. AI Use Outputs<br>
slide33. RAID Checklist – F. Object of AI Action<br>
slide34. RAID Checklist – G. Use case environment<br>
slide35. RAID Checklist – H. System of control<br>
slide36. Glossary and Defintions AI – Artificial Intelligence; a broad term used to describe a collection of technologies able to solve problems and perform tasks without explicit human guidance.
AI functionality – refers to the computational operations that the AI is designed or expected to undertake.
AI capability – refers to a product that comprises of or includes an element of AI functionality.
Component – a part that makes up an AI system, the system it operates in, and those things that are affected by the AI when used by Defence.
Data sanitisation – the deletion, amendment or cleaning of data to remove unwanted bias or error.
Direct supervision - means having a human in the loop, in the loop for exception, or on the loop, capable of influencing the outcome of the AI action.
Element – a measurable part of the risk mitigation process that ensures that the legal and ethical issues relevant to the life-cycle of the AI capability are addressed LEAPP – Legal and Ethical Assurance Program Plan
Means of warfare – weapon or weapons system; a means includes sub-systems that enable the weapon functionality include AI decision support tools, AI enhances sensor and communications networks.
Method of warfare – the way or manner in which weapons and weapon systems are to be used.
ODSC – One Defence Capability System. 
RAID – Responsible AI for Defence.
RAS-AI – Robotic and Autonomous Systems – Artificial Intelligence.
Source Code – Computer program in its original programming language, human readable, before translation into object code usually by a compiler or an interpreter. It consists of algorithms, computer instructions and may include developer's comments.<br>
slide37. Additional resources A. The components and elements in detail
B. The Policies, Guides and Frameworks
C. Developing AI Standards – domestic and international<br>
slide38. Resources: A. The components and elements in detail<br>
slide39. A.1. The components of responsible AI A. AI
B. AI system
C. Design inputs
D. HMI
E. AI Use Inputs
F. AI Use Outputs
G. Object of AI Action
H. Use case environment
I. System of control<br>
slide40. A.1.A. The components of responsible AI - AI The AI is the computational component of the system and the parts that make up the AI system. This is the software, hardware and platform that the AI operates.
It refers to the software process by which the information is analysed, whether it be via a convolutional neural network, rules-based, reactive, limited memory or theory of mind or self-aware.
It also includes the broader design of the AI capability, which consists of the design approach to its software, hardware and associated platform. That is, the AI is the computational component, but it is attached to a physical capability – whether that be a computer screen that provides the output data for a human to read, or a platform that is physical moved or changed as a result of the AI output.<br>
slide41. A.1.B. The components of responsible AI – Design inputs This component describes the processes of design of the AI capability.
It addresses the systems engineering of the AI capability, and the testing, evaluation, validation and verification processes applied to the AI capability as it is designed.
This component includes an assessment of the input data for the design and training of the AI capability (as compared to AI Use Input which deals with the input data when in use).
It also includes the broader design of the AI capability, which consists of the design approach to its software, hardware and associated platform.<br>
slide42. A.1.C. The components of responsible AI - HMI The Human Machine Interaction (HMI) is the component that describes how the human operators engage with the AI capability.
It describes how they can control the capability and the interface system of the AI capability.
This component addresses the physical interaction, the software controls that are available to the human operators, and the physiological and psychological interface between human and machine.<br>
slide43. A.1.D. The components of responsible AI – AI Use Inputs The AI Use Inputs is the data that is fed into the AI capability for it to undertake its computation process when in operation.
This includes the algorithm itself (as the instructions to the AI) when in use, as well as source data that is analysed by the AI capability to produce its analysis.<br>
slide44. A.1.E. The components of responsible AI – AI Use Outputs The AI Use Outputs are those outcomes that result from the AI computational process.
This is the result of the data analysis that is undertaken by the AI capability.
Whether a recommendation for a decision maker, or a summary of existing data, this output is then used within the system to influence an action that will affect the object of the AI action.<br>
slide45. A.1.F. The components of responsible AI – Object of AI action The Object of AI Action is the thing or process that is affected by the AI. It is the object that the AI is designed to influence as a result of its computational process.
It is not the operator of the AI, but rather the person or thing that the decision or algorithmic process is designed to analyse or assess. It may be a person, for example, in the case of an AI capability that is designed to undertake surveillance, it could be property, or it could be data.<br>
slide46. A.1.G. The components of responsible AI – Use case environment The use case environment is the location in which the AI operates.
The use case environment sits within the system of controls as a whole.
There may be multiple use case environments for on AI capability, however, individual assessments must be made in respect of each use case environment.
It includes an assessment of the military domain in which the capability will be fielded, given different operating environments result in different legal and ethical considerations in the use of an AI capability.<br>
slide47. A.1.H. The components of responsible AI – System of control The system of control describes the broader Defence system within which the AI will operate.
It describes the systems, processes and authorities that are in place across the lifecycle of use and in-service deployment and disposal of an AI capability.
It is reflective of Australia’s ‘system of controls’ which provide a layered approach to the control mechanisms and processes that control how an AI capability will be used.
It also includes analysis of how the AI capability integrates within the system. System integration considers internal and external connectivity, data pathways, and the physical integration of the AI componentry. This analysis is considered closely with the HMI component when it analyses the human-machine interface.<br>
slide48. A.2. The elements of responsible AI 1. Responsible
2. Accountable
3. Understandable
4. Explainable
5. Reviewable
6. Reliable 7. Predictable
8. Compliant
9. Controllable
10. Integrated
11. Safe
12. Secure<br>
slide49. A.2.1. The first element of responsible AI - responsible What is responsibility?

Human beings should exercise appropriate levels of judgment and care; and remain responsible for the development, deployment, use, and outcomes and consequences of AI systems.<br>
slide50. A.2.2. The second element of responsible AI - accountable What is accountability?

People responsible for the different phases of the AI system lifecycle should be identifiable and accountable for the outcomes of the AI systems, and human oversight of AI systems should be enabled.<br>
slide51. A.2.3. The third element of responsible AI - understandable What is understandability?

AI-enabled systems, and their actions, decisions, behaviours, intention and predicated impact, must be appropriately understood by relevant individuals, with mechanisms to enable this understanding made an explicit part of system design.<br>
slide52. A.2.4. The fourth element of responsible AI - explainable What is explainable?

For any given use of the AI, the operation, the output, and the impact are intelligible.<br>
slide53. A.2.5. The fifth element of responsible AI - reviewable What is reviewability?

When AI functionality is exercised the relevant aspects of the exercise of that functionality must be discoverable, recordable; and then capable of being audited to determine if AI functionality operated as intended when it was engaged.<br>
slide54. A.2.6. The sixth element of responsible AI - reliable What is reliability?

AI systems and their algorithms should reliably operate consistently in accordance with their intended purpose and explicit, defined use cases. Reliability is the extent to which a system does or does not fail.<br>
slide55. A.2.7. The seventh element of responsible AI - predictable What is predictability?

The actions and effects of the operation of the AI capability should be anticipated and intended.<br>
slide56. A.2.8. The eighth element of responsible AI - compliant What is compliance?

The activation and operation of the AI capability must be compliant with the applicable law, ethics, the governance framework, and the broader system of control.<br>
slide57. A.2.9. The ninth element of responsible AI - controllable What is controllability?

The AI must operate within a responsible chain of human command and control and be capable of human influence as designed.<br>
slide58. A.2.10. The tenth element of responsible AI - integrated What is ability to integrate?

The ability of the AI to integrate into the AI capability, any system it controls or supports, into the HMI, and into the broader system of control.<br>
slide59. A.2.11. The eleventh element of responsible AI - safe What is safety?

The AI capability should not pose unreasonable safety risks, and should adopt safety measures that are proportionate to the magnitude of potential risks.<br>
slide60. A.2.12. The twelfth element of responsible AI - secure What is security?

The AI capability, including its data, must be physically, cyber and electromagnetically secure.<br>
slide61. Resources: B. The Policies, Guides and Frameworks<br>
slide62. Sample of frameworks, guides and policies reviewed during the creation of the RAID framework: Australia:
Department of Industry, Innovation and Science, (2021). Australia’s Artificial Intelligence Action Plan.; Australia’s Artificial Intelligence Ethics Plan, and Australia’s Data Strategy;
Defence Science and Technology Group, A Method for Ethical AI in Defence,
CSIRO, Artificial Intelligence.
Department of Defence, , ADF Concept for Robotics and Autonomous Systems.;
Australian Army, Australian Army RAS-AI Strategy, V2.0.
Royal Australian Navy, Warfare Innovation Navy. RAS-AI Strategy 2040
UK: ‘Ambitious, Safe, Responsible: Our approach to the delivery of AI-enabled capability in Defence’, Ministry of Defence,; UK Ministry of Defence; UK Ministry of Defence Defence Artificial Intelligence Strategy.
US: AI Principles: Recommendations on the Ethical Use of Artificial Intelligence by the Department of Defense and Department of Defense: Responsible Artificial Intelligence Strategy and implementation pathway, Defense Innovation Board; National Institute of Standards and Technology AI Risk Management Framework. U.S. Department of Commerce; Government of the United States, Responsible AI Guidelines.
OECD: The OECD AI Principles.
NATO: NATO Artificial Intelligence Strategy.
Various state submissions to the Convention on Certain Conventional Weapons Group of Government Experts on emerging technologies in the area of lethal autonomous weapons systems, including:
Finland, France, Germany, the Netherlands, Norway, Spain, Sweden, Australia, Canada, Japan, the Republic of Korea, the United Kingdom, the United States, Argentina, Ecuador, Costa Rica, Nigeria, Panama, the Philippines, Sierra Leone, the Russian Federation, Uruguay, the People’s Republic of China.
Industry:
Rolls Royce, The Aletheia Framework
PwC, ‘PwC’s Responsible AI Toolkit in PwC’
Principles of Trust and Transparency.<br>
slide63. Resources: C. Developing AI Standards (domestic and international)<br>
slide64. D1. Domestic AI Standards IEC/ISO/JTC1 family include subcommittees (SC) for data sharing and use include:
SC 27 - Information Security, Cybersecurity and Privacy Protection
SC 32 - Data Management and Interchange
Within SC 32, Working Group 6 (WG6) on Data Usage
SC 38 - Cloud Computing and Distributed Platforms
SC 40 - IT Service Management and IT Governance
SC 41 – Internet of Things and Digital Twin
SC 42 - Artificial Intelligence<br>
slide65. D2. International AI Standards ISO/IEC 23894 on Artificial Intelligence and Risk Management
ISO/IEC 42001 on Artificial Intelligence — Management System
ISO/IEC 38507 on Governance implications of the use of artificial intelligence by organizations
IEEE P2863 on Recommended Practice for Organizational Governance of Artificial Intelligence 
IEEE 7000-2021 on Model Process for Addressing Ethical Concerns During System Design
IEEE 7010-2020 on Recommended Practice for Assessing the Impact of Autonomous and Intelligent Systems on Human Well-Being Draft NISTIR 8332 on Trust and Artificial Intelligence
NIST Special Publication 1270 on A proposal<br>