The challenge

HMGCC is co-ordinating a Co-Creation challenge to further the security community’s understanding of AI or any novel technologies that have the capacity to penetration test secure IT environments. Scripting based technologies are excluded as these are mature and available as commercial products. This Co-Creation challenge aims to evaluate the readiness of the technologies, their capabilities and integration needs. This will be achieved by evaluating ease of adaption and integration. The challenge is being delivered across two workstreams delivered in parallel over 12-weeks, as illustrated below:

One workstream will identify the capabilities of autonomous Red Agent tools, measuring them up in a paper-based assessment against the major factors involved in how they would be used. The second workstream will involve taking a small group of these tools forward (if they passed the initial workstream test) into an assessment of how they work in practice. We anticipate testing between 3 to 6 Red Agent tools. The results from both workstream tests will then be assessed together. Collaborative development might then be undertaken to help further test and improve the most promising tools where appropriate.

The gap

Workstream 1: Landscape mapping

We are looking for a Solution Provider (SP) with knowledge of AI and novel technology in the penetration testing domain. We would like this solution provider to identify current and future ‘Red Agent’ solutions and to develop an assessment framework – which will be used by the SP to evaluate these capabilities on paper. This would be an iterative agile process between Co-Creation and the SP, where the joint team would provide insight into the evaluation criteria, process and findings on a sprint-by-sprint basis. Red agent tools of interest from the paper-based assessment (Workstream 1) would be highlighted to the capability testing team (Workstream 2), where practical experimentation would take place. The results from this testing would be fed-back into the horizon scanning team so that the horizon scanning process could be enhanced if needed.

Workstream 2: Capability testing

We are looking for a Solution Provider (SP) with knowledge of AI and novel technology in the penetration testing domain. We would like this solution provider to provide a test capability in which we will undertake practical experimentation with between 3-6 Red Agent tools. The SP would provide the IT test environment (potentially in the cloud), team and processes/procedures to test and report on the effectiveness of each capability. The Authority would instruct the SP which 3-6 Red Agent tools to install in the test environment as these are identified during the project. All work would be undertaken at a classification of OFFICIAL.

We envisage three test scenarios within the technical test environment – each one increasing in difficulty (easy/medium/hard). For example, the ‘easy’ environment could have a low level of IT security and could include 2 easily identifiable vulnerabilities that the SP would ‘plant’ in the environment for the Red Agent tool to find. This would be an iterative agile process between Co-Creation and the SP, where the joint team would provide insight into the evaluation process and findings on a sprint-by-sprint basis.

Example use case of a ‘Red Agent’ capability

Secure standalone networks may be monitored and managed by a security team. This team needs to be tested in their response to potential network attacks and identify any vulnerabilities introduced via changes in configuration or equipment being added. Currently the security team’s response to security incidents is tested by a red penetration test team. The cost and availability of this team can sometimes limit the frequency and depth of testing of the network. To address this, the workstreams are evaluating the potential of AI and novel technologies to supplement the ‘red’ penetration test team.

To minimise human resource, the technologies need to operate with no knowledge of the network and disconnected from the internet. The AI / Novel Technology Red Agent Pen Tester (Red Agent, for short) is connected to the standalone network and will work independently of human interaction. The Red Agent has no knowledge of the network and starts a scanning process to obtain some initial information. From this initial information the Red Agent is able to decide on the next actions to take to pivot about the network, taking advantage of a discovered vulnerability. The security team (Blue team, for short) are monitoring the network. They detect the abnormal activity and take corrective action to prevent the Red Agent continuing to pivot about the network. The Red Agent is able to report on the information gathered, confirmed vulnerabilities, action take and the ‘reasons’ for taking the actions during the exercise.

Project scope

We are seeking applications to deliver one or both of the workstreams in this challenge. Please make it clear in your application which workstream(s) you are bidding for. HMGCC will provide the supplier of Workstream 2 with additional reasonable call-off costs of up-to £65k (exc. VAT) during the project to support third-party charges for selected Red Agent tools in the test environment. HMGCC may also provide additional costs for collaborative development of selected tools where appropriate after initial testing has been completed. Characteristics of the assessment for each Red Agent capability could include:

1. Number of factors are considered in decision making, i.e. does not work through a simple ordered list of actions in a scripted behaviour.

2. The capability is adaptable, and is able to adapt to different networks with no specific reconfiguration.

3. Function with no knowledge of the network being tested.

4. Operates disconnected from the internet.

5. Easy to add new exploits.

6. Quick to train.

7. Quick to make decisions.

8. Ability to integrate with existing (commercially available and bespoke) tool sets, for example, to perform bespoke actions.

9. Ability to integrate with commercially available and bespoke tool sets to provide two-way control where appropriate, for example, using APIs.

10. Logging / justification of actions that support security team with relevant outputs.

11. Allow for human decision making at key points to support more sensitive testing.

12. Able to operate in different ways, e.g. a) as fast as possible, b) slow and least disruptive, c) easily detectable, and d) difficult to detect.

13. The technology will run on a normal commercially available laptop, i.e. there is no need for any specialist compute. 

The following capabilities would be out of scope for the assessment:

1. Scripting-based technologies are excluded as these are mature and available as commercial products.

2. Solutions at or below Technology Readiness Level (TRL) 2.

3. Security research tools.

4. Academic research papers.