The challenge

Context of the challenge

National security organisations undertake sensitive activities but also depend on complex supply chains to acquire and maintain the technology they need to operate. As part of that, security researchers carry out detailed tear-downs examining software, hardware and data components to identify possible vulnerabilities.

The first stage of this process is research. When a security researcher is tasked to examine a new product, particularly in the context of industrial control systems, they need to draw on open-source information at a micro-component level, such as technical specifications, datasheets, schematics and technical forum discussions. It is laborious and takes time that could be better spent on the analysis itself.

This challenge is about changing that. We believe that human-machine teaming offers a real opportunity to reduce the research burden. Specifically, we are looking for a system that can do three things:

• Index both structured and unstructured technical information about a product and its components.
• Generate a clear technical summary of the product and its individual components.
• Allow the researcher to ask natural-language questions about the product and explore the information interactively, adapting their line of investigation as new information emerges.

The gap

Industrial control systems can be highly complex and thus time consuming to index and query related information. Complexities can arise from the following:

• Products vary significantly, meaning there is no one size fits all.
• There can be multiple product versions with varied components and software updates.
• Security researchers rely on their experience, processes and trusted sources, such as information directly from the vendor and trusted online forums.
• A chain of trust is formed from:
  • Physical components such as filters, fuses, processors and memory sensors.
  • Software across a range of forms including source code or binary for multiple different processors and operating systems in the same product.

Example use case

Alicia is an experienced security researcher with a focus on industrial control systems. She has been tasked to assess an industrial additive manufacturing machine. The machine will be used in a manufacturing facility without an internet connection, to build critical, classified, components for national security and defence operations. Any vulnerabilities must be understood and mitigated.

She begins with the vendor manual, supplied in paper copy and PDF.

Using the wiring diagram and schematic she starts to investigate the hardware architecture, including interfaces and components and microprocessors. She sources datasheets online for each component and also finds photos of various tear-downs.

She starts to pull-out all the available code supplied by the vendor.

She consults online forums, some trusted and some are new to her.

Alicia starts to build up a large library of information on the product and its components. She drags and drops each bit of information into her ‘tear down assistant’ tool.

As she builds this library of information, she naturally starts to learn about how the machine works, but she also needs to be able to call back on this vast amount of information efficiently. An intelligent, easy to use search and summary capability is essential.

When she wants to explore the machine’s interfaces, she types a query into the tool and receives a conversational response, backed by a reliable source. She builds on this with follow-up questions, each time receiving a well-grounded answer, citing sources. Where answers are not clear, this is highlighted with alternative theories. As the assistant starts to learn Alicia’s behaviour, it adapts to her needs.

This operates more than just a search tool but is more like a personal assistant who really understands the subject matter, and Alicia.

Project scope

This challenge focuses on building a standalone software tool that can ingest relevant open-source information, compile it into a searchable library, support natural language queries across multi-modal formats and provide conversational intelligent and well-informed answers. We would like to see proposals which don’t just focus on off the shelf Retrieval-Augmented Generation systems.

After the 12-week project, the final deliverable should be a software tool meeting the stated requirements for testing in-house at HMGCC.

Essential requirements:

• The tool must have the ability to understand system architecture of a selected machine. A non-exhaustive list of components to understand are the physical interface interactions, data interfaces and protocols.
• Have an ability to check and validate responses before publishing, to prevent erroneous information and hallucinations.
• Characterise from multimedia inputs, such as including manuals, schematics, datasheets, corporate databases, images, code, handwritten annotations.
• Verify information by listing sources and cross checking against high confidence data such as industry publications, academic research and manufacturer documentation.
• Flag a confidence score and if more source data is required.
• The solution should be capable of operating on a laptop without an internet connection, allowing users to characterise complex systems and identify vulnerabilities in environments with limited or no connectivity.
• Provide an easy to search and intelligent function to query the dataset in a chat-like manner.
• Keep a memory of queries so conversations can be continued over several weeks without repetition of prompts.

Desirable requirements:

• Build a profile of the user and adapt to their needs, for example to present information in preferred formats and even proactively provide information that is frequently requested.
• Ability to translate and index non-English data sources (e.g. datasheets and forum posts).
• Recognise and mitigate cultural biases to ensure a nuanced understanding.
• Ensure the software tool remains up-to-date when offline. Consider in a future iteration how the solution may incorporate a mechanism for periodic updates of the core tool and its indexing/search algorithms.

Constraints:

• The tool must work without an internet connection.

Not required:

• For this challenge, the system does not need to autonomously identify or search for source data (e.g. datasheets, schematics and forum posts). Test data will be provided.

Eligibility

This challenge is open to sole innovators, industry, academic and research organisations of all types and sizes. There is no requirement for security clearances.

Solution providers or direct collaboration from countries listed by the UK government under trade sanctions and/or arms embargoes, are not eligible for HMGCC Co-Creation challenges.

Invitation to present

Successful applicants will be invited to a pitch day, giving them a chance to meet the HMGCC Co-Creation team and pitch the proposal during a 20-minute presentation, followed by questions.

After the pitch day, a final funding decision will be made. For unsuccessful applicants, feedback will be given in a timely manner.

Clarifying questions

Clarifying questions or general requests for assistance can be submitted directly to cocreation@hmgcc.gov.uk before the deadline with the challenge title as the subject. These clarifying questions may be technical, procedural, or commercial in subject, or anything else where assistance is required. Please note that answered questions will be published to facilitate a fair and open competition.

How to apply

Please submit your application on the HMGCC Co-Creation website. Any queries please email Co-Creation@dstl.gov.uk and cocreation@hmgcc.gov.uk.

All information you provide to us as part of your application will be handled in confidence.

Applications must be no more than six pages or six slides in length. HMGCC Co-Creation reserves the right to stop reading after six pages if this limit is breached. The page/slide limit excludes title pages, references, personnel CVs and organisational profiles.