The challenge

Context of the challenge

Smart glasses are now commercially available, offering features including integrated cameras, AI voice assistance and heads-up displays.

For security and defence operators, smart glasses could provide a first-person reconnaissance view of a terrain but, for this capability to be used, government would need a robust, secure, and automated software chain able to convert raw sensor streams into 3D geospatial intelligence.

The gap

Human reconnaissance is a specialised task undertaken when drones and satellites can’t be used, for example when scouting concealed targets, operating in adverse weather conditions or where agility is required.

In high stress situations, it is crucial a system is available that is easy to use. This system should be able to take images of the situation being studied, so the data can be interpreted by an analyst.

This challenge really asks whether commercial smart glasses could help provide a lower cost solution to some of the existing specialised systems.

But questions remain:

• Who can access the raw data and under which legal frameworks?
• Can the camera deliver sufficient resolution and stability for accurate mapping?
• Can the sensor suite be leveraged to generated reliable 3D models without additional hardware, while processing in real time?

Example use case

Michael is on a reconnaissance mission in a city. His objective is to gather a detailed visual and acoustic map for operational planning, which could help future colleagues travelling in the city have a better understanding of the area.

Michael wears a commercial pair of smart glasses that continuously record a high-resolution video feed. They tag each frame with head-orientation data using built-in inertial measurements, and capture sound. All data remains encrypted on the device and is processed in real-time.

The data is only downloaded after the mission, eliminating any transmission risk.

When back at base, analyst Julie inspects the raw and the real-time processed data. She can see that the 3D model has already been created, the algorithm on an edge device was able to compensate for rapid head movements, stitching overlapping frames into seamless panoramas.

This data can now be easily accessed by local staff so they can avoid dangerous areas and see up to date maps of the local environment.

Project scope

The focus of this challenge is on processing sensor information from smart glasses. The deliverable should be a proof of concept to at least Technology Readiness Level (TRL) 4.

Essential requirements:

• Assess the data generated from smart glasses and provide a report on where the data may go and who may have access.
• Use camera sensor information from smart glasses to provide high-quality images of an urban environment in daytime.
• Demonstrate a proof of concept using camera information to re-create a 3D map.

Desirable requirements:

• Use camera sensor information from smart glasses to provide high-quality images of an urban environment in low light conditions.
• Demonstrate a proof of concept using audio sensor information to add to reconnaissance information.
• Explore all essential requirements across multiple smart glasses from different manufacturers.
• A stretch goal is to combine audio and visual data to overcome common mapping hurdles, such as reflections (which confuse cameras) and solid walls (which block sight).
• Reports on this work should detail future roadmaps of smart glasses manufacturers, including projecting the types of new sensors and other features that could be incorporated in the next three years.

Constraints:

• We are open to a wide range of smart glasses on the market.
• All smart glasses sensor information gathered must be legally compliant, for example with GDPR.
• The terms and conditions of the use of procured smart glasses must not be breached.

Not required:

• We do not want a redesign of smart glasses.
• We do not want to develop new smart glasses.
• We do not want to change the physical structure of smart glasses.

Eligibility

This challenge is open to sole innovators, industry, academic and research organisations of all types and sizes. There is no requirement for security clearances.

Solution providers or direct collaboration from countries listed by the UK government under trade sanctions and/or arms embargoes, are not eligible for HMGCC Co-Creation challenges.

Invitation to present

Successful applicants will be invited to a pitch day, giving them a chance to meet the HMGCC Co-Creation team and pitch the proposal during a 20-minute presentation, followed by questions.

After the pitch day, a final funding decision will be made. For unsuccessful applicants, feedback will be given in a timely manner.

Clarifying questions

Clarifying questions or general requests for assistance can be submitted directly to  [email protected] before the deadline with the challenge title as the subject. These clarifying questions may be technical, procedural, or commercial in subject, or anything else where assistance is required. Please note that answered questions will be published to facilitate a fair and open competition.

How to apply

Please submit your application on the HMGCC Co-Creation website. Any queries please email [email protected] and [email protected].

All information you provide to us as part of your application will be handled in confidence.