Automatic Generation of Realistic In-Vehicle Image & Video Data for Machine Learning AI Algorithm Development

Challenge

There is a high demand internally to collect large amounts of video and image data, which is currently manually carried out, to be used for AI algorithm development. There is a lack of suitable and sufficient datasets from an automotive environment, both inside and outside the car.

Current Existing Method

Collection of video recording data in vehicle is carried out manually.

The manual collection data has certain restriction and gaps to be addressed, e.g.

• Inability to cover dangerous driving scenario on the road.
• Inability to cover all possible cases and combinations of driving behaviour and activities.
• Lack of flexibility (different camera position, field of view, colour/Infrared (IR) camera types, etc)

Solution Sought

We are therefore seeking a technology or solution that is able to automatically generate realistic image and video data to be used in Computer Vision and Machine Learning Artificial Intelligence (AI) algorithm development.
 

• The proposed software solution should generate realistic image and video data for in-vehicle view of drivers, covering different relevant human behaviour inside the car cabin environment. For example, a driver making a phone call, drinking, eating, smoking, having an object in hand, head and body movements, hand gestures, etc.
• An example of a solution we are seeking is to replace a person’s face in video with someone else’s via deepfake technology. This is to be able to automatically generate multiple images and videos of individuals with different facial features, exhibiting a range of behaviours, and carrying out various actions.
• The types of camera views we are looking at that are applicable to deepfake technology are those from RGB and infrared (greyscale) cameras, with resolutions of 1280x800 and 1600x1300.  The common viewing angle is the front and side (45 degree angle) view of the driver.
• The solution should be highly flexible and configurable, to be able to generate the variety of in-vehicle view scenarios specified above.
• The operating and verification environment involves hardware or software in the loop testing.

There are already known solutions such as synthetic data generation to address certain use cases. Nevertheless, there is still further potential make these generated data more realistic with different techniques.

Ground truth image and video data captured by cameras may be provided to the solution provider for evaluation if required. Potential solution provider should be able to demonstrate a good concept with video created based on in-vehicle video recorded on their own or those downloaded from the Internet. Specific sample videos and ground truth data may be provided by Continental (with NDA) for further evaluation at a later stage.

Geographical Restrictions

Usage of Deepfakes could be restricted at different region due to legal law.

Minimum Required Technology Readiness Level (TRL)

Level 7

We aim for the generated data by AI algorithms to able to achieve equivalent levels of accuracy of AI systems that are trained with real-life images and videos. 

We intend to achieve cost savings with this solution. This approach will reduce labour cost and effort required in collecting large amounts of image and video for development.

The developed technique has also the potential to be applicable not only to this specific use-case, but also to other industries which implement computer vision AI and machine learning.

6 months

Further Details

Further details of the challenge statement may be found at the Briefings Page, in the form of video recordings of the virtual briefing sessions, Q&A transcripts and presentation slides if available.

Instructions on Submitting Proposals

Download the proposal submission form in Microsoft Word format at the 'Attachment' section of the Challenge Statement Listing Page and fill in the details of your project proposal. You can then proceed to the proposal submission page by logging in and clicking the 'Submit Proposal' button on the right side of this page, where you will have to fill in the form fields and attach the filled Microsoft Word project submission form before submitting. Do also upload supporting documents/files that may support your proposal.