Validating Simulated Camera Sensor Model Ability to Accurately Perceive Texture in Unstructured Outdoor Scenes, 10-R6437

Background 

Uncrewed systems often use camera sensor data to perceive the world and make decisions. Many image processing algorithms rely on accurate perception of texture. Therefore, to make simulated models of camera sensors useful for testing, they need to be validated for their ability to see textures in the same way as the real sensors. 

Approach 

We developed a novel sensor model validation approach and quantitative metrics based on image entropy and structural texture similarity to compare texture information in images produced by real and simulated camera sensors. Our validation method involves creating simulation scenes using a photogrammetry process to capture accurate natural texture information in the sensor’s operating environment. 

To test the efficacy of our sensor model validation procedure, we conducted an experiment with a visual odometry (VO) perception algorithm. VO requires image texture information that is non-repetitive and has high information content (high entropy). Our independent variable was the gamma setting of the camera sensor simulation model. Our goal was to see if there is a correlation between our quantitative metrics and the performance of the VO algorithm on simulated data versus real data. 

Accomplishments 

Experimental results indicated our metrics are useful for comparing real and simulated data collected in a variety of unstructured natural environments, such as where uncrewed and autonomous systems might operate. Results showed good agreement in both entropy and structural texture similarity between the real and simulated data, indicating that our method of creating simulation scenes produces realistic texture information. 

Our validation procedure was used successfully in an example VO perception algorithm application. Results showed that our metrics are a useful guide for tuning the gamma parameter of the simulated camera sensor model. The metrics showed an increasing difference between the real and simulated data when values of the simulated gamma deviated from the value that most closely matches that of the real sensor.