Light is a Silicon Valley-based startup that uses computational imaging to enable machines to see like humans. We worked with Light to improve the accuracy of their stereo depth and sensor geometric calibration. With support from Softbank’s Vision Fund, Light has become the world’s most advanced imaging platform, working across a range of applications from self-driving vehicles to robotics and smart cities.
Blender, Unreal Engine
Light’s product is a software-defined camera that combines breakthrough optics technology with sophisticated computational software. Apart from creating more vivid and more detailed HDR images, it uses stereo vision and calculates the depth of the image.
Accuracy of depth is an important part of providing high-quality, flawless images. Proper sensor geometric calibration is also critical as individual cameras are not glued to any surface and are prone to micro movements when the device is used.
To improve the sensor geometric calibration.
To improve the quality of stereo depth reconstruction.
To surpass the current state-of-the-art in the field.
The project was divided into 3 parts: online calibration research, lense distortion correction research, and depth pipeline improvements. First, we focused on the theoretical and practical advantages of using online calibration, then we worked on improving the quality of stereo depth reconstruction
The following deliverables were defined and successfully implemented:
A framework for camera calibration testing with respect to synthetic 3D points
A framework for online camera calibration configuration comparison using both synthetic and real-world data
Tools for 3D point cloud storage and visualization
A parameterized online calibration module allowing one to parameterize the bundle adjustment problem for camera intrinsics, camera extrinsics, point position and point depth
An exhaustive evaluation of the impact which online calibration has on stereo depth quality
Conclusive proof that the correct set up of online calibration is capable of improving camera parameters that have been perturbed
Significantly improving the average reprojection error for both synthetic and real-world images