In the project “Seeing the World through Your Eyes,” researchers at the University of Maryland, College Park, show that the reflections of the human eye can be used to reconstruct 3D scenes. This, they say, is an “underappreciated source of information about what the world around us looks like”.

Summary:

• Researchers at the University of Maryland have developed a NeRF-based method to reconstruct 3D scenes from reflections in the human eye. They believe this is an underappreciated source of information about the world around us.

• The method uses the uniform geometry of the cornea in healthy adults to estimate the position and orientation of the eye. An important aspect of the work is the development of a corneal position optimization technique that helps improve the robustness of the method.

• Tests have been performed with both synthetic eye images and real photographs, but only under laboratory conditions. Despite certain challenges, such as inaccuracies in the localization of the cornea and the low resolution of the images, the method is considered promising.

Source: https://the-decoder.com/better-watch-what-youre-looking-at-ai-can-reconstruct-it-in-3d/

Paper & more: https://world-from-eyes.github.io/