The system consists of a digital camera and digital projector. The projector beams a series of black and white pixels at a scene and the camera captures the way the light bounces off objects in the scene. The heart of the system is a computer algorithm that continually monitors the data and changes the patterns in order to gain the needed information.
What happens to the patterns from the time they leave the projector to the time they are picked up by the camera "tells us how the light... interacts with the scene," said Pradeep Sen, an electrical engineering researcher at Stanford University. Each pixel of light coming from the projector might bounce off a surface, refract or hit nothing. "Each of these interactions will modify the ray of light," said Sen.