Google has published an 18-page study fully detailing the constructed depth-of-field applied science that makes its single-camera Portrait Manner possible. Google introduced its evolved Portrait Mode feature on the Pixel 2 and Pixel 2 Xl, though neither smartphone model has the dual-camera hardware typically required to produce this upshot.

The in-depth paper shows a caste of openness unusual for the smartphone and photographic camera industries. Smartphones with a single camera produce images where everything is generally in focus. Dual-camera phones paired with a stereo algorithm get around this limitation by matching points in images from both cameras to determine depth within the captured scene. Having caused that depth information, some pixels can be selectively blurred to produce the shallow DOF consequence, Google explained in a web log postal service concluding year.

Achieving this same effect using only a unmarried camera is difficult. Some mobile camera apps endeavour to simulate a shallow DOF by separating an image'southward pixels into two layers, isolating the foreground, so blurring the remaining pixels; this is called semantic segmentation. The lack of depth information, notwithstanding, ways the software doesn't know how much mistiness to apply to any capricious object in the scene. The results tin can often exist lackluster or unrealistic, without the gradual optical mistiness expected of objects receding into the distance.

That's where Google's "accurate defocus" technology comes in. The Pixel 2 smartphones utilise the semantic segmentation method for images taken with the forepart-facing photographic camera, but they as well use a stereo algorithm for images taken with the rear photographic camera... despite in that location only being a single lens. Google provided an overview of how it achieves that on its AI blog in October.

At that place are advantages to Google's applied science versus using a second camera, including reducing the space taken up by the imaging module, reduced power consumption, and helping go along costs downward.

Put just, Google repurposes its dual-pixel auto focus hardware utilized increasingly in mobile cameras for fast AF. Each pixel on the sensor is split into 2 photodiodes; the left- and right-looking (or up- and downwardly-looking) photodiodes essentially establish two perspectives of the scene with a ~1mm stereo baseline. A burst of images are aligned and averaged to reduce noise, and a stereo algorithm computes a depth map from the 2 perspectives. This simulates the data that would be provided by two physical cameras next to each other, enabling Google's software to determine the depth of every signal within the captured scene.

There's a lot more to Google'south approach, including even advantages over traditional optics - for example in its choice to force a larger depth-of-field effectually the focus plane to ensure a abrupt subject, something impossible to attain optically. The study likewise points out that there are advantages to Google's technology versus using a 2nd camera, including reducing the space taken up by the imaging module, reduced ability consumption, and helping keep costs downwards.

Read the full PDF here.

Via: Cornell University Library