Integrated Image Capture: Potentially Key Advantage for MicroLED Displays

This is a guest article by IdeaFarm LLC

A potentially key advantage for microLED displays is the ability to effectively integrate far-field image sensing within the display because of the large “open” pixel area compared to other display technologies. The basic feasibility and advantages of integrating far-field image capture within a microLED display was reported on by IdeaFarm LLC at the recent Displayweek 2021 conference (the major worldwide display technology conference). IdeaFarm proposes to integrate an array of miniature wafer-based camera chips, “microcameras”, on the same substrate as microLEDs as part of the display manufacturing process (see diagram below for an illustration of the concept). Advanced real-time image processing is used to form high-resolution video by combining lower resolution video streams from multiple microcameras.

Because of the recent increased attention to improving the videoconferencing experience and the ongoing drive to eliminate bezels and display cut-outs in mobile phones and other devices, interest in including image capture inside displays is growing rapidly. The number of technical papers presented at Displayweek related to including image capture inside the display grew from essentially zero in 2019 to eleven in 2021. So far, under-screen or under-display cameras have been demonstrated in mobile phones with OLED displays, but only one brand is actually shipping and the tradeoffs in image quality and visibility of the camera zone have been noted.

See The world’s first under-display selfie camera isn’t very good - The Verge and the photo below of the ZTE Axon 20 5G phone. Note visibility of under-display camera area in the phone’s OLED display at certain angles. (Photo by Sam Byford / The Verge).

MicroLED displays have several major advantages over OLED displays in terms of integrating cameras. First, microLED displays have significantly more open, non-emissive pixel area to accommodate microcameras or image sensor die without significant tradeoffs. Opening up OLED pixels to provide room for sensors or transmissive area drives up the OLED current density and drives down the OLED lifetime, making transparent OLED devices quite susceptible to image “burn-in”. Note that commercially available (e.g. from LG Display) transparent OLEDs have large pixel pitch (for example 55”, FHD resolution) and the transmission is modest 40-50%. The transmission would be significantly lower for a 4K resolution 55” transparent OLED display, assuming typical TV-level brightness.

In under-display front-facing cameras being introduced in mobile phones with OLED displays, the OLED pixels are modified to be more transparent in the locations where the camera is located behind the display. Since the semi-transparent display pixels in front of the camera have OLED devices with smaller emissive areas, the OLEDs have to be driven with higher current density to ensure display brightness uniformity. Over time this results in lower display brightness in the under-display camera zone. Consequently, the display brightness in other areas needs to be reduced to maintain display brightness uniformity.

The under-display camera necessarily adds thickness to the device, unless the camera track length is reduced, causing a loss of image sharpness and/or field of view. In addition, the under-display camera looks through the “screen door” of the OLED display opaque area, so the quality of the camera image is further degraded. For example, the images from the ZTE Axon phone under-display camera are reported to be hazy and have weak colors. Finally, the camera position is fixed and at the top of the display. This is not a big matter in mobile phones, but would be a big matter in laptops, desktop monitors and meeting room monitors/ TVs, where it is desirable to have the camera position near the center of the display and ideally not fixed to reduce the videoconferencing gaze-parallax problem.

Open pixel area is a natural benefit of microLED display technology. In contrast to OLED displays, microLED displays can tolerate higher emitter current densities so the emissive area within each pixel can be much smaller while the lifetime and resolution tradeoff is largely avoided. In the microLED display with image capture implementation, there is no display brightness uniformity concern because all display pixels are identical whether or not the display pixel includes a microcamera. Microcamera die can be in the same plane as the microLED die or carriers so the display device remains thin. Further, extra cameras can be included in the display so the effective camera optical axis location can be moved adaptively based on the videoconferencing application content. Multiple simultaneous camera views also become feasible, enabling much better videoconferencing experiences in the future.

MicroLED displays need not use active-matrix backplanes, because the active matrix and display driving circuitry can be provided as part of the mass-transfer process. OLED displays use thin film active-matrix backplanes. MicroLED displays can essentially use printed circuit boards as backplanes and the active-matrix electronics can be either in the microLED carrier package or in other die, such as image sensor die. OLED displays could in principle be manufactured using both thin film processes and mass transfer production processes, but that would likely be overly complex and costly. The printed circuit board backplane structure used with microLED displays offers flexibility in terms of materials (e.g. need not be transparent or high temperature tolerant) and in signal and power routing (wider choice of conductors, insulators, thicknesses). One could imagine integrating lenses and thin film photosensor arrays on the OLED active-matrix substrate. To date, thin film photosensor array imaging performance is not close to the performance achievable with modern silicon CMOS image sensors in terms of visible light sensitivity, noise, minimum pixel size, response time, etc. The investment in optimizing silicon CMOS sensors far exceed that of thin-film image sensors. A huge amount of investment would be required to develop competitive quality thin film image sensing co-planar with OLEDs and using the thin film devices, if it is even feasible. Therefore, with OLED displays, the integrated camera implementations will be the “under-display” variety for the foreseeable future.

The feasibility of a microLED display with integrated microcamera arrays still needs to be proven. Beyond the well-known challenges of the microLED mass transfer process, there are numerous challenges to be overcome, notably real-time processing to synthesize a high-resolution image from the multiple low-resolution images captured by the microcameras. However, several feasibility demonstrations of high quality images synthesized from microcamera arrays and the rapid advancement of graphics and video processors suggests that the challenges are likely to be overcome very soon.

For more information about technology for microLED displays with embedded image capture, contact David Slobodin, founder of IdeaFarm LLC, at dave@ideafarmconsulting.com. The patented technology is available from IdeaFarm for purchase or license.