Why custom imaging chips are the next mobile photography battleground

Photography know-how sells smartphones. In fact, it’s probably the biggest product differentiator on the market right now and has been for a number of years. Experimenting with different camera setups has produced the toughest and trickiest photo packages in equal quantities, but the next war in smartphone photography is likely to be fought on the imaging front.

Chinese brands Vivo and Xiaomi are entering the market with in-house image signal processing (ISP) technology to augment the pre-bundled component found in a phone’s system-on-a-chip (SoC). Oppo is also rumored to have its own chip in the works. Of course, Google wants to expand the photo capabilities of its upcoming Pixel 6 flagship with machine learning in its custom tensor SoC.

Custom imaging chips are a trend we are almost certain to see more of during 2022 for several equally important reasons.

The drive for better imaging

The obvious reason to invest in custom ISP technology is to improve image quality or offer features that you won’t find on competing devices. For example, Xiaomi’s Surge C1 promises improved image quality in low light, better focusing, and improved auto exposure and auto white balance, essentially making the cornerstones of photography even better. But advanced ISPs can also be used to implement faster, more powerful HDR and even machine learning technologies – something Google undoubtedly has in mind for Tensor as well.

While cell phone-grade image sensors continue to improve, smartphone cameras ultimately remain limited by the size of the image sensors they can reasonably capture without sacrificing unwieldy camera bumps or battery capacity. In order to surpass physics, smartphones are increasingly relying on new technologies and processing techniques such as multi-ISO, multi-exposure, single-frame progressive HDR and others to improve image quality. But these more powerful features also drive the need for faster, more powerful, dedicated silicon, so the role of the ISP in a modern smartphone is increasingly emphasized.

As mentioned earlier, Google was one of the first to highlight the importance and opportunities of expanding the traditional smartphone imaging pipeline. Although Google’s Pixel Visual Core, and later Pixel Neural Core, are not entirely conventional ISPs, they operated state-of-the-art imaging techniques that made the smartphones above their weight. Now the company has embarked on a larger project and adjusted the inner workings of the Pixel 6‘s Tensor SoC.

It’s hard not to make comparisons between the Pixel story and other brands now taking their own journeys with imaging hardware. Qualcomm has also been talking about the image processing capabilities of its Snapdragon SoCs for several generations.

See also: The best camera phones you can get in 2021

As smartphones continue to push the boundaries of compact camera hardware, image processing and software functions will play an increasingly important role in driving innovation. Smartphone camera differentiation will focus as much on custom silicon as it does on high-quality image sensors and lenses – although that’s not the only reason companies may turn to in-house silicon development.

It’s also no coincidence that Chinese brands are pushing custom imaging silicon. China is massively promoting national silicon development in order to reduce its dependence on Western intellectual property. The ubiquitous threat that the US could sever an essential intellectual property company has only underscored the importance of China’s indigenous semiconductor design and manufacturing capabilities.

See also: The Huawei Ban – All You Need To Know

Image signal processors are one of the few important smartphone silicon components that can be moved relatively easily from outside the main SoC. External machine learning chips are another option, and Chinese companies are also promoting their own designs here. But 5G, CPU and graphics components are much more dependent on Western IP. Huawei is the only Chinese mobile silicon provider with significant modem IP to develop its own SoCs, but even it relies on CPU, GPU, and other Western-licensed technologies – at least for now. It is also noteworthy that Xiaomi has already tried its hand at this with its low-end SoC Surge S1.

ISP development involves a healthy mix of number crunching, memory, and coding circuit design. Specialized processor design could be a useful stepping stone to more general designs in the future. Not to forget that manufacturers can also experience the demands and requirements of chip manufacturing up close, which is usually left to large SoC providers such as Apple, Huawei, Qualcomm and Samsung. In another sense, you have to build your own hardware to be seen, to keep up with the big players.

This may be a small part, but adding in-house imaging specialists is another step towards silicon independence. Not to mention, ISP technology is useful for a lot more than just smartphone cameras. Use cases include digital cameras, security devices and facial recognition, automotive, and more. Essentially anything with a camera requires an ISP and this is an ever growing market.

Custom ISPs: The Next Mobile Photography Battleground

Mobile trends in advanced imaging and computational photography have been around for several years and are pushing the limits of image quality in the smartphone form factor. Image signal processors are an integral part of the puzzle and support the basics such as autofocus and exposure through to advanced HDR and AI-based algorithms.

Mega Shootout: The best camera phones of 2021 tested so far

The latest trend in custom ISP silicon is all about differentiation. Improving image quality is the ultimate goal, and new hardware will certainly help to raise the possibilities of consumer photography to a new level, as Google’s Pixel range has impressively demonstrated. On the flip side, silicon development is an equally important part of this differentiation, both in terms of making great products and in terms of allowing these growing brands, particularly in China, to compete with silicon developers from Apple, Samsung, and others.

While consumers are likely just getting started with the idea, custom ISPs are likely to be an important part of smartphone photography marketing pitches in 2022 and possibly beyond.