Animage processor, also known as animage processing engine,image processing unit (IPU), orimage signal processor (ISP), is a type ofmedia processor or specializeddigital signal processor (DSP) used forimage processing, indigital cameras or other devices.[1][2]Image processors often employparallel computing even withSIMD orMIMD technologies to increase speed and efficiency.[3] Thedigital image processing engine can perform a range of tasks. To increase the system integration onembedded devices, often it is asystem on a chip withmulti-core processor architecture.
 | This sectiondoes notcite anysources. Please helpimprove this section byadding citations to reliable sources. Unsourced material may be challenged andremoved.(October 2017) (Learn how and when to remove this message) |
Thephotodiodes employed in animage sensor are color-blind by nature: they can only recordshades of grey. To getcolor into the picture, they are covered with different color filters:red,green andblue (RGB) according to the pattern designated by theBayer filter.[4] As each photodiode records the color information for exactly onepixel of the image, without an image processor there would be a green pixel next to each red and blue pixel.
This process, however, is quite complex, and involves a number of different operations. Its quality depends largely on the effectiveness of thealgorithms applied to the raw data coming from the sensor. The mathematically manipulated data becomes the recorded photo file.
As stated above, the image processor evaluates the color andbrightness data of a given pixel, compares them with the data from neighboring pixels, and then uses ademosaicing algorithm to produce an appropriate color and brightness value for the pixel.[5] The image processor also assesses the whole picture to guess at the correct distribution ofcontrast. By adjusting thegamma value (heightening or lowering the contrast range of an image's mid-tones), subtle tonal gradations, such as inhuman skin or the blue of thesky, become much more realistic.
Noise is a phenomenon found in anyelectronic circuitry. Indigital photography its effect is often visible as random spots of obviously wrong color in an otherwise smoothly-colored area. Noise increases with temperature andexposure times. When higherISO settings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lowersignal-to-noise ratio. The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures which, if treated as noise, may lose some of their definition.[6]
As the color and brightness values for each pixel areinterpolated someimage sharpening is applied to even out any fuzziness that has occurred. To preserve the impression ofdepth, clarity and fine details, the image processor must sharpen edges and contours. It therefore mustdetect edges correctly and reproduce them smoothly and without over-sharpening.
Image processor users are using industry standard products, application-specific standard products (ASSP) or evenapplication-specific integrated circuits (ASIC) with trade names: Canon's is calledDIGIC, Nikon'sExpeed, Olympus' TruePic, Panasonic'sVenus Engine and Sony'sBionz. Some are known to be based on theFujitsuMilbeaut, theTexas InstrumentsOMAP,PanasonicMN103,Zoran Coach, Altek Sunny orSanyo image/video processors.
ARM architecture processors with itsNEON SIMDMedia Processing Engines (MPE) are often used inmobile phones.
With the ever-higher pixel count in image sensors, the image processor's speed becomes more critical:photographers don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, image processors must be optimised to cope with more data in the same or even a shorter period of time.
libcamera is a software library that supports using image signal processors for the capture of pictures.