ThePixel Visual Core (PVC) is a series ofARM-basedsystem in package (SiP)image processors designed byGoogle.[1] The PVC is a fully programmableimage,vision andAI multi-core domain-specific architecture (DSA) for mobile devices and in future forIoT.[2]It first appeared in theGoogle Pixel 2 and 2 XL which were introduced on October 19, 2017. It has also appeared in theGoogle Pixel 3 and 3 XL. Starting with the Pixel 4, this chip was replaced with thePixel Neural Core.
Google previously usedQualcomm Snapdragon'sCPU,GPU,IPU, andDSP to handle itsimage processing for theirGoogle Nexus andGoogle Pixel devices. With the increasing importance ofcomputational photography techniques, Google developed the Pixel Visual Core (PVC). Google claims the PVC uses less power than usingCPU andGPU while still being fully programmable, unlike theirtensor processing unit (TPU)application-specific integrated circuit (ASIC).Indeed, classicalmobile devices equip animage signal processor (ISP) that is a fixed functionalityimage processing pipeline. In contrast to this, the PVC has a flexible programmable functionality, not limited only to image processing.
The PVC in theGoogle Pixel 2 and 2 XL is labeled SR3HX X726C502.[3]
The PVC in theGoogle Pixel 3 and 3 XL is labeled SR3HX X739F030.[4]
Thanks to the PVC, the Pixel 2 and Pixel 3 obtained a mobileDxOMark of 98[5] and 101.[6]The latter one was the top-ranked single-lens mobile DxOMark score, tied with the iPhone XR.[7]
A typical image-processing program of the PVC is written inHalide. Currently, it supports just a subset of Halide programming language without floating point operations and with limited memory access patterns.[8]Halide is adomain-specific language that lets the user decouple thealgorithm and thescheduling of its execution.In this way, the developer can write a program that is optimized for the target hardware architecture.[2]
The PVC has two types ofinstruction set architecture (ISA), a virtual and a physical one. First, a high-level language program is compiled into avirtual ISA (vISA), inspired byRISC-V ISA,[2] which abstracts completely from the target hardware generation. Then, the vISA program is compiled into the so-calledphysical ISA (pISA), that is aVLIW ISA. This compilation step takes into account the target hardware parameters (e.g. array of PEs size, STP size, etc...) and specify explicitly memory movements. The decoupling ofvISA andpISA lets the first one to be cross-architecture and generation-independent, whilepISA can be compiled offline or throughJIT compilation.[8]
The Pixel Visual Core is designed to be a scalable multi-core energy-efficient architecture, ranging from even numbers between 2 and 16 core designs.[2] The core of a PVC is theimage processing unit (IPU) a programmable unit tailored for image processing. The Pixel Visual Core architecture was also designed either to be its own chip, like the SR3HX, or as anIP block forSystem on a chip (SOC).[2]
The IPU core has a stencil processor (STP), a line buffer pool (LBP) and aNoC.The STP mainly provides a 2-DSIMD array of processing elements (PEs) able to performstencil computations, a small neighborhood of pixels. Though it seems similar tosystolic array and wavefront computations, the STP has an explicit software controlled data movement. Each PEs features 2x 16-bitarithmetic logic units (ALUs), 1x 16-bitMultiplier–accumulator unit (MAC), 10x 16-bitregisters, and 10x 1-bit predicate registers.[2]
Considering that one of the most energy costly operation is DRAM access, each STP has temporary buffers to increasedata locality, namely LBP. The used LBP is a 2-DFIFO that accommodates different sizes of reading and writing. The LBP uses single-producer multi-consumer behavioral model. Each LBP can have eight logical LB memories and one forDMA input-output operations.[8]Due to the real high complexity of the memory system, the PVC designers state the LBP controller as one of the most challenging components.[2]The NoC used is a ring network on chip used to communicate with only neighbor cores for energy savings and pipelined computational pattern preservation.[2]
The STP has a 2-D array of PEs: for example, a 16x16 array of full PEs and four lanes of simplified PEs called"halo".The STP has a scalar processor, called scalar lane (SCL), that adds control instructions with a small instruction memory.The last component of an STP is a load store unit called sheet generator (SHG), where the sheet is the PVC memory access unit.[2]
The SR3HX PVC features a 64-bit ARMv8aARM Cortex-A53 CPU, 8x image processing unit (IPU) cores, 512 MBLPDDR4, MIPI, PCIe.The IPU cores each have 512arithmetic logic units (ALUs) consisting of 256 processing elements (PEs) arranged as a 16 x 16 2-dimensional array. Those cores execute a custom VLIW ISA.There are two 16-bit ALUs per processing element and they can operate in three distinct ways: independent, joined, and fused.[9] The SR3HX PVC is manufactured as aSiP byTSMC using their28HPMHKMG process.[1] It was designed over 4 years in partnership withIntel. (Codename: Monette Hill)[10] Google claims the SR3HX PVC is 7-16x more energy-efficient than theSnapdragon 835.[1] And that the SR3HX PVC can perform 3 trillion operations per second, HDR+ can run 5x faster and at less than one-tenth the energy than theSnapdragon 835.[11] It supportsHalide for image processing andTensorFlow for machine learning.[11] The current chip runs at 426 MHz and the single IPU is able to perform more than 1 TeraOPS.[2][8]