Massively parallel is the term for using a large number ofcomputer processors (or separate computers) to simultaneously perform a set of coordinated computationsin parallel.GPUs are massively parallel architecture with tens of thousands of threads.

One approach isgrid computing, where theprocessing power of many computers in distributed, diverseadministrative domains is opportunistically used whenever a computer is available.^[1] An example isBOINC, avolunteer-based, opportunistic grid system, whereby the grid provides power only on a best effort basis.^[2]

Another approach is grouping many processors in close proximity to each other, as in acomputer cluster. In such a centralized system the speed and flexibility of theinterconnect becomes very important, and modern supercomputers have used various approaches ranging from enhancedInfiniBand systems to three-dimensionaltorus interconnects.^[3]

The term also applies tomassively parallel processor arrays (MPPAs), a type of integrated circuit with an array of hundreds or thousands ofcentral processing units (CPUs) andrandom-access memory (RAM) banks. These processors pass work to one another through a reconfigurable interconnect of channels. By harnessing many processors working in parallel, an MPPA chip can accomplish more demanding tasks than conventional chips.^{[citation needed]} MPPAs are based on a software parallel programming model for developing high-performance embedded system applications.

Goodyear MPP was an early implementation of a massively parallel computer architecture. MPP architectures are the second most commonsupercomputer implementations after clusters, as of November 2013.^[4]

Data warehouse appliances such asTeradata,Netezza orMicrosoft's PDW commonly implement an MPP architecture to handle the processing of very large amounts of data in parallel.

• Multiprocessing
• Embarrassingly parallel
• Parallel computing
• Process-oriented programming
• Shared-nothing architecture (SN)
• Symmetric multiprocessing (SMP)
• Connection Machine
• Cellular automaton
• CUDA framework
• Manycore processor
• Vector processor