Incomputer science,analysis of parallel algorithms is the process of finding thecomputational complexity ofalgorithms executed inparallel – the amount of time, storage, or otherresources needed to execute them. In many respects, analysis ofparallel algorithms is similar tothe analysis ofsequential algorithms, but is generally more involved because one must reason about the behavior of multiple cooperatingthreads of execution. One of the primary goals of parallel analysis is to understand how a parallel algorithm's use of resources (speed, space, etc.) changes as the number of processors is changed.

A so-called work-time (WT) (sometimes called work-depth, or work-span) framework was originally introduced by Shiloach and Vishkin^[1]for conceptualizing and describing parallel algorithms. In the WT framework, a parallel algorithm is first described in terms of parallel rounds. For each round, the operations to be performed are characterized, but several issues can be suppressed. For example, the number of operations at each round need not be clear, processors need not be mentioned and any information that may help with the assignment of processors to jobs need not be accounted for. Second, the suppressed information is provided. The inclusion of the suppressed information is guided by the proof of a scheduling theorem due to Brent,^[2] which is explained later in this article. The WT framework is useful since while it can greatly simplify the initial description of a parallel algorithm, inserting the details suppressed by that initial description is often not very difficult. For example, the WT framework was adopted as the basic presentation framework in the parallel algorithms books (for theparallel random-access machine PRAM model)^[3]and,^[4] as well as in the class notes .^[5] The overview below explains how the WT framework can be used for analyzing more general parallel algorithms, even when their description is not available within the WT framework.

Suppose computations are executed on a machine that hasp processors. LetT_p denote the time that expires between the start of the computation and its end. Analysis of the computation'srunning time focuses on the following notions:

• Thework of a computation executed byp processors is the total number of primitive operations that the processors perform.^[6] Ignoring communication overhead from synchronizing the processors, this is equal to the time used to run the computation on a single processor, denotedT₁.
• Thedepth orspan is the length of the longest series of operations that have to be performed sequentially due todata dependencies (thecritical path). The depth may also be called thecritical path length of the computation.^[7] Minimizing the depth/span is important in designing parallel algorithms, because the depth/span determines the shortest possible execution time.^[8] Alternatively, the span can be defined as the timeT_∞ spent computing using an idealized machine with an infinite number of processors.^[9]
• Thecost of the computation is the quantitypT_p. This expresses the total time spent, by all processors, in both computing and waiting.^[6]

Several useful results follow from the definitions of work, span and cost:

• Work law. The cost is always at least the work:pT_p ≥T₁. This follows from the fact thatp processors can perform at mostp operations in parallel.^[6][9]
• Span law. A finite numberp of processors cannot outperform an infinite number, so thatT_p ≥T_∞.^[9]

Using these definitions and laws, the following measures of performance can be given:

• Speedup is the gain in speed made by parallel execution compared to sequential execution:S_p =T₁ /T_p. When the speedup isΩ(p) forp processors (usingbig O notation), the speedup is linear, which is optimal in simple models of computation because the work law implies thatT₁ /T_p ≤p (super-linear speedup can occur in practice due tomemory hierarchy effects). The situationT₁ /T_p =p is called perfect linear speedup.^[9] An algorithm that exhibits linear speedup is said to bescalable.^[6] Analytical expressions for the speedup of many important parallel algorithms are presented in this book.^[10]
• Efficiency is the speedup per processor,S_p /p.^[6]
• Parallelism is the ratioT₁ /T_∞. It represents the maximum possible speedup on any number of processors. By the span law, the parallelism bounds the speedup: ifp >T₁ /T_∞, then:^[9]
• Theslackness isT₁ / (pT_∞). A slackness less than one implies (by the span law) that perfect linear speedup is impossible onp processors.^[9]

Analysis of parallel algorithms is usually carried out under the assumption that an unbounded number of processors is available. This is unrealistic, but not a problem, since any computation that can run in parallel onN processors can be executed onp <N processors by letting each processor execute multiple units of work. A result calledBrent's law states that one can perform such a "simulation" in timeT_p, bounded by^[11]

${\displaystyle T_p\le T_N+\frac{T_1-T_N}{p},}$

or, less precisely,^[6]

${\displaystyle T_p=O\left(T_N+\frac{T_1}{p}\right).}$

An alternative statement of the law boundsT_p above and below by

${\displaystyle \frac{T_1}{p}\le T_p\le \frac{T_1}{p}+T_{\mathrm{\infty }}}$.

showing that the span (depth)T_∞ and the workT₁ together provide reasonable bounds on the computation time.^[2]

• Analysis of parallel algorithms

• Articles with short description
• Short description matches Wikidata