Can we renamea/b todst/src so it's easier to correspond with the usage below?

Sure (although note that the table is actually symmetric in the two arguments).

Inthe FreeType docs forFT_Render_Glyph, it suggests that applying the bitmap should use the OVER operator:

dst = alpha * src + (1 - alpha) * dst ,

I'm not following if this is the same operation in 8-bit, or a different one.

It also suggests applying gamma correction, though I think that would not happen here, but in the application of the final bitmap to its colour on the Agg side.

Actually, here we don't really want to composite the glyph over a pre-existing bitmap, but rather (because the rendering occurs to a temporary buffer where all the glyphs of the string will be drawn) compute the "total" alpha coverage map that will result from all the glyphs. (See the explanation on alpha coverage maps in the link you provided.) (Also, I am indeed ignoring gamma correction here.)

If I draw two glyphs with coverage a1 and a2 in 0-1 scale (and color src) at a given pixel, in that order, over a bitmap (dst), then the bitmap color becomes (by applying the OVER formula twice)

dst = a2 * src + (1 - a2) * (a1 * src + (1 - a1) * dst)    = (1 - (1 - a1) * (1 - a2)) * src + (1 - a1) * (1 - a2) * dst

i.e. it's as if I drew a single glyph with coverageatotal = 1 - (1 - a1) * (1 - a2) = a1 + a2 - a1 * a2 at that position (again, note that this is symmetric in a1 and a2).

Multuplying to 0-255 scale (A1 = 255*a1) yieldsAtotal = A1 + A2 - A1 * A2 / 255, which is the formula I implemented (where the+0x7f term leads to rounding in the division term rather than truncation).

Can we add a comment saying it's alpha coverage merging then? Thealpha_compositing name suggests something different, I think.

Sure, done.