<row>

<entry><literal><function>pg_jit_available()</function></literal></entry>

<entry><type>boolean</type></entry>

<entry>is <acronym>JIT</acronym> available in this session (see <xref

linkend="jit"/>)? Returns <literal>false</literal> if <xref

<entry>is <acronym>JIT</acronym>compilationavailable in this session

(see <xreflinkend="jit"/>)? Returns <literal>false</literal> if <xref

linkend="guc-jit"/> is set to false.</entry>

</row>

</para>

<sect1 id="jit-reason">

<title>What is <acronym>JIT</acronym>?</title>

<title>What is <acronym>JIT</acronym> compilation?</title>

<para>

Just-in-time compilation (<acronym>JIT</acronym>) is the process of turning

<para>

<productname>PostgreSQL</productname> has builtin support to perform

<acronym>JIT</acronym> using <ulink

<acronym>JIT</acronym>compilationusing <ulink

url="https://llvm.org/"><productname>LLVM</productname></ulink> when

<productname>PostgreSQL</productname> was built with

<literal>--with-llvm</literal> (see <xref linkend="configure-with-llvm"/>).

<title>When to <acronym>JIT</acronym>?</title>

<para>

<acronym>JIT</acronym> is beneficial primarily for long-running CPU bound

queries. Frequently these will be analytical queries. For short queries

the overhead of performing <acronym>JIT</acronym>will often be higher than

the time it can save.

<acronym>JIT</acronym>compilationis beneficial primarily for long-running

CPU boundqueries. Frequently these will be analytical queries. For short

queriestheaddedoverhead of performing <acronym>JIT</acronym>compilation

will often be higher thanthe time it can save.

</para>

<para>

To determine whether <acronym>JIT</acronym> is used, the total cost of a

query (see <xref linkend="planner-stats-details"/> and <xref

To determine whether <acronym>JIT</acronym>compilationis used, the total

cost of aquery (see <xref linkend="planner-stats-details"/> and <xref

linkend="runtime-config-query-constants"/>) is used.

</para>

<para>

If the planner, based on the above criterion, decided that

<acronym>JIT</acronym> is beneficial, two further decisions are

<acronym>JIT</acronym>compilationis beneficial, two further decisions are

made. Firstly, if the query is more costly than the <xref

linkend="guc-jit-optimize-above-cost"/>, GUC expensive optimizations are

linkend="guc-jit-optimize-above-cost"/> GUC, expensive optimizations are

used to improve the generated code. Secondly, if the query is more costly

than the <xref linkend="guc-jit-inline-above-cost"/> GUC, short functions

and operators used in the query will be inlined. Both of these operations

└─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

</programlisting>

As visible here, <acronym>JIT</acronym> was used, but inlining and

optimization were not. If <xref linkend="guc-jit-optimize-above-cost"/>,

<xref linkend="guc-jit-inline-above-cost"/> were lowered, just like <xref

expensive optimization were not. If <xref

linkend="guc-jit-optimize-above-cost"/>, <xref

linkend="guc-jit-inline-above-cost"/> were lowered, just like <xref

linkend="guc-jit-above-cost"/>, that would change.

</para>

</sect1>

<title>Configuration</title>

<para>

<xref linkend="guc-jit"/> determines whether <acronym>JIT</acronym> is

enabled or disabled.

<xref linkend="guc-jit"/> determines whether <acronym>JIT</acronym>

compilation isenabled or disabled.

</para>

<para>

That this is done at query execution time, possibly even only in cases

the relevant task is done a number of times, makes it JIT, rather than

ahead-of-time (AOT). Given the way JIT compilation is used in

postgres, the lines between interpretation, AOT and JIT are somewhat

PostgreSQL, the lines between interpretation, AOT and JIT are somewhat

blurry.

Note that the interpreted program turned into a native program does

not necessarily have to be a program in the classical sense. E.g. it

is highly beneficial JIT compile tuple deforming into a native

is highly beneficialtoJIT compile tuple deforming into a native

function just handling a specific type of table, despite tuple

deforming not commonly being understood as a "program".

Why JIT?

========

Parts ofpostgres are commonly bottlenecked by comparatively small

Parts ofPostgreSQL are commonly bottlenecked by comparatively small

pieces of CPU intensive code. In a number of cases that is because the

relevant code has to be very generic (e.g. handling arbitrary SQL

level expressions, over arbitrary tables, with arbitrary extensions

How to JIT

==========

Postgres, by default, uses LLVM to perform JIT. LLVM was chosen

PostgreSQL, by default, uses LLVM to perform JIT. LLVM was chosen

because it is developed by several large corporations and therefore

unlikely to be discontinued, because it has a license compatible with

PostgreSQL, and because itsLLVMIR can be generated from C

using the clangcompiler.

PostgreSQL, and because its IR can be generated from C using the Clang

compiler.

Shared Library Separation

evaluate JIT compilation that does not use LLVM, by changing out the

shared library used to provide JIT compilation.

To achieve this code,e.g. expression evaluation, intending to perform

JIT,callsa LLVM independent wrapper located in jit.c to do so. If

theshared library providing JIT support can be loaded (i.e.postgres

wascompiled with LLVM support and the shared library is installed),

the taskof JIT compiling an expression gets handedof to shared

library. Thisobviously requires that the function in jit.c is allowed

to fail in caseno JIT provider can be loaded.

To achieve this, code intending to perform JIT (e.g. expression evaluation)

callsan LLVM independent wrapper located in jit.c to do so. If the

shared library providing JIT support can be loaded (i.e.PostgreSQL was

compiled with LLVM support and the shared library is installed), the task

of JIT compiling an expression gets handedoff totheshared library. This

obviously requires that the function in jit.c is allowed to fail in case

no JIT provider can be loaded.

Which shared library is loaded is determined by the jit_provider GUC,

defaulting to "llvmjit".

Cloistering code performing JIT into a shared library unfortunately

also means that code doing JIT compilation for various parts of code

has to be located separately from the code doing so without

JIT. E.g. theJITed version of execExprInterp.c is located in

jit/llvm/rather than executor/.

JIT. E.g. theJIT version of execExprInterp.c is located in jit/llvm/

rather than executor/.

JIT Context

Emitting individual functions separately is more expensive than

emitting several functions at once, and emitting them together can

provide additional optimization opportunities. To facilitate that the

LLVM provider separatesfunction definition fromemitting them in an

executableway.

provide additional optimization opportunities. To facilitate that, the

LLVM provider separatesdefining functions fromoptimizing and

emitting functions in anexecutablemanner.

Creating functions into the current mutable module (a module

essentially is LLVM's equivalent of a translation unit in C) is done

Error Handling

--------------

There are two aspectsto error handling. Firstly, generated (LLVM IR)

There are two aspectsof error handling. Firstly, generated (LLVM IR)

and emitted functions (mmap()ed segments) need to be cleaned up both

after a successful query execution and after an error. This is done by

registering each created JITContext with the current resource owner,

LLVM itself runs out of memory. LLVM by default does *not* use any C++

exceptions. Its allocations are primarily funneled through the

standard "new" handlers, and some direct use of malloc() and

mmap(). For the former a 'new handler' exists

http://en.cppreference.com/w/cpp/memory/new/set_new_handler for the

latter LLVM providescallback that get called upon failure

(unfortunately mmap() failures are treated as fatal rather than OOM

errors).What we've, for now,chosen to do, is tohave two functions

that LLVM using codemust use:

mmap(). For the former a 'new handler' exists:

http://en.cppreference.com/w/cpp/memory/new/set_new_handler

For thelatter LLVM providescallbacks that get called upon failure

(unfortunately mmap() failures are treated as fatal rather than OOM errors).

What we'vechosen to do for now ishave two functions that LLVM using code

must use:

extern void llvm_enter_fatal_on_oom(void);

extern void llvm_leave_fatal_on_oom(void);

before interacting with LLVM code.

Using a relatively small enter/leave protected section of code, rather

than setting up these handlers globally, avoids negative interactions

with extensions that might use C++like e.g. postgis. As LLVM code

with extensions that might use C++such as PostGIS. As LLVM code

generation should never execute arbitrary code, just setting these

handlers temporarily ought to suffice.

Type Synchronization

--------------------

To able to generate codeperforming tasks that are donein "interpreted"

postgres, it obviously is required that code generation knows about at

least a fewpostgres types. While it is possible to inform LLVM about

Tobeable to generate codethat can perform tasks doneby "interpreted"

PostgreSQL, it obviously is required that code generation knows about at

least a fewPostgreSQL types. While it is possible to inform LLVM about

type definitions by recreating them manually in C code, that is failure

prone and labor intensive.

Instead there is one small file (llvmjit_types.c) which references each of

the types required for JITing. That file is translated to bitcode at

compile time, and loaded when LLVM is initialized in a backend.

That works very well to synchronize the type definition, unfortunately

That works very well to synchronize the type definition,butunfortunately

it does *not* synchronize offsets as the IR level representation doesn't

know field names. Instead required offsets are maintained as defines in

the original struct definition. E.g.

know field names. Instead, required offsets are maintained as defines in

the original struct definition, like so:

#define FIELDNO_TUPLETABLESLOT_NVALID 9

int tts_nvalid; /* # of valid values in tts_values */

while that still needs to be defined, it's only required for a

While that still needs to be defined, it's only required for a

relatively small number of fields, and it's bunched together with the

struct definition, so it's easily kept synchronized.

--------

One big advantage of JITing expressions is that it can significantly

reduce the overhead ofpostgres's extensible function/operator

mechanism, by inlining the body of called functions /operators.

reduce the overhead ofPostgreSQL's extensible function/operator

mechanism, by inlining the body of called functions/operators.

It obviously is undesirable to maintain a second implementation of

commonly used functions, just for inlining purposes. Instead we take

advantage of the fact that theclang compiler can emit LLVM IR.

advantage of the fact that theClang compiler can emit LLVM IR.

The ability to do so allows us to get the LLVM IR for all operators

(e.g. int8eq, float8pl etc), without maintaining two copies. These

Currently it is not yet possible to cache generated functions, even

though that'd be desirable from a performance point of view. The

problem is that the generated functions commonly contain pointers into

per-execution memory. The expression evaluationfunctionality needs to

per-execution memory. The expression evaluationmachinery needs to

be redesigned a bit to avoid that. Basically all per-execution memory

needs to be referenced as an offset to one block of memory stored in

an ExprState, rather than absolute pointers into memory.

- jit_inline_above_cost = -1, 0-DBL_MAX - inlining is tried if query has

higher cost.

whenever a query's total cost is above these limits, JITing is

Whenever a query's total cost is above these limits, JITing is

performed.

Alternative costing models, e.g. by generating separate paths for

The obvious seeming approach of JITing expressions individually after

a number of execution turns out not to work too well. Primarily

because emitting many small functions individually has significant

overhead. Secondarily because the timetill JITing occurs causes

overhead. Secondarily because the timeuntil JITing occurs causes

relative slowdowns that eat into the gain of JIT compilation.