Repository files navigation

go-pgquery is a drop-in replacement forpg_query_go which uses cgo to invoke thelibpg_query library. By default, the c code is compiled to a WebAssembly moduleand accessed with the pure Go runtime,wazero. This means that it is compatible withany Go application, regardless of availability of cgo, and will not have toolchain issuessuch as those that libpg_query commonly sees with MacOS upgrades.

Unlike other wasilibs libraries, this library cannot be used with TinyGo because it relieson WebAssembly threads support, which is not expected in TinyGo for quite some time.

The library is expected to be fully compatible with pg_query_go. The same tests from pg_query_goare run, and in addition test cases from libpg_query are also imported to increase coverage.There are no known behavior differences with upstream libraries.

While all tests from both upstream sources currently execute, this is despiteissues that werefound that only affected the WebAssembly toolchain. While coverage is expected to be high, it isnot perfect and it is possible future issues that only affect WebAssembly compilation will befound. If encountering any issues, always feel free to file an issue.

go-pgquery is a standard Go library package and can be added to a go.mod file. It will work fine inany Go project.

go get github.com/wasilibs/go-pgquery

Because the library is a drop-in replacement for pg_query_go, it is sufficient to changeimport statements.

import"github.com/pganalyze/pg_query_go/v6/parser"

can be changed to

import"github.com/wasilibs/go-pgquery"

Note that to allow users to write transformation logic targeting both libraries, this libraryreturns parse trees using the types from pg_query_go. This means you will have both librariesin your requirements and will need to be careful calling the entry point functions likeParsefrom go-pgquery, not pg_query_go. This may change in the future.

This library also supports opting into using cgo to wrap libpg_query instead of using WebAssembly.The build tagpgquery_cgo can be used to enable cgo support. Note, this is exactly the samecode aspg_query_go - if you only need cgo support, it is recommended to use the officiallibrary instead of this one.

Benchmarks are run against every commit in thebench workflow. GitHub action runners are highlyvirtualized and do not have stable performance across runs, but the relative numbers within a runshould still be somewhat, though not precisely, informative.

One run looks like this

                              │ build/bench_default.txt │           build/bench.txt           │         build/bench_cgo.txt         │                              │         sec/op          │    sec/op      vs base              │    sec/op     vs base               │ParseSelect1-4                            24.574µ ± ∞ ¹   24.859µ ± ∞ ¹       ~ (p=0.421 n=5)   6.558µ ± ∞ ¹  -73.31% (p=0.008 n=5)ParseSelect2-4                             82.56µ ± ∞ ¹    81.40µ ± ∞ ¹  -1.41% (p=0.032 n=5)   21.68µ ± ∞ ¹  -73.74% (p=0.008 n=5)ParseCreateTable-4                        186.70µ ± ∞ ¹   186.20µ ± ∞ ¹       ~ (p=1.000 n=5)   51.30µ ± ∞ ¹  -72.52% (p=0.008 n=5)ParseSelect1Parallel-4                    12.027µ ± ∞ ¹   11.793µ ± ∞ ¹       ~ (p=0.548 n=5)   3.294µ ± ∞ ¹  -72.61% (p=0.008 n=5)ParseSelect2Parallel-4                     39.83µ ± ∞ ¹    39.23µ ± ∞ ¹  -1.50% (p=0.032 n=5)   11.41µ ± ∞ ¹  -71.36% (p=0.008 n=5)ParseCreateTableParallel-4                 91.27µ ± ∞ ¹    90.28µ ± ∞ ¹       ~ (p=0.222 n=5)   27.76µ ± ∞ ¹  -69.58% (p=0.008 n=5)RawParseSelect1-4                         22.439µ ± ∞ ¹   22.049µ ± ∞ ¹       ~ (p=0.548 n=5)   4.613µ ± ∞ ¹  -79.44% (p=0.008 n=5)RawParseSelect2-4                          73.07µ ± ∞ ¹    74.15µ ± ∞ ¹       ~ (p=0.690 n=5)   15.50µ ± ∞ ¹  -78.80% (p=0.008 n=5)RawParseCreateTable-4                     167.01µ ± ∞ ¹   167.32µ ± ∞ ¹       ~ (p=1.000 n=5)   36.12µ ± ∞ ¹  -78.38% (p=0.008 n=5)RawParseSelect1Parallel-4                 10.667µ ± ∞ ¹   10.552µ ± ∞ ¹       ~ (p=0.095 n=5)   2.280µ ± ∞ ¹  -78.63% (p=0.008 n=5)RawParseSelect2Parallel-4                 35.777µ ± ∞ ¹   35.309µ ± ∞ ¹       ~ (p=0.135 n=5)   7.798µ ± ∞ ¹  -78.20% (p=0.008 n=5)RawParseCreateTableParallel-4              82.15µ ± ∞ ¹    81.99µ ± ∞ ¹       ~ (p=0.079 n=5)   18.68µ ± ∞ ¹  -77.26% (p=0.008 n=5)FingerprintSelect1-4                      15.984µ ± ∞ ¹   15.631µ ± ∞ ¹       ~ (p=0.310 n=5)   2.465µ ± ∞ ¹  -84.58% (p=0.008 n=5)FingerprintSelect2-4                      36.803µ ± ∞ ¹   36.140µ ± ∞ ¹       ~ (p=0.095 n=5)   5.006µ ± ∞ ¹  -86.40% (p=0.008 n=5)FingerprintCreateTable-4                  60.191µ ± ∞ ¹   60.951µ ± ∞ ¹       ~ (p=0.151 n=5)   9.172µ ± ∞ ¹  -84.76% (p=0.008 n=5)NormalizeSelect1-4                         9.864µ ± ∞ ¹   10.151µ ± ∞ ¹       ~ (p=0.095 n=5)   1.691µ ± ∞ ¹  -82.86% (p=0.008 n=5)NormalizeSelect2-4                        21.154µ ± ∞ ¹   22.804µ ± ∞ ¹  +7.80% (p=0.008 n=5)   2.987µ ± ∞ ¹  -85.88% (p=0.008 n=5)NormalizeCreateTable-4                    21.750µ ± ∞ ¹   22.841µ ± ∞ ¹  +5.02% (p=0.008 n=5)   4.312µ ± ∞ ¹  -80.17% (p=0.008 n=5)

We see that the WebAssembly version performs about 4-5x slower than cgo. This is a big difference and the largest we'vefound in wasilibs. It may be because of inefficiency in the implementation of exception handling. If your applicationrequires online processing of high volume of queries, it may be better to continue with cgo. However, if it is an offlinetool that only parses a small number of queries, it may still be reasonable performance in exchange for a simplifiedbuild toolchain - operations still complete within dozens of microseconds.