Synopsis

codeql dataset upgrade [--threads=<num>] [--ram=<MB>] <options>... -- <dataset>

codeql dataset upgrade [--threads=<num>] [--ram=<MB>] <options>... -- <dataset>

Description

[Plumbing] Upgrade a dataset so it is usable by the current tools.

This does the same ascodeql database upgrade but works on a raw dataset rather than an entire database.

Options

Primary Options

<dataset>

[Mandatory] Path to the raw QL dataset to upgrade.

--search-path=<dir>[:<dir>...]

A list of directories under which QL packs containing upgrade recipesmay be found. Each directory can either be a QL pack (or bundle of packscontaining a.codeqlmanifest.json file at the root) or the immediateparent of one or more such directories.

If the path contains directories trees, their order defines precedencebetween them: if a pack name that must be resolved is matched in morethan one of the directory trees, the one given first wins.

Pointing this at a checkout of the open-source CodeQL repository oughtto work when querying one of the languages that live there.

(Note: On Windows the path separator is;).

--additional-packs=<dir>[:<dir>...]

[Advanced] If this list of directories is given, they will be searchedfor upgrades before the ones in--search-path. The order between thesedoesn't matter; it is an error if a pack name is found in two differentplaces through this list.

This is useful if you're temporarily developing a new version of a packthat also appears in the default path. On the other hand it isnotrecommended to override this option in a config file; some internalactions will add this option on the fly, overriding any configuredvalue.

(Note: On Windows the path separator is;).

--target-dbscheme=<file>

Thetarget dbscheme we want to upgrade to. If this is not given, amaximal upgrade path will be constructed

--target-sha=<sha>

[Advanced] An alternative to--target-dbscheme that gives theinternal hash of the target dbscheme instead of the dbscheme file.

--[no-]allow-downgrades

Include any relevant downgrades if there are no upgrades

Options to control evaluation of upgrade queries

--[no-]tuple-counting

[Advanced] Display tuple counts for each evaluation step in the queryevaluator logs. If the--evaluator-log option is provided, tuplecounts will be included in both the text-based and structured JSON logsproduced by the command. (This can be useful for performanceoptimization of complex QL code).

--timeout=<seconds>

[Advanced] Set the timeout length for query evaluation, in seconds.

The timeout feature is intended to catch cases where a complex querywould take "forever" to evaluate. It is not an effective way to limitthe total amount of time the query evaluation can take. The evaluationwill be allowed to continue as long as each separately timed part of thecomputation completes within the timeout. Currently these separatelytimed parts are "RA layers" of the optimized query, but that mightchange in the future.

If no timeout is specified, or is given as 0, no timeout will be set(except forcodeql test run, where the default timeout is 5 minutes).

-j, --threads=<num>

Use this many threads to evaluate queries.

Defaults to 1. You can pass 0 to use one thread per core on the machine,or -N to leaveN cores unused (except still use at least onethread).

--[no-]save-cache

[Advanced] Aggressively write intermediate results to the disk cache.This takes more time and uses (much) more disk space, but may speed upthe subsequent execution of similar queries.

--[no-]expect-discarded-cache

[Advanced] Make decisions about which predicates to evaluate, and whatto write to the disk cache, based on the assumption that the cache willbe discarded after the queries have been executed.

--[no-]keep-full-cache

[Advanced] Don't clean up the disk cache after evaluation completes.This may save time if you're going to docodeql dataset cleanup orcodeql database cleanup afterwards anyway.

--max-disk-cache=<MB>

Set the maximum amount of space that the disk cache for intermediatequery results can use.

If this size is not configured explicitly, the evaluator will try to usea "reasonable" amount of cache space, based on the size of the datasetand the complexity of the queries. Explicitly setting a higher limitthan this default usage will enable additional caching which can speedup later queries.

--min-disk-free=<MB>

[Advanced] Set target amount of free space on file system.

If--max-disk-cache is not given, the evaluator will try hard tocurtail disk cache usage if the free space on the file system dropsbelow this value.

--min-disk-free-pct=<pct>

[Advanced] Set target fraction of free space on file system.

If--max-disk-cache is not given, the evaluator will try hard tocurtail disk cache usage if the free space on the file system dropsbelow this percentage.

--external=<pred>=<file.csv>

A CSV file that contains rows for external predicate<pred>.Multiple--external options can be supplied.

--xterm-progress=<mode>

[Advanced] Controls whether to show progress tracking during QLevaluation using xterm control sequences. Possible values are:

no: Never produce fancy progress; assume a dumb terminal.

auto(default): Autodetect whether the command is running in anappropriate terminal.

yes: Assume the terminal can understand xterm control sequences. Thefeature still depends on being able to autodetect thesize of theterminal, and will also be disabled if-q is given.

25x80 (or similar): Likeyes, and also explicitly give the size ofthe terminal.

25x80:/dev/pts/17 (or similar): show fancy progress on adifferentterminal than stderr. Mostly useful for internal testing.

Options for controlling outputting of structured evaluator logs

--evaluator-log=<file>

[Advanced] Output structured logs about evaluator performance to thegiven file. The format of this log file is subject to change with nonotice, but will be a stream of JSON objects separated by either twonewline characters (by default) or one if the--evaluator-log-minifyoption is passed. Please usecodeql generate log-summary <file> toproduce a more stable summary of this file, and avoid parsing the filedirectly. The file will be overwritten if it already exists.

--evaluator-log-minify

[Advanced] If the--evaluator-log option is passed, also passingthis option will minimize the size of the JSON log produced, at theexpense of making it much less human readable.

Options to control RAM usage of the upgrade process

-M, --ram=<MB>

The query evaluator will try hard to keep its total memory footprintbelow this value. (However, for large databases it is possible that thethreshold may be broken by file-backed memory maps, which can be swappedto disk in case of memory pressure).

The value should be at least 2048 MB; smaller values will betransparently rounded up.

Common options

-h, --help

Show this help text.

-J=<opt>

[Advanced] Give option to the JVM running the command.

(Beware that options containing spaces will not be handled correctly.)

-v, --verbose

Incrementally increase the number of progress messages printed.

-q, --quiet

Incrementally decrease the number of progress messages printed.

--verbosity=<level>

[Advanced] Explicitly set the verbosity level to one of errors,warnings, progress, progress+, progress++, progress+++. Overrides-vand-q.

--logdir=<dir>

[Advanced] Write detailed logs to one or more files in the givendirectory, with generated names that include timestamps and the name ofthe running subcommand.

(To write a log file with a name you have full control over, insteadgive--log-to-stderr and redirect stderr as desired.)

--common-caches=<dir>

[Advanced] Controls the location of cached data on disk that willpersist between several runs of the CLI, such as downloaded QL packs andcompiled query plans. If not set explicitly, this defaults to adirectory named.codeql in the user's home directory; it will becreated if it doesn't already exist.

Available sincev2.15.2.