Clang tools can help with global refactorings of Chromium code. Clang tools can take advantage of clang's AST to perform refactorings that would be impossible with a traditional find-and-replace regexp:
scoped_ptr<T> fromNULL:https://crbug.com/173286scoped_refptr<T> toT*:https://crbug.com/110610base::Value APIs:https://crbug.com/581865target_os="win" build won't update code that is guarded byOS_POSIX. Performing a global refactoring will often require running the tool once for each build config.A Chromium checkout created withfetch should have everything needed.
For convenience, addthird_party/llvm-build/Release+Asserts/bin to$PATH.
LLVM uses C++11 and CMake. Source code for Chromium clang tools lives in//tools/clang. It is generally easiest to use one of the already-written tools as the base for writing a new tool; the tool in//tools/clang/ast_rewriter is designed for this purpose, and includes explanations its major parts.
Chromium clang tools generally follow this pattern:
clang::ast_matchers::MatchFinder.clang-query is of great use for this partclang::ast_matchers::MatchFinder::MatchCallback to determine what actions to take on each match, and register it withaddMatcher().clang::tooling::FrontendActionFactory from theMatchFinder.clang::tooling::ClangTool::run.clang::tooling::Replacements tostdout.Useful references when writing the tool:
Tools do not directly edit files; rather, they output a series ofedits to be applied later, which have the following format:
==== BEGIN EDITS ====r:::path/to/file/to/edit:::offset1:::length1:::replacement textr:::path/to/file/to/edit:::offset2:::length2:::replacement textr:::path/to/file2/to/edit:::offset3:::length3:::replacement textinclude-user-header:::path/to/file2/to/edit:::-1:::-1:::header/file/to/include.h ...==== END EDITS ====
The header and footer are required. Each line between the header and footer represents one edit. Fields are separated by:::, and the first field must ber (for replacement) orinclude-user-header. A deletion is an edit with no replacement text.
The edits are applied byapply_edits.py, which understands certain conventions:
\0. The script knows to translate\0 back to newlines when applying edits.TODO: Document more aboutSourceLocation and how spelling loc differs from expansion loc, etc.
While clang has aclang::tooling::RefactoringTool to automatically apply the generated replacements and save the results, it doesn't work well for Chromium:
Synopsis:
tools/clang/scripts/build.py--bootstrap--without-android--without-fuchsia \--extra-tools rewrite_to_chrome_style
Running this command builds theOilpan plugin, theChrome style plugin, and theBlink to Chrome style rewriter. Additional arguments to--extra-tools should be the name of subdirectories in//tools/clang. The tool binary will be located inthird_party/llvm-build/Release+Asserts/bin.
It is important to use --bootstrap as there appear to bebugs in the clang library this script produces if you build it with gcc, which is the default.
Once clang is bootsrapped, incremental builds can be done by invokingninja in thethird_party/llvm-build/Release+Asserts directory. In particular, recompiling solely the tool you are writing can be accomplished by executingninja rewrite_to_chrome_style (replacerewrite_to_chrome_style with your tool's name).
First, build all Chromium targets to avoid failures due to missing dependencies that are generated as part of the build:
ninja-Cout/Debug# For non-Windowsninja-d keeprsp-Cout/Debug# For Windows# experimental alternative:$gen_targets= $(ninja-Cout/Debug-t targets all \| grep'^gen/[^: ]*\.[ch][pc]*:' \| cut-f1-d:)ninja-Cout/Debug $gen_targets
Note that running the clang tool with precompiled headers enabled currently produces errors. This can be avoided by settingenable_precompiled_headers = false in the build's gn args.
Then run the actual clang tool to generate a list of edits:
tools/clang/scripts/run_tool.py--tool<path to tool> \--generate-compdb-pout/Debug<path1><path2>...>/tmp/list-of-edits.debug
--generate-compdb can be omitted if the compile DB was already generated and the list of build flags and source files has not changed since generation.
If cross-compiling, specify--target_os. Seegn help target_os for possible values. For example, when cross-compiling a Windows build on Linux/Mac, use--target_os=win.
<path 1>,<path 2>, etc are optional arguments to filter the files to run the tool against. This is helpful when sharding global refactorings into smaller chunks. For example, the following command will run theempty_string tool against just the.c,.cc,.cpp,.m,.mm files in//net. Note that the filtering is not applied to theoutput of the tool - the tool can emit edits that apply to files outside of//net (i.e. edits that apply to headers from//base that got included by source files in//net).
tools/clang/scripts/run_tool.py--tool empty_string \--generate-compdb \-pout/Debug net>/tmp/list-of-edits.debug
Note that some header files might only be included from generated files (e.g. from only from some.cpp files under out/Debug/gen). To make sure that contents of such header files are processed by the clang tool, the clang tool needs to be run against the generated files. The only way to accomplish this today is to pass--all switch torun_tool.py - this will run the clang tool against all the sources from the compilation database.
Finally, apply the edits as follows:
cat/tmp/list-of-edits.debug \| tools/clang/scripts/extract_edits.py \| tools/clang/scripts/apply_edits.py-pout/Debug<path1><path2>...
The apply_edits.py tool will only apply edits to files actually under control ofgit.<path 1>,<path 2>, etc are optional arguments to further filter the files that the edits are applied to. Note that semantics of these filters is distinctly different from the arguments ofrun_tool.py filters - one set of filters controls which files are edited, the other set of filters controls which files the clang tool is run against.
Dumping the AST for a file:
clang++-Xclang-ast-dump-std=c++14 foo.cc| less-R
Usingclang-query to dynamically test matchers (requires checking out and buildingclang-tools-extra; this should happen automatically). The binary is located inthird_party/llvm-build/Release+Asserts/bin:
clang-query-p path/to/compdbbase/memory/ref_counted.cc
If you're running it on a test file instead of a real one, the compdb is optional; it will complain but it still works. Test matchers against the specified file by runningmatch <matcher>, or simplym <matcher>. Use ofrlwrap is highly recommended.
printf debugging:
clang::Decl* decl= result.Nodes.getNodeAs<clang::Decl>("decl"); decl->dumpColor(); clang::Stmt* stmt= result.Nodes.getNodeAs<clang::Stmt>("stmt"); stmt->dumpColor();
By default, the script hides the output of the tool. The easiest way to change that is toreturn 1 from themain() function of the clang tool.
Synposis:
tools/clang/scripts/test_tool.py<tool name>[--apply-edits]
The name of the tool binary and the subdirectory for the tool in//tools/clang must match. The test runner finds all files that match the pattern//tools/clang/<tool name>/tests/*-original.cc, and runs the tool across those files. If--apply-edits switch is presented, tool outputs are applied to respective files and compared to the*-expected.cc version. If there is a mismatch, the result is saved in*-actual.cc. When--apply-edits switch is not presented, tool outputs are compared to*-expected.txt and if different, the result is saved in*-actual.txt. Note that in this case, only one test file is expected.