Repository files navigation

A tool for transforming and moving code between repositories.

Copybara is a tool used internally at Google. It transforms and moves code between repositories.

Often, source code needs to exist in multiple repositories, and Copybara allows you to transformand move source code between these repositories. A common case is a project that involvesmaintaining a confidential repository and a public repository in sync.

Copybara requires you to choose one of the repositories to be the authoritative repository, so thatthere is always one source of truth. However, the tool allows contributions to any repository, andany repository can be used to cut a release.

The most common use case involves repetitive movement of code from one repository to another.Copybara can also be used for moving code once to a new repository.

Examples uses of Copybara include:

• Importing sections of code from a confidential repository to a public repository.

• Importing code from a public repository to a confidential repository.

• Importing a change from a non-authoritative repository into the authoritative repository. Whena change is made in the non-authoritative repository (for example, a contributor in the publicrepository), Copybara transforms and moves that change into the appropriate place in theauthoritative repository. Any merge conflicts are dealt with in the same way as an out-of-datechange within the authoritative repository.

One of the main features of Copybara is that it is stateless, or more specifically, that it storesthe state in the destination repository (As a label in the commit message). This allows severalusers (or a service) to use Copybara for the same config/repositories and get the same result.

Currently, the only supported type of repository is Git. Copybara is also ableto read from Mercurial repositories, but the feature is still experimental.The extensible architecture allows adding bespoke origins and destinationsfor almost any use case.Official support for other repositories types will be added in the future.

core.workflow(name="default",origin=git.github_origin(url="https://github.com/google/copybara.git",ref="master",    ),destination=git.destination(url="file:///tmp/foo",    ),# Copy everything but don't remove a README_INTERNAL.txt file if it exists.destination_files=glob(["third_party/copybara/**"],exclude= ["README_INTERNAL.txt"]),authoring=authoring.pass_thru("Default email <default@default.com>"),transformations= [core.replace(before="//third_party/bazel/bashunit",after="//another/path:bashunit",paths=glob(["**/BUILD"])),core.move("","third_party/copybara")    ],)

Run:

$ (mkdir /tmp/foo;cd /tmp/foo; git init --bare)$ copybara copy.bara.sky

The easiest way to start is with weekly "snapshot" releases, that include pre-built a binary.Note that these are released automatically without any manual testing, version compatibility or correctness guarantees.

Choose a release fromhttps://github.com/google/copybara/releases.

To use an unreleased version of copybara, so you need to compile from HEAD.In order to do that, you need to do the following:

• Install JDK 11.
• Install Bazel.
• Clone the copybara source locally:
  • git clone https://github.com/google/copybara.git
• Build:
  • bazel build //java/com/google/copybara
  • bazel build //java/com/google/copybara:copybara_deploy.jar to create an executable uberjar.
• Tests:bazel test //... if you want to ensure you are not using a broken version. Note thatcertain tests require the underlying tool to be installed(e.g. Mercurial, Quilt, etc.). It isfine to skip those tests if your Pull Request is unrelated to those modules (And our CI willrun all the tests anyway).

These packages can be installed using the appropriate package manager for yoursystem.

• aur/copybara-git

If you use Intellij and the Bazel plugin, use this project configuration:

directories:  copybara/integration  java/com/google/copybara  javatests/com/google/copybara  third_partytargets:  //copybara/integration/...  //java/com/google/copybara/...  //javatests/com/google/copybara/...  //third_party/...

Note: configuration files can be stored in any place, even in a local folder.We recommend using a VCS (like git) to store them; treat them as source code.

If using a weekly snapshot release, install Copybara as follows:

1. Copybara ships with class files with version 65.0, so it must be run with Java Runtime 21 or greater. Add to your.bazelrc file:run --java_runtime_version=remotejdk_21
2. Usehttp_jar to download the release artifact.
   • In WORKSPACE:load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_jar")
   • In MODULE.bazel:http_jar = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_jar")
3. In WORKSPACE or MODULE.bazel, fill in the[version] placeholder:

   http_jar(name="com_github_google_copybara",# Fill in from https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar.sha256# sha256 = "",urls= ["https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar"],)

4. In any BUILD file (perhaps/tools/BUILD.bazel) declare thejava_binary:

   load("@rules_java//java:java_binary.bzl","java_binary")java_binary(name="copybara",main_class="com.google.copybara.Main",runtime_deps= ["@com_github_google_copybara//jar"],)

5. Use that target withbazel run, for examplebazel run //tools:copybara -- migrate copy.bara.sky

There are convenience macros defined for all of Copybara's dependencies. Add thefollowing code to yourWORKSPACE file, replacing{{ sha256sum }} and{{ commit }} as necessary.

http_archive(name="com_github_google_copybara",sha256="{{ sha256sum }}",strip_prefix="copybara-{{ commit }}",url="https://github.com/google/copybara/archive/{{ commit }}.zip",)load("@com_github_google_copybara//:repositories.bzl","copybara_repositories")copybara_repositories()load("@com_github_google_copybara//:repositories.maven.bzl","copybara_maven_repositories")copybara_maven_repositories()load("@com_github_google_copybara//:repositories.go.bzl","copybara_go_repositories")copybara_go_repositories()

You can then build and run the Copybara tool from within your workspace:

bazel run @com_github_google_copybara//java/com/google/copybara --<args...>

NOTE: Docker use is currently experimental, and we encourage feedback or contributions.

You can build copybara using Docker like so

docker build --rm -t copybara.

Once this has finished building, you can run the image like so from the root ofthe code you are trying to use Copybara on:

docker run -it -v"$(pwd)":/usr/src/app copybarahelp

In addition to passing cmd args to the container, you can also set the followingenvironment variables as an alternative:

• COPYBARA_SUBCOMMAND=migrate
  • allows you to change the command run, defaults tomigrate
• COPYBARA_CONFIG=copy.bara.sky
  • allows you to specify a path to a config file, defaults to rootcopy.bara.sky
• COPYBARA_WORKFLOW=default
  • allows you to specify the workflow to run, defaults todefault
• COPYBARA_SOURCEREF=''
  • allows you to specify the sourceref, defaults to none
• COPYBARA_OPTIONS=''
  • allows you to specify options for copybara, defaults to none

docker run \    -e COPYBARA_SUBCOMMAND='validate' \    -e COPYBARA_CONFIG='other.config.sky' \    -v"$(pwd)":/usr/src/app \    -it copybara

There are a number of ways by which to share your git config and ssh credentialswith the Docker container, an example is below:

docker run \    -v~/.gitconfig:/root/.gitconfig:ro \    -v~/.ssh:/root/.ssh \    -v${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK} -e SSH_AUTH_SOCK    -v"$(pwd)":/usr/src/app \    -it copybara

We are still working on the documentation. Here are some resources:

• Reference documentation
• Examples
• Tutorial on how to get started

If you have any questions about how Copybara works, please contact us at ourmailing list.

• If you want to see the test errors in Bazel, instead of having tocat thelogs, add this line to your~/.bazelrc:

  test --test_output=streamed