TL;DR

If you have the prerequisites, running

  $ bazel build gerrit

should generate a .war file under bazel-bin/gerrit.war.

Prerequisites

To build Gerrit from source, you need:

  • A Linux or macOS system (Windows is not supported)

  • A JDK for Java 11 or Java 17

  • Python 3

  • Node.js (including npm)

  • Bower (npm install -g bower)

  • Bazel -launched with Bazelisk

  • Maven

  • zip, unzip

  • curl

  • gcc

Bazel

Bazelisk is a version manager for Bazel, similar to how nvm manages npm versions. It takes care of downloading and installing Bazel itself, so you don’t have to worry about using the correct version of Bazel. One particular advantage to using Bazelisk is that you can jump between different versions of Gerrit and not worry about which version of Bazel you need.

Bazelisk can be installed in different ways: Bazelisk Installation. To execute the correct version of Bazel using Bazelisk you simply replace the bazel command with bazelisk.

Java

Ensure that the required Java version is installed and that JAVA_HOME is set to it.

To check the installed version of Java, open a terminal window and run:

java -version

Java 11 support

To build Gerrit with Java 11 language level, run:

  $ bazelisk build --config=java11 :release

Java 17 support

Java 17 is supported. To build Gerrit with Java 17, run:

  $ bazelisk build :release

To run the tests with Java 17, run:

  $ bazelisk test //...

Node.js and npm packages

Building on the Command Line

Gerrit Development WAR File

To build the Gerrit web application:

  bazelisk build gerrit

The output executable WAR will be placed in:

  bazel-bin/gerrit.war

Gerrit Release WAR File

To build the Gerrit web application that includes the Gerrit UI, core plugins and documentation:

  bazelisk build release

The output executable WAR will be placed in:

  bazel-bin/release.war

Headless Mode

To build Gerrit in headless mode, i.e. without the Gerrit UI:

  bazelisk build headless

The output executable WAR will be placed in:

  bazel-bin/headless.war

Extension and Plugin API JAR Files

To build the extension, plugin and acceptance-framework JAR files:

  bazelisk build api

The output archive that contains Java binaries, Java sources and Java docs will be placed in:

  bazel-bin/api.zip

Install {extension,plugin,acceptance-framework}-api to the local maven repository:

  tools/maven/api.sh install

Install gerrit.war to the local maven repository:

  tools/maven/api.sh war_install

Plugins

  bazelisk build plugins:core

The output JAR files for individual plugins will be placed in:

  bazel-bin/plugins/<name>/<name>.jar

The JAR files will also be packaged in:

  bazel-bin/plugins/core.zip

To build a specific plugin:

  bazelisk build plugins/<name>

The output JAR file will be be placed in:

  bazel-bin/plugins/<name>/<name>.jar

Note that when building an individual plugin, the core.zip package is not regenerated.

Using an IDE.

IntelliJ

The Gerrit build works with Bazel’s IntelliJ plugin. Please follow the instructions on IntelliJ Setup.

Eclipse

Generating the Eclipse Project

Create the Eclipse project:

  tools/eclipse/project.py

and then follow the setup instructions.

Refreshing the Classpath

If an updated classpath is needed, the Eclipse project can be refreshed and missing dependency JARs can be downloaded by running project.py again. For IntelliJ, you need to click the Sync Project with BUILD Files button of Bazel plugin.

Documentation

To build only the documentation for testing or static hosting:

  bazelisk build Documentation:searchfree

The html files will be bundled into searchfree.zip in this location:

  bazel-bin/Documentation/searchfree.zip

To use local fonts with the searchfree target:

  bazel build Documentation:searchfree_safe

The html files will be bundled into searchfree.zip or searchfree_safe.zip in this location:

  bazel-bin/Documentation/searchfree.zip
  bazel-bin/Documentation/searchfree_safe.zip

To generate HTML files skipping the zip archiving:

  bazelisk build Documentation

And open bazel-bin/Documentation/index.html.

To build the Gerrit executable WAR with the documentation included:

  bazelisk build withdocs

The WAR file will be placed in:

  bazel-bin/withdocs.war

Alternatively, one can generate the documentation as flat files:

  bazelisk build Documentation:Documentation

The html, css, js files are placed in:

 `bazel-bin/Documentation/`

Running Unit Tests

Bazel BUILD files define test targets for Gerrit. You can run all declared test targets with:

  bazelisk test --build_tests_only //...

Running Test Groups

To run one or more specific labeled groups of tests:

  bazelisk test --test_tag_filters=api,git //...

The following label values are currently supported for the group name:

  • annotation

  • api

  • edit

  • git

  • git-protocol-v2

  • git-upload-archive

  • notedb

  • pgm

  • rest

  • server

  • ssh

We can also select tests within a specific BUILD target group. For example javatests/com/google/gerrit/acceptance/rest/account/BUILD declares a rest_account test target group:

  bazelisk test //javatests/com/google/gerrit/acceptance/rest/account:rest_account

Debugging Tests

To debug specific tests you will need to select the test target containing that test then use --test_filter to select the specific test you want. This --test_filter is a regex and can be used to select multiple tests out of the target:

  bazelisk test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget

For example javatests/com/google/gerrit/acceptance/api/change/BUILD defines a test target group for every *IT.java file in the directory. We can execute the single getAmbiguous() test found in ChangeIT.java using this --test_filter and target:

  bazelisk test --test_output=streamed \
    --test_filter=com.google.gerrit.acceptance.api.change.ChangeIT.getAmbiguous \
    //javatests/com/google/gerrit/acceptance/api/change:ChangeIT

Additional Test Filtering

To run only tests that do not use SSH:

  bazelisk test --test_env=GERRIT_USE_SSH=NO //...

To exclude tests that have been marked as flaky:

  bazelisk test --test_tag_filters=-flaky //...

To exclude tests that require very recent git client version:

  bazelisk test --test_tag_filters=-git-protocol-v2 //...

To run the tests against a specific index backend (LUCENE, FAKE):

  bazelisk test --test_env=GERRIT_INDEX_TYPE=LUCENE //...

Bazel itself supports a multitude of ways to specify targets for fine-grained test selection that can be combined with many of the examples above.

Test Caching

By default Bazel caches test results and will not reexecute tests unless they or their dependencies have been modified. To ignore cached test results and force the tests to rerun:

  bazelisk test --cache_test_results=NO //...

Running Plugin Tests

Running tests for Gerrit plugins follows the process above. From within the Gerrit project root with the desired plugins checked out into plugins/ we execute Bazel with the appropriate target:

  bazelisk test //plugins/replication/...

Debugging Unit Tests

In some cases it may be necessary to debug a test while running it in bazel. For example, when we observe a different test result in Eclipse and bazel. Using the --java_debug option will start the JVM in debug mode and await for a remote debugger to attach.

Example:

  bazelisk test --java_debug --test_tag_filters=delete-project //...
  ...
  Listening for transport dt_socket at address: 5005
  ...

Now attach with a debugger to the port 5005. For example use "Remote Java Application" launch configuration in Eclipse and specify the port 5005.

Controlling logging level

Per default, logging level is set to INFO level for all tests. The DEBUG log level can be enabled for the tests.

In IDE, set -Dgerrit.logLevel=debug as a VM argument. With bazel, pass GERRIT_LOG_LEVEL=debug environment variable:

  bazelisk test --test_filter=com.google.gerrit.server.notedb.ChangeNotesTest \
    --test_env=GERRIT_LOG_LEVEL=debug \
    javatests/com/google/gerrit/server:server_tests

The log results can be found in: bazel-testlogs/javatests/com/google/gerrit/server/server_tests/test.log.

Dependencies

Dependency JARs are normally downloaded as needed, but you can download everything upfront. This is useful to enable subsequent builds to run without network access:

  bazelisk fetch //...

When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by curl:

  export http_proxy=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>

Redirection to local mirrors of Maven Central and the Gerrit storage bucket is supported by defining specific properties in local.properties, a file that is not tracked by Git:

  echo download.GERRIT = http://nexus.my-company.com/ >>local.properties
  echo download.MAVEN_CENTRAL = http://nexus.my-company.com/ >>local.properties

The local.properties file may be placed in the root of the gerrit repository being built, or in ~/.gerritcodereview/. The file in the root of the gerrit repository has precedence.

Building against unpublished Maven JARs

To build against unpublished Maven JARs, like PrologCafe, the custom JARs must be installed in the local Maven repository (mvn clean install) and maven_jar() must be updated to point to the MAVEN_LOCAL Maven repository for that artifact:

 maven_jar(
   name = 'prolog-runtime',
   artifact = 'com.googlecode.prolog-cafe:prolog-runtime:42',
   repository = MAVEN_LOCAL,
 )

Building against artifacts from custom Maven repositories

To build against custom Maven repositories, two modes of operations are supported: with rewrite in local.properties and without.

Without rewrite the URL of custom Maven repository can be directly passed to the maven_jar() function:

  GERRIT_FORGE = 'http://gerritforge.com/snapshot'

  maven_jar(
    name = 'gitblit',
    artifact = 'com.gitblit:gitblit:1.4.0',
    sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa',
    repository = GERRIT_FORGE,
 )

When the custom URL has to be rewritten, then the same logic as with Gerrit known Maven repository is used: Repo name must be defined that matches an entry in local.properties file:

  download.GERRIT_FORGE = http://my.company.mirror/gerrit-forge

And corresponding WORKSPACE excerpt:

  GERRIT_FORGE = 'GERRIT_FORGE:'

  maven_jar(
    name = 'gitblit',
    artifact = 'com.gitblit:gitblit:1.4.0',
    sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa',
    repository = GERRIT_FORGE,
 )

Building against SNAPSHOT Maven JARs

To build against SNAPSHOT Maven JARs, the complete SNAPSHOT version must be used:

 maven_jar(
   name = "pac4j-core",
   artifact = "org.pac4j:pac4j-core:3.5.0-SNAPSHOT-20190112.120241-16",
   sha1 = "da2b1cb68a8f87bfd40813179abd368de9f3a746",
 )

To accelerate builds, several caches are activated per default:

  • ~/.gerritcodereview/bazel-cache/downloaded-artifacts

  • ~/.gerritcodereview/bazel-cache/repository

  • ~/.gerritcodereview/bazel-cache/cas

The downloaded-artifacts cache can be relocated by setting the GERRIT_CACHE_HOME environment variable. The other two can be adjusted with bazelisk build options --repository_cache and --disk_cache respectively.

Currently none of these caches have a maximum size limit. See this bazel issue for details. Users should watch the cache sizes and clean them manually if necessary.

NPM Binaries

Parts of the Gerrit web app build require running NPM-based JavaScript programs as "binaries". We don’t attempt to resolve and download NPM dependencies at build time, but instead use pre-built bundles of the NPM binary along with all its dependencies. Some packages on registry.npmjs.org come with their dependencies bundled, but this is the exception rather than the rule. More commonly, to add a new binary to this list, you will need to bundle the binary yourself.

Note
We can only use binaries that meet certain licensing requirements, and that do not include any native code.

Start by checking that the license and file types of the bundle are acceptable:

  gerrit_repo=/path/to/gerrit
  package=some-npm-package
  version=1.2.3

  # Note - yarn must be installed before running the following commands
  yarn global add license-checker && \
  rm -rf /tmp/$package-$version && mkdir -p /tmp/$package-$version && \
  cd /tmp/$package-$version && \
  yarn add $package@$version && \
  license-checker | grep licenses: | sort -u

This will output a list of the different licenses used by the package and all its transitive dependencies. We can only legally distribute a bundle via our storage bucket if the licenses allow us to do so. As long as all of the listed license are allowed by Google’s. Any by_exception_only, commercial, prohibited, or unlisted licenses are not allowed; otherwise, it is ok to distribute the source. If in doubt, contact a maintainer who is a Googler.

Next, check the file types:

  cd /tmp/$package-$version
  find . -type f | xargs file | grep -v 'ASCII\|UTF-8\|empty$'

If you see anything that looks like a native library or binary, then we can’t use the bundle.

If everything looks good, install the package with the following command:

# Add to ui_npm. Other packages.json can be updated in the same way
cd $gerrit_repo/polygerrit-ui/app
bazelisk run @yarn//:yarn add $package

Update the polygerrit-ui/app/node_modules_licenses/licenses.ts file. You should add licenses for the package itself and for all transitive dependencies. If you forgot to add a license, the Documentation:check_licenses test will fail.

After the update, commit all changes to the repository (including yarn.lock).

Note

If a npm package has transitive dependencies (or just several files) with a not allowed license and you can’t avoid use it in release, then you can add this package. For example some packages contain demo-code with a different license. Another example - optional dependencies, which are not needed to build the Gerrit web app, but they are installed together with the package anyway.

In this case you should exclude all files and/or transitive dependencies with a not allowed license. Adding such package requires additional updates:

  • Add dependencies (or files) to the license.ts with an appropriate license marked with allowed: false.

  • update package.json postinstall script to remove all non-allowed files (if you don’t update postinstall script, Documentation:check_licenses test will fail.)

Update NPM Binaries

To update a NPM binary the same actions as for a new one must be done (check licenses, update licenses.ts file, etc…​). The only difference is a command to install a package: instead of bazelisk run @yarn//:yarn add $package you should run the bazelisk run @yarn//:yarn upgrade …​ command with correct arguments. You can find the list of arguments in the yarn upgrade doc.

Google Remote Build Support

The Bazel build can be used with Google’s Remote Build Execution.

This needs the following setup steps:

gcloud auth application-default login
gcloud services enable remotebuildexecution.googleapis.com  --project=${PROJECT}

Create a worker pool. The instances should have at least 4 CPUs each for adequate performance.

gcloud alpha remote-build-execution worker-pools create default \
    --project=${PROJECT} \
    --instance=default_instance \
    --worker-count=50 \
    --machine-type=e2-standard-4 \
    --disk-size=200

Due to outdated Git version in official RBE docker images, a custom RBE docker image must be used. To build custom docker imager, change to the directory tools/platforms and build and publish custom RBE docker image.

To build the custom RBE docker image, run:

docker build -t gcr.io/api-project-164060093628/rbe-ubuntu18-04 .

To publish the custom RBE docker image, run:

docker push gcr.io/api-project-164060093628/rbe-ubuntu18-04
[...]
latest: digest: sha256:de5186d4313630a6111f9a2449b72563d0bc59ec9fb60956f063b69a38a76834 size: 1584

Re-build rbe_autoconfig project conduct a new release and switch to using it in WORKSPACE file.

Note, to authenticate to the gcr.io registry, the following command must be used:

gcloud auth configure-docker

To see the documentation, developer must be added to this group: https://groups.google.com/forum/#!forum/rbe-alpha-customers.

To use RBE, execute

bazelisk test --config=remote \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    javatests/...

BuildBuddy Remote Build Support

To utilize the BuildBuddy Remote Build Execution service, please consult the documentation available at the following link: BuildBuddy.

To use RBE, execute

bazelisk test --config=remote_bb \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    --remote_header=x-buildbuddy-api-key=YOUR_API_KEY \
    javatests/...