LMAX welcomes outside contributions to the Disruptor. The following documentation should help you get checked out, set up, and ready to start making changes to the codebase.

Getting & Building the Disruptor

A guide to checking out the code and building it

Getting Started

  1. Check out the project locally to your machine

    $ git clone git://github.com/LMAX-Exchange/disruptor.git
$ cd disruptor

  2. Build a distribution

    $ ./gradlew clean build

    As a result of the build you should find the following files:

    • disruptor/build/libs/disruptor-{VERSION_NUMBER}-SNAPSHOT.jar

    • disruptor/build/libs/disruptor-{VERSION_NUMBER}-SNAPSHOT-javadoc.jar

    • disruptor/build/libs/disruptor-{VERSION_NUMBER}-SNAPSHOT-sources.jar

  3. Run the tests

    $ ./gradlew test

Disruptor Performance Tests

Run the performance tests

When making changes to the Disruptor it is important to be aware of the performance impact of your change.

To measure the performance characteristics of the Disruptor there are many tests in the perftest and jmh sourcesets.

Running Perf Tests

The perf tests are all under src/perftest and are built using a custom framework.

To run them you need to fist build the perf jar and then run a single test at a time, for example:

$ ./gradlew perfJar
$ java -cp build/libs/disruptor-perf-*.jar com.lmax.disruptor.sequenced.OneToOneSequencedThroughputTest

Running JMH Tests

There are also JMH Benchmarks for testing the performance of the Disruptor, these can be run with a gradle command

$ ./gradlew jmh
Isolated CPU benchmarking

Some JMH Benchmarks can be run on machines with isolated cpus to get results with less error.

Build the jmhJar on your development machine and transfer it to your benchmarking machine with isolated cpus:

dev-machine /path/to/disruptor $ ./gradlew jmhJar
dev-machine /path/to/disruptor $ scp dev-machine:build/lib/disruptor-*-jmh.jar bench-machine:

Assuming a system set up with isolated cores, e.g.

bench-machine ~ $ cat /proc/cmdline
... isolcpus=31,33,35,37,39,41,43,45,47,7,9,11,13,15,17,19,21,23 nohz_full=31,33,35,37,39,41,43,45,47,7,9,11,13,15,17,19,21,23 ...

And the system may have a cpuset setup to split application threads from sharing cpus with kernel code:

bench-machine ~ $ cat /cpusets/app/cpus
5,7-23,29,31-47

You can run the benchmarks taskset to some of those cpus so that Java threads (like GC and complication) run on some cores and JMH Benchmark threads are pinned to isolated cpus using the following command (which is running just the one benchmark, MultiProducersSequencerBenchmark):

bench-machine ~ $ cat runBenchmarks.sh
#!/bin/bash
JAVA_HOME=/opt/jdk11/1.11.0.6_zulu11.37.17_ca-1
ISOLATED_CPUS=7,9,11,13,15,17,19,21,23 $JAVA_HOME/bin/java -jar ./disruptor-4.0.0-SNAPSHOT-jmh.jar -rf json -rff /tmp/jmh-result.json -foe true -v NORMAL -prof perf -jvmArgsPrepend -Xmx256m -jvmArgsPrepend -Xms256m -jvmArgsPrepend -XX:MaxDirectMemorySize=1g $@

bench-machine ~ $ sudo cset proc -v --exec app -- taskset -c 5,8,10,12,14,16,18,20,22,29,32,34,36,38,40,42,44,46 ./runBenchmarks.sh MultiProducersSequencerBenchmark

jcstress Tests

https://github.com/openjdk/jcstress/

The Java Concurrency Stress (jcstress) is the experimental harness and a suite of tests to aid the research in the correctness of concurrency support in the JVM, class libraries, and hardware.

The Disruptor has some jcstress tests to experiment and validate the correctness of the concurrency promises that the Disruptor makes.

Running jcstress Tests

$ ./gradlew jcstress

Publishing Release

Prerequisites

  1. All tests should be completing successfully in CI and locally

  2. The changelog is updated with relevant information

  3. build.gradle has been updated with the version information for the proposed release, and pushed to GitHub

  4. Create a new release via GitHub UI, this should tag the commit

GPG Key

To sign the artifact, a requirement for public release, you must have a signing GPG key set up.

Creating a new GPG signing key

If you already have a GPG signing key, you can skip to the next section.

  • If you are on version 2.1.17 or greater, run the command below to generate a GPG key pair:

    $ gpg --full-generate-key

    • At the prompt, specify the kind of key you want, or press Enter to accept the default RSA and RSA.

    • Enter the desired key size. Your key must be at least 4096 bits.

    • Enter the length of time the key should be valid. Press Enter to specify the default selection, indicating that the key doesn’t expire.

  • If you are not on version 2.1.17 or greater, the gpg --full-generate-key command doesn’t work. Use the following command instead:

    $ gpg --default-new-key-algo rsa4096 --gen-key

  • Verify that your selections are correct.

  • Enter your user ID information.

  • Type a secure passphrase.

  • (is using gpg >= 2.1) Export the keyring to a gpg file

    gpg --keyring secring.gpg --export-secret-keys > ~/.gnupg/secring.gpg

Adding signing options for gradle

  • signing.keyId is the last 8 characters of the public key id for the signing key, in this example, 2657EDD1.

  • signing.password is the passphrase for your keyring

  • signing.secretKeyRingFile is the absolute path to the secret key ring file containing your private key

$ gpg --list-secret-keys --keyid-format SHORT
/home/dev/.gnupg/pubring.kbx
-----------------------------
sec   rsa4096/2657EDD1 2021-12-31 [SC]
      EA38CE5CFD74BBB831611491F9CE691A2657EDD1
uid         [ultimate] LMAX Coders (LMAX Dev Team) <coders@lmax.com>
ssb   rsa4096/F4831415 2021-12-31 [E]


$ cat gradle.properties
signing.keyId=2657EDD1
signing.password=<passphrase used to protect your private key>
signing.secretKeyRingFile=/home/dev/.gnupg/secring.gpg

See: https://docs.gradle.org/current/userguide/signing_plugin.html

Publish your public key

You need to publish your public key so that sonatype can verify who signed the release artifacts.

gpg --keyserver https://keyserver.ubuntu.com/ --send-keys 2657EDD1

Note: Other keyservers are available

Release process

  1. Clean any existing build data

    $ ./gradlew clean

  2. Add gradle.properties file in the root of the disruptor project with the following content

    $ cat gradle.properties

sonatypeUsername=<username for sonatype>
sonatypePassword=<password for sonatype>
signing.keyId=<signing-key>
signing.password=<password>>
signing.secretKeyRingFile=/home/dev/.gnupg/secring.gpg

  3. Build & upload to Sonatype

    $ ./gradlew publish

    Note: Make sure the signing step was not skipped

  4. Release in to Sonatype

  5. Remove gradle.properties file

    This file is ignored from git, but I would not advise leaving it on disk long term as it has 2 passwords in plain text.

Development tips

Git Hooks

This project uses GitHub Actions for CI which runs the gradle build task to check the project can be built on each commit/pull request.

So that you don’t push changes that will fail the simple test & check steps we would suggest adding the git pre-commit hook which will block any commit you try to do locally if the code fails these steps.

The git hook can be set up using the following gradle task:

$ ./gradlew setUpGitHooks

Signed Commits

Git is cryptographically secure, but it’s not foolproof. If you’re taking work from others on the internet and want to verify that commits are actually from a trusted source, Git has a few ways to sign and verify work using GPG.