Thom Chiovoloni

16 minute read

Here’s some blocks of yaml you can copy-paste to make github actions work for your Rust project. Go nuts.

This is my third1 time trying to write a github actions post, and so here’s the version where I just dump a whole buncha code on you. This is my attempt to document a bunch of crap I’ve had to figure out the hard way, and also show people like me who thought github actions were “okay but very heavyweight and clunky” that some of that is just the sample code.

Caveats and Disclaimers

Anyway a couple things to get out of the way:

  1. Check the date. This is the kind of blog post that could easily get stale, and the stale info provide little value. I’m writing this on 2020-09-062, and the last update was on 2020-09-11. If that sounds like kind of a long time ago to you (to be concrete: if the time since the last update is measured in years, take everything with a grain of salt).

  2. Feel free to copy/paste these configs/snippets into your project. I disclaim any copyright possible, and attempt to provide it under the public domain, or licensed as CC0 if that’s not possible.

  3. In general I’ve favored just simplest/shortest/most obvious way of doing things, even if there are ways with various other benefits3.

  4. I do cover ways some of this can go wrong, but in general, not exhaustively. In particular you should be familiar with the requirements to build your crate, and you should expect to have a bit of trouble if you have native (e.g. C) dependencies and want to do things like sanitizers or cross. Trying to cover ways C code may fail to build is a way too big topic for this blog post anyway…

Anyway, don’t read into my choices too deeply for a lot of these things. For example: I’ve used hecrj/setup-rust-action over actions-rs/toolchain in most of these, but both are great, I just needed to pick one.

Quick Setup / Overview

If you don’t know how to start at all: Make a file in $your_repo/.github/workflows/ci.yml (the name ci.yml doesn’t seem to really matter beyond occasionally being that shows up in some UI. Call it whatever you want), creating any folders as needed.

Put this in the file:

# You can call this whatever you want — the name doesn't really matter.
name: CI
# Runs on all PRs and pushes to any branch. See the section
# on event hooks for other options.
on:
  pull_request:
  push:
    branches:
      # Delete whichever of these you aren't using.
      - master
      - main

jobs:
  # Jobs go here. Most of is article is about things
  # to put in this section.

Find jobs that go in jobs:, and possibly adjust the other bits as described in the .

Jobs

Quick note on the convention used in these. Each “recipe” will start like:

jobs:
  # ...
  some-job-name:
    # Lots of code here

This doesn’t mean you should have multiple jobs: blocks in your file (doing so would be invalid YAML), but is to indicate that the some-job-name: (and its contents) go inside the existing jobs: dictionary.

Tests

Run tests on stable/beta/nightly

This is a good starting point for most crates

jobs:
  # ...
  test-versions:
    name: Test Rust ${{ matrix.rust }}
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        # if you have an MSRV, you can of course include it here too.
        rust: [stable, beta, nightly]
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: ${{ matrix.rust }}
      # You may want to test other featuresets here...
      - run: cargo test --verbose --workspace --all-features
      - run: cargo test --verbose --workspace --no-default-features
      # ...

Run tests on macOS and Windows (in various configurations)

Note: This requires that you can build on these platforms, of course.

jobs:
  # ...
  test-mac-win:
    name: Test Rust ${{ matrix.rust }} on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
        - { rust: stable,            os: macos-latest }
        - { rust: stable,            os: windows-latest }
        # Note: If you don't know if you want to be
        # testing on these configurations, you probably
        # don't need to include these lines.
        - { rust: stable-x86_64-gnu, os: windows-latest }
        - { rust: stable-i686-msvc,  os: windows-latest }
        - { rust: stable-i686-gnu,   os: windows-latest }
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: ${{ matrix.rust }}
      - run: cargo test --verbose --workspace --all-features
      - run: cargo test --verbose --workspace --no-default-features
      # ...

Run Tests on ARM, MIPS, … using cross

This has a lot of caveats and can easily fail if you have native deps, but is very cool for cases where you know you’re pure rust or have native deps which can be cross compiled.

See https://github.com/rust-embedded/cross#supported-targets for the list of targets (But note: an absent checkbox from the test column doesn’t mean you can’t run tests on it, just that it might fail because of a QEMU bug or something).

jobs:
  # ...
  cross-test:
    name: Test on ${{ matrix.target }} (using cross)
    runs-on: ubuntu-latest
    env:
      # Optional: see note below
      RUSTFLAGS: '--cfg using_cross'
      RUSTDOCFLAGS: '--cfg using_cross'
    strategy:
      fail-fast: false
      matrix:
        target:
          # 32-bit x86
          - i686-unknown-linux-gnu
          # 32-bit ARM (on Android)
          - armv7-linux-androideabi
          # 64-bit ARM (on Android)
          - aarch64-linux-android
          # 32-bit MIPS(-BE) (that is: big endian)
          - mips-unknown-linux-gnu
          # 64-bit MIPS(-BE) (that is: big endian)
          - mips64-unknown-linux-gnuabi64
          # Tons of others...
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
      - run: cargo install cross
      # Note: just use `cross` as you would `cargo`, but always
      # pass the `--target=${{ matrix.target }}` arg. (Yes, really).
      - run: cross test --verbose --target=${{ matrix.target }} --no-default-features
      - run: cross test --verbose --target=${{ matrix.target }} --all-features
      # ...

Note that cross can have trouble with code that uses threads sometimes. The best way I’ve found around this to add --cfg using_cross to RUSTFLAGS/RUSTDOCFLAGS (as shown above) and then potentially disable certain tests by using: something like:

#[test]
#[cfg(not(using_cross))]
fn test_that_upsets_cross() {
  // ...
}

Using Sanitizers/Checkers On Tests

If you write unsafe code, these might be be a good idea. Unfortunately, they’re also in the category of “things that are vastly harder to do if you have C dependencies”. In particular:

  • miri doesn’t support calling C functions.
  • The sanitizers (especially mem-san and thread-san) can give false positives if you call into code which was not also compiled with the same sanitizer flags, so any C dependencies you use must themselves be compiled with the same sanitizer enabled (this is often tricky).

Run Tests Under miri

Miri is a great undefined behavior checker for Rust, that can tell you if you’re misusing unsafe in a whole bunch of cases, with the caveats that:

  • It cannot call into C code, so any native dependencies are out of the question.
  • It has some false positives, and can’t handle threading (or couldn’t last I checked).
    • Also, if you use a lot of libraries you’re likely to hit these false positives IME.
  • It can be very, very slow, even compared to tests run on a debug build.
  • I’m probably forgetting something, but it’s best for pure-rust library code.

However: it defines a #[cfg(miri)], which you may need to use to either disable tests or reduce the size of their input, etc.

jobs:
  # ...
  miri-test:
    name: Test with miri
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: nightly
          components: miri
      - run: cargo miri test --verbose --no-default-features
      - run: cargo miri test --verbose --all-features
      # ...

Run Tests Under Sanitizers

There are four sanitizers right now:

  • address-sanitizer (aka asan) detects a bunch memory safety errors, as well as some memory leaks.
  • memory-sanitizer (aka msan) detects uninitialized reads of memory
  • thread-sanitizer (aka tsan) detects data races. (Note: it doesn’t support std::sync::atomic::fence, although it can be emulated)
  • leak-sanitizer (aka lsan) also detects memory leaks, but possibly a different set of memory leaks than asan? No clue.

Anyway, you can run your code with them like:

jobs:
  # ...
  sanitizer-test:
    name: Test with -Zsanitizer=${{ matrix.sanitizer }}
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        sanitizer: [address, memory, thread, leak]
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: nightly
          components: rust-src
      - name: Test with sanitizer
        env:
          RUSTFLAGS: -Zsanitizer=${{ matrix.sanitizer }}
          RUSTDOCFLAGS: -Zsanitizer=${{ matrix.sanitizer }}
          # only needed by asan
          ASAN_OPTIONS: detect_stack_use_after_return=1
          # Asan's leak detection occasionally complains
          # about some small leaks if backtraces are captured,
          # so ensure they're not
          RUST_BACKTRACE: 0
      run: |
        cargo test -Zbuild-std --verbose --target=x86_64-unknown-linux-gnu --no-default-features
        cargo test -Zbuild-std --verbose --target=x86_64-unknown-linux-gnu --all-features
        # ...

Now, these do support use with native code (unlike miri)… But unless you compile that code with the same sanitizer enabled, they are either “worse” or “unusable”.

For example, you need to ensure: if you have -Zsanitizer=memory as a RUSTFLAG, any C code / C++ code is compiled with -fsanitize=memory. This is hard

Some sanitizers are more or less strict about this:

  • msan is almost entirely useless everything you call was compiled with msan.
  • asan works, but not as well. It shouldn’t produce false positives, though.
  • tsan can work so long as at least the code which does performs synchronization operations was compiled with tsan enabled.
  • lsan seems to be like asan but I honestly don’t know.

Linting and similar

Preventing Warnings with cargo check

jobs:
  # ...
  cargo-check:
    name: Check for warnings
    runs-on: ubuntu-latest
    env:
      RUSTFLAGS: -Dwarnings
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
      - run: cargo check --workspace --all-targets --verbose --no-default-features
      - run: cargo check --workspace --all-targets --verbose --all-features
      # ...

Linting with clippy

Note that clippy will run cargo check too, so there’s no need to do both clippy and cargo check

jobs:
  # ...
  clippy:
    name: Lint with Clippy
    runs-on: ubuntu-latest
    env:
      RUSTFLAGS: -Dwarnings
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          components: clippy
      - run: cargo clippy --workspace --all-targets --verbose --no-default-features
      - run: cargo clippy --workspace --all-targets --verbose --all-features
      # ...

Verifying Code Formatting with rustfmt

jobs:
  # ...
  rustfmt:
    name: Verify code formatting
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          components: rustfmt
      - run: cargo fmt --all -- --check

Verifies that if you do [SomeThing] or [MyFoo::bar] (et cetera) in a rustdoc comment, that that is resolvable (or, will be on docs.rs, which uses nightly and has intra-doc links on).

jobs:
  # ...
  check-rustdoc-links:
    name: Check intra-doc links
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: nightly
      # Important: This doesn't work for multi-crate workspaces, which
      # will require you call `cargo rustdoc -p` for each package! (See below)
      # (Also, note that this is `cargo rustdoc` and not `cargo doc`)
      - run: cargo rustdoc --all-features -- -D warnings

If you only have a few members in your workspace, it’s probably easiest to either have multiple lines like

- run: cargo rustdoc -p some_package --all-features -- -D warnings
- run: cargo rustdoc -p other_package --all-features -- -D warnings
# ... maybe just 1 or 2 more.

You can also use matrix for this of course. However, for big crates, you can also do something like this inside the run directive:

for package in $(cargo metadata --no-deps --format-version=1 | jq -r '.packages[] | .name'); do
  cargo rustdoc -p "$package" --all-features -- -D warnings
done

Code coverage

Coverage with cargo-tarpaulin

cargo-tarpaulin is nice because it works on stable rust and generates much better quality coverage info than anything else I’ve used.

Note that it cannot generate branch coverage though, and some people have issues with using it in docker in some cases. It also only works on linux, like a lot of these tools. Never been an issue for me so far, but it can have trouble with some native libraries, and isn’t quite as flexible as grcov.

jobs:
  # ...
  codecov-tarpaulin:
    name: Generate code coverage
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
      # Note: If you need to combine the coverage info of multiple
      # feature sets, you need a `.tarpaulin.toml` config file, see
      # the link above for those docs.
      - uses: actions-rs/tarpaulin@v0.1
        with:
          args: --all-features
      # Note: closed-source code needs to provide a token,
      # but open source code does not.
      - name: Upload to codecov.io
        uses: codecov/codecov-action@v1

Coverage with grcov

⚠️Major Caveat⚠️: While this works (as in, executes successfully and submits coverage to codecov.io), it generates really low quality coverage currently. Like, almost unusably bad. Locally, the coverage it generates is usually fine (but worse than tarpaulin), but I suspect I’m passing different arguments. The actions-rs/grcov doesn’t seem to support passing args, but grcov’s args can be specified in a config file instead, which you’ll almost certainly need to do.

grcov is a tool that can be used to combine the gcda/gcno files that are produced by code compiled with the -Zprofile (unstable) rust flag. It works on most machines, but IME generates lower quality coverage info than tarpaulin.

While it can generate branch coverage, in practice it’s of limited usefulness because of issues like https://github.com/mozilla/grcov/issues/476.

jobs:
  # ...
  codecov-grcov:
    name: Generate code coverage
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: nightly
      - name: Run test for coverage
        env:
          CARGO_INCREMENTAL: 0
          RUSTFLAGS: -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort
          RUSTDOCFLAGS: -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort
        run: |
          cargo test --workspace --all-features --verbose
          cargo test --workspace --no-default-features --verbose
          # If you need to test multiple feature sets, add
          # those tests calls here too
      - uses: actions-rs/grcov@v0.1
      # Note: closed-source code needs to provide a token,
      # but open source code does not.
      - name: Upload to codecov.io
        uses: codecov/codecov-action@v1

Other parts of the .yml file:

Top level env:

It’s often useful to put an env: block at the top level like this for global configuration:

env:
  RUST_LOG: info
  RUST_BACKTRACE: 1

Note that if you’re using sanitizers, having RUST_BACKTRACE on can cause false positives in memory leak detection if panics do occur. So, it can be good to explicitly disable it (which we do in the example for sanitizers above).

Event hooks (e.g. on:)

This controls when your CI runs.

I already gave you a decent default for getting started in the “Quick setup” section, but some other options are below.

I definitely recommend the official docs here, which have a lot of good examples.

Running CI Every Day (or Week)

on:
  # ... `pull_requests`/`push` here ...
  schedule:
    # Daily
    - cron: '0 0 * * *'
    # Or, if that's too frewuent, weekly
    # - cron: '0 0 * * FRI'

So, I’ll start this by saying: take care of your mental health, and if this sounds like something that will stress you out/contribute to burnout/etc don’t do it.

With that caveat in mind, I recommend4 running your CI automatically on a schedule. This helps you flakey tests more quickly, but also lets you know if your project needs a rustfmt/lint fix (e.g. after a new Rust release), which can be discouraging for new contributors5.

I usually set this to run once per day, but if that sounds too frequent (if you only do Rust on the weekend, for example), weekly sounds reasonable too, and is given as an example.

The syntax of the cron entries is cron syntax, and has good documentation here: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events. Note that it doesn’t support the @foo (e.g. @daily) extension you see in some places, like https://crontab.guru/ (which is still helpful so long as you know to avoid that).

Run on: Pushes to main + PRs to main or dev

on:
  # If your default branch is not named `main`, you would
  # adjust this to match.
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
      - dev

It’s common to want to filter this, and you can get very fancy here (globing, exclusions, …). The section on pull_request and push in the official docs is great and has tons more examples.

Conclusion

It’s done…ish! That took forever.

Honestly in retrospect it feels a little silly for these to be on some dude’s personal blog and not in a public github repo containing a mdbook or whatever, but yeah. I’ll try to get to that later, and will post about it if I do

Also apologies in advance if I messed any of these up in-between testing them and putting them into the blog. It seems plausible that I did in some cases, let me know if so.

Anyway, I hope this helps someone in some way!

Appendix

Basic Rust Starter Template

This is a decent starting point for a lot of Rust projects.

Includes:

  • Tests on reasonably common platforms: stable/beta/nightly on linux and stable on macos/win64/win32
  • Linting via clippy
  • Format checking via rustfmt.

In specific it doesn’t have most of the parts that get crates in trouble, however some users may need to disable tests on e.g. windows, but at least they’ll be doing that deliberately. It can also easily be extended to meet the involved cases.

name: CI

on:
  pull_request:
  push:
    branches:
      # You probably don't need both of these.
      - main
      - master

env:
  RUST_BACKTRACE: 1

jobs:
  test:
    name: Test Rust ${{ matrix.rust }} on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - { rust: stable,           os: ubuntu-latest }
          - { rust: stable,           os: macos-latest }
          - { rust: stable,           os: windows-latest }
          - { rust: stable-i686-msvc, os: windows-latest }
          - { rust: beta,             os: ubuntu-latest }
          - { rust: nightly,          os: ubuntu-latest }
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          rust-version: ${{ matrix.rust }}
      - run: cargo test --verbose --workspace
      - run: cargo test --verbose --workspace --all-features
      - run: cargo test --verbose --workspace --no-default-features

  clippy:
    name: Lint with clippy
    runs-on: ubuntu-latest
    env:
      RUSTFLAGS: -Dwarnings
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          components: clippy
      - run: cargo clippy --workspace --all-targets --verbose
      - run: cargo clippy --workspace --all-targets --verbose --no-default-features
      - run: cargo clippy --workspace --all-targets --verbose --all-features

  rustfmt:
    name: Verify code formatting
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: hecrj/setup-rust-action@v1
        with:
          components: rustfmt
      - run: cargo fmt --all -- --check

  1. Yep… First I was going to go in-depth into what I think a good setup for uses like mine are, and painstakingly walk you through why I like to do it in the specific way I do… Unfortunately, this is extremely time consuming for me, boring for you, and not very useful.

    … Then I thought about doing a cookbook with mdbook or w/e and put it on github, but… Given that I’d like to get a blog post up this weekend… Deciding that “instead of a blog post, why not write a small book” sounds like it would totally kill any chance I have of doing a blog post.

    So. Here we are. ↩

  2. Yes, right in the thick of it! ↩

  3. Especially if I’m not super convinced of those benefits, like with using actions-rs/cargo over e.g. run: cargo6. ↩

  4. For now anyway… Perhaps eventually there will be a day of reckoning, where a new version of rustfmt or clippy causes all my projects to break at once, drowning me in CI failure emails… ↩

  5. Submitting a PR and having it fail for a lint/fmt error in some file you didn’t touch can be rough, especially if you’re new. This gets worse the larger and more daunting the project is. ↩

  6. To be clear: That’s not intended to be a dig at actions-rs, actions-rs/cargo, or you if you like or use actions-rs/cargo… I like a lot of the projects under actions-rs, and a lot of this is my own personal preference and such. ↩

You May Also Enjoy

Miri Feature Wishlist

8 minute read

Miri is an evaluator for rust that detects many kinds of undefined behavior. Here are some things it could (possibly) do that would catch bugs I’ve had/seen before.