Bits - Quick Start Guide

Bits is a build orchestration tool for complex software stacks. It fetches sources, resolves dependencies, and builds packages in a reproducible, parallel environment.

Beyond building, bits covers the path to deployment:

• Binary reuse from a content-addressed S3 store, trusted through signed manifests (bits certify / bits sign).
• CVMFS publishing where each package is published once per build architecture and releases are views over it (symlink releases plus an LCG-style merged view with a self-locating setup.sh).
• Supply chain: deterministic tarballs, CycloneDX/SPDX SBOMs (bits sbom), and source/patch checksums with git commit pins (bits checksums).
• Per-directory profiles (bits use) so repeated options stay out of every command line.

Full documentation is available in docs/USERGUIDE.md, docs/COOKBOOK.md, and docs/REFERENCE.md. This guide covers only the essentials.

git clone https://github.com/bitsorg/bits.git
cd bits
export PATH=$PWD:$PATH          # add bits to your PATH
python -m venv .venv
source .venv/bin/activate
pip install -e .                # install Python dependencies

Requirements: Python 3.8+, git, and Environment Modules (modulecmd).
On macOS: brew install modules. The first build on macOS records the Homebrew packages the stack needs in sw/<arch>/Brewfile and stops until you run brew bundle --file sw/<arch>/Brewfile (or re-run with --brew to install them on demand; bits brew regenerates the file).
On Debian/Ubuntu: apt-get install environment-modules
On RHEL/CentOS: yum install environment-modules

bits builds from a community recipe repository: a *.bits repository with the community's defaults, CVMFS layout and recipes. Clone one and run bits inside it (bits uses the current directory as its recipe directory):

git clone https://github.com/bitsorg/stacks.bits && cd stacks.bits
#   or: bits init stacks.bits && cd stacks.bits   (resolved in the bits-providers registry)
bits build --defaults gcc15 ROOT

# Enter the built environment and run
bits enter ROOT/latest
root -b
exit

Community repositories: stacks.bits (LCG-based stacks), alice.bits, atlas.bits, cms.bits, key4hep.bits, lhcb.bits, ship.bits. They pull shared recipe pools (lcg.bits, common.bits, alidist.bits) on demand; bits init <name>.bits clones a community repository from the bits-providers registry. ALICE's classic workflow keeps working through the aliBuild wrapper: aliBuild init checks out alidist, then aliBuild build O2.

bits doctor                  # is this machine set up to run bits? (Python modules, git, compiler, container engine, disk, stores)
bits doctor ROOT             # are ROOT's system requirements satisfied?
bits build --dry-run ROOT    # per-package plan: installed, local tarball, remote store, reuse overlay or build
bits build --parallel 4 ROOT # build up to 4 independent packages at once (--builders is an alias)

Two admin/CI groups act on shared infrastructure: bits store (ls, rm, verify, cat, gc, stats, upload) manages the S3 binary store, and bits cvmfs (platforms, show, summary, stage, publish) inspects a deployed CVMFS tree and drives producer-side publishing. bits publish publishes packages to CVMFS (bare bits publish bulk-uploads the latest build manifest to the S3 store); single-package S3 uploads are bits store upload. The old spellings bits store-stats, bits cvmfs-stage and bits cvmfs-publish still work but are deprecated.

Full command reference

Record per-directory settings with bits use, so you do not repeat them on every command. Options after a section name apply to that command; without one they go to [common], applied to every architecture-aware command:

bits use --architecture x86_64-el9-gcc14-opt
bits use build --work-dir /path/to/sw --remote-store https://mybucket/builds --docker

The profile is stored in ./.bitsuse (or under ~/.bits/use/ when the current directory is not writeable). Each bits use SECTION … replaces that section; bits use alone shows the active profile, bits use --clear [SECTION] removes it and bits use --help lists the forms. A .bitsuse file is only honoured when it is owned by you. Profiles replace the retired bits.rc file.

bits init with configuration options and no package (e.g. bits init --work-dir /path/to/sw --remote-store URL) writes the same profile: --architecture to [common]; the store, defaults, config-dir, work-dir and reference-sources options to [build], replacing those sections.

Global settings come from environment variables:

$BITS_ORGANISATION is set uppercase (ALICE, LHCB, …). Bits lowercases it internally when resolving the community recipe repository from bits-providers (e.g. LHCB → lhcb.bits.sh → https://github.com/bitsorg/lhcb.bits). Normally you do not need it: check out the community repository and run bits inside it.

Configuration details

bits keeps the tool, the policy, and the recipes in separate versioned repositories. The registry bits-providers maps a community name to its repo; a community/policy repo such as stacks.bits sets defaults and CVMFS layout and requires: a shared recipe pool like lcg.bits (~1100 recipes). Any recipe with provides_repository: true is cloned on demand and added to the search path, so a build pulls in the pools it needs automatically — and because the binary store is content-addressed, matching artifacts are reused across communities.

# Build an LCG-stack package: run from the community repo; bits auto-pulls lcg.bits
git clone https://github.com/bitsorg/stacks.bits && cd stacks.bits
bits build ROOT --defaults gcc15

The entry point is defaults-release.sh (composition: stacks.bits → defaults-release.sh → lcg.bits). For the full model see bits-providers and each *.bits repository (e.g. stacks.bits, alice.bits).

Create a file <package>.sh inside a *.bits directory with:

package: mylib
version: "1.0"
source: https://github.com/example/mylib.git
tag: v1.0
requires:
  - zlib
---
./configure --prefix="$INSTALLROOT"
make -j${JOBS:-1}
make install

Complete recipe reference

bits clean                # remove temporary build directories
bits clean --aggressive-cleanup   # also remove source mirrors and tarballs

# Persistent workDir cache management (evict old / low-disk-space packages)
bits prune --max-age 14           # evict packages not used in the last 14 days
bits prune --min-free 100         # free space until at least 100 GiB available
bits prune -n                     # dry-run: show what would be removed

Cleaning options

# Build inside a Docker container for a specific Linux version
bits build --docker --architecture ubuntu2004_x86-64 ROOT

# Cross-compile for ARM64 on an x86-64 host (requires QEMU binfmt handlers)
bits build --docker --architecture slc9_aarch64 MyAnalysis

# Use a remote binary store (S3, HTTP, rsync) to share pre-built artifacts
bits build --remote-store s3://mybucket/builds ROOT

# Read from and upload to the same writable store
bits build --remote-store s3://mybucket/builds::rw ROOT

--write-store URL alone only uploads; it is also the read store where no default remote store applies. --store is a deprecated spelling of --remote-store on bits publish, certify, sign, compliance, prune and bits store upload (not on bits build).

The --cvmfs-prefix flag (which embeds the final CVMFS deployment path at compile time so no relocation is needed at publish time) and bits publish --no-relocate are used by the bits-console-triggered CI pipeline on the build runners — they are not normally typed by end users. See WORKFLOWS.md Phase 4 for the user-facing workflow and docs/REFERENCE.md §22 for the flag reference.

Docker support | Cross-compilation via QEMU | Remote stores

# Check this machine is set up for bits (Python modules, git, compiler, container engine, stores)
bits doctor

# On a CI build runner: Docker daemon, podman, QEMU binfmt, CVMFS mounts, disk, store, prepub
bits doctor --runner --cvmfs-repos /cvmfs/alice.cern.ch [--prepub-url URL] [--json]

# Verify that a live CVMFS deployment matches a recorded build manifest
bits verify --from-manifest alice-o2-20260411.json \
            --cvmfs-root /cvmfs/alice.cern.ch

bits doctor reference | bits verify reference

# Make a build trusted for binary reuse (upload missing tarballs, passkey approval, certification MR)
bits certify

# Publish a release to CVMFS as a view over packages published once per architecture
bits publish --release-view MyStack

# Software Bill of Materials of a build (CycloneDX 1.6 + SPDX 2.3)
bits sbom sw/MANIFESTS/bits-manifest-<build>.json -o sbom/

# Check (and with --write, record) checksums and git commit pins of a recipe repository
bits checksums -c lcg.bits

A community's CVMFS layout comes from its defaults: cvmfs_packages_template places each package once per build architecture (skipped when the same build hash is already published) and cvmfs_views_template adds a merged bin/ lib/ include/ … view with a self-locating setup.sh; templates accept {arch} and {day} tokens. --view is the deprecated spelling of --release-view.

Publishing and certifying | bits sbom | bits checksums

git clone https://github.com/bitsorg/bits.git
cd bits
python -m venv .venv
source .venv/bin/activate
pip install -e .[test]

# Run tests
tox                     # full suite on Linux
tox -e darwin           # reduced suite on macOS
pytest                  # fast unit tests only

Developer guide

bits uses a single toolchain from your laptop to experiment-wide CVMFS. Clone a package source next to your recipe checkout and bits detects it automatically, building your local version while resolving all other dependencies from the shared recipe repo. Once tested locally, the change follows an unbroken path: commit → recipe MR → CI build → bits publish → CVMFS. Group admins publish full experiment stacks; individual users can publish single packages to a separate namespace — both paths use the same commands and the same recipes.

See WORKFLOWS.md for the full phase-by-phase walkthrough and workflow diagram.

• User Guide — installation, configuration, building, environments, cleaning up
• Cookbook — practical recipes for common tasks
• Reference Manual — command-line flags, recipe format, environment variables, Docker, stores, CVMFS pipeline, developer guide
• Workflows — development-to-deployment walkthrough and diagram
• Roadmap — planned features and priorities

Note: Bits is under active development. For the most up-to-date information, see the full docs/REFERENCE.md.

The bits ecosystem spans several repositories under two licenses, chosen by provenance rather than preference.

bits and its recipe repositories descend from ALICE's aliBuild and alidist, both licensed under GPL-3.0. Under the GPL's copyleft, these derivative works must remain GPL-3.0-or-later.

The newer services written from scratch for the CVMFS publish chain — the Go publisher (cvmfs-bits / cvmfs-prepub) and its deployment example (cvmfs-testbed) — contain no aliBuild code, so they use the permissive Apache-2.0 license. Apache-2.0 is one-way compatible into GPL-licensed combinations, so these components can still be combined with the GPL parts.

Each licensed source file carries an SPDX-License-Identifier header (Python modules and CLI scripts in bits; Go in cvmfs-bits; JS/config in bits-console; shell/compose in cvmfs-testbed). Two deliberate exceptions keep build hashes and generated output stable, and are governed by their repository-level LICENSE/COPYRIGHT instead:

• the bits-recipe-tools recipe snippets (sourced and hashed by bits); and
• the bits build harness sourced into per-package builds or copied into tarballs (bits_helpers/build_template.sh, tar_template.sh, relocate-me.sh) and the Jinja scaffolding templates (templates/*.jnj).

The recipe repositories (lcg.bits, common.bits, stacks.bits) and the bits-providers data repository are likewise covered by their repository-level LICENSE/COPYRIGHT only — recipes are content-addressed, so per-file headers are omitted to keep their hashes stable.

Copyright (C) CERN and the bits project contributors. Work produced by CERN personnel is owned by CERN; please involve CERN Knowledge Transfer before changing any license.

Contributions are accepted under the Developer Certificate of Origin (DCO): sign off your commits with git commit -s.