Skip to main content

How to make new xPack LLVM clang releases

GitHub package.json version NPM Version GitHub issues GitHub pulls licence

This page is designed for maintainers of the xPack LLVM clang project and provides documentation on creating new releases.

Prerequisites​

@xpack-dev-tools/clang is an xpm package that packages platform-specific archives, built with the xPack Build Box (XBB).

The build scripts execute on GNU/Linux and macOS. The Windows binaries are compiled on x64 GNU/Linux, utilising mingw-w64.

For details on installing the prerequisites, please read the Build Prerequisites page.

Obtain project sources​

The project is hosted on GitHub:

Branches

This project utilises multiple branches:

  • master, not actively used
  • xpack, containing the latest stable version (default)
  • xpack-development, containing the current development version
  • website, containing the current website content; pushes to this branch automatically trigger publication of the main website
  • development, containing the current preview website content; pushes to this branch automatically trigger publication of the preview website

All development is conducted in the xpack-development branch, and contributions via Pull Requests should be directed to this branch.

When new releases are published, the xpack-development branch is merged into xpack.

To clone the stable branch (xpack), execute the following commands in a terminal (on Windows use the Git Bash console):

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack \
  https://github.com/xpack-dev-tools/clang-xpack.git \
  ~/Work/xpack-dev-tools/clang-xpack.git
For development purposes, clone the xpack-development branch.
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack-development \
  https://github.com/xpack-dev-tools/clang-xpack.git \
  ~/Work/xpack-dev-tools/clang-xpack.git

Alternatively, if the repository has already been cloned:

git -C ~/Work/xpack-dev-tools/clang-xpack.git pull
tip

To contribute Pull Requests, fork the project and ensure the Copy the master branch only is disabled.

Utilise the xpack-development branch and ensure you contribute the Pull Requests back to the xpack-development branch.

During development, it is convenient to maintain a writable instance of the library to enable changes in parallel with the parent project.

To facilitate the use of a writable instance of this library in other projects, add a link from the user's global xPacks store to this local development folder:

xpm link -C ~/Work/xpack-dev-tools/clang-xpack.git

And in the projects referring it:

xpm link @xpack-dev-tools/clang
Obtain the writable helper sources (optional, for development purposes)

The project has a dependency on a common helper, which is normally installed as a read-only dependency; for development purposes, to enable modifications to the scripts located within the helper, clone the xpack-development branch and link it to the user's global xPacks store:

rm -rf ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack-development \
  https://github.com/xpack-dev-tools/xbb-helper-xpack.git \
  ~/Work/xpack-dev-tools/xbb-helper-xpack.git

Alternatively, if the repository has already been cloned:

git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull

If a writable instance of this library is required in another project, add a link from the user's global xPacks store to it:

xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git

Other repositories in use are:

Other projects referred are:

Release schedule​

This distribution generally follows the official LLVM clang releases, which are 2 major releases per year.

Given that between the major releases there are 5-8 intermediate releases (spaced 2 weeks apart), only the final patch of each minor version is retained (like 20.1.[5678]). The rule is to wait for the new upstream release (like 21.1.0), and release the previous one (20.1.[5678])

However, in order to spot possible issues, it is recommended to run builds as soon as new versions are available (like 20.1.1), without making public releases.

Prepare a new release​

Before starting the build, perform some checks and adjustments.

Download the build scripts​

The build scripts are available in the build-assets/scripts folder of the xpack-dev-tools/clang-xpack Git repository.

To download them on a new machine, clone the xpack-development branch, as described above.

Check Git​

In the xpack-dev-tools/clang-xpack Git repository:

  • Switch to the xpack-development branch
  • Pull new changes
  • If necessary, merge the xpack branch
  • If necessary, merge the website branch
caution

This is critically important; otherwise you will release the previous content again!

The xpack branch should remain unchanged since the previous release and will be updated when the new release is published.

Update helper & other dependencies​

Check the latest versions at https://github.com/xpack-dev-tools/ and update the dependencies in build-assets/package.json.

Generate the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/clang-xpack.git

Commit the top changes​

  • Stage GitHub workflows and package.json files (top and from build-assets)
  • Commit with the message re-generate top commons

Generate the website commons​

Execute the generate-website-commons npm script from website/package.json within the project folder.

npm run generate-website-commons -C ~/Work/xpack-dev-tools/clang-xpack.git/website

Commit the website changes​

  • Stage the website folder and top README.md
  • Commit with the message re-generate website commons

Check the latest upstream release​

The latest releases must have been announced in LLVM Announcements and LLVM GitHub Releases. Find the latest release that seems stable, usually like 20.1.6, sometimes 20.1.7, even 20.1.8.

Compare the latest upstream release with the latest xPack release. If necessary, update the triplet, otherwise increase the fourth number, as below.

Increase the version and update the top package.json​

As required by npm projects, this one also utilises SemVer.

  • Determine the next version (such as 20.1.8-1.1)
  • Update the version in the top package.json file
  • Use the new version, suffixed by .pre, such as 20.1.8-1.1.pre.

Update VERSION​

Update the build-assets/scripts/VERSION file; the format is 20.1.8-1. The fourth number is the xPack release number of this version. A fifth number will be added to the version in package.json when the package is published to the npm server.

Update the websiteConfig (if necessary)​

Check and possibly update the release-specific properties (like the release date) within website/package.json.

Update the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/clang-xpack.git

Update the website commons​

Execute the generate-website-commons npm script from website/package.json within the project folder.

npm run generate-website-commons -C ~/Work/xpack-dev-tools/clang-xpack.git/website

Start the local web server​

Execute the npm script clear followed by start in the website sub-project, or execute the following within the project folder:

npm run clear -C ~/Work/xpack-dev-tools/clang-xpack.git/website
npm run start -C ~/Work/xpack-dev-tools/clang-xpack.git/website

Navigate to the Maintainer Info page, the Start the local web server section.

Update the version specific code​

  • open the build-assets/scripts/versioning.sh file
  • add a new if with the new version before the existing code

Update Windows specifics​

Identify the release and the tag at:

rm -rf ~/Work/mstorsjo/llvm-mingw.git
mkdir -pv ~/Work/mstorsjo
git clone https://github.com/mstorsjo/llvm-mingw ~/Work/mstorsjo/llvm-mingw.git

git -C ~/Work/mstorsjo/llvm-mingw.git checkout undefined

Compare with the previous release. If the differences are small, possibly make the necessary adjustments and proceed.

If the changes are major, run the llvm-mingw build as documented in the README-DEVELOP-MSTORSJO to understand the extend of the changes.

Check if there are changes in the tests, and update.

Check if there are changes in the wrappers, and update.

Fix possible open issues​

Check GitHub Issues and Pull Requests:

and resolve them; assign them to a milestone (such as 20.1.8-1.1).

Update CHANGELOG.md​

  • Open the CHANGELOG.md file
  • Check if all previously fixed issues are included
  • Check the latest commits npm run git-log; if necessary, copy/paste lines, group by dates and edit them using the regular expressions below
  • To convert the dates into headings, change from: 


    ([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]) [*]
    to: 


    ## $1

    *
  • To remove the remaining dates, change from: 
    ^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [*]
    to: 
    *
  • Add a new entry such as * v20.1.8-1 prepared
  • Commit with the message prepare v20.1.8-1

Push the changes to the xpack-development branch​

Ensure that the latest commit that updates VERSION is pushed to the xpack-development branch!

caution

This is critically important; otherwise the next steps will build the previous release again!

Execute the development builds​

The builds currently execute on 4 dedicated machines (x64 GNU/Linux, x64 macOS, arm64 macOS, arm64 GNU/Linux).

Before the actual build, execute test/development builds on all platforms.

Bootstrap​

For Windows, it is necessary to build a bootstrap; for a better control, it is recommended to do a first run only to build and test only the bootstrap; for this un-comment a line in application.sh:

XBB_APPLICATION_BOOTSTRAP_ONLY="y"

After the bootstrap passes the tests, comment out this line and repeat to build the final binaries.

Visual Studio Code extension​

All actions are defined as xpm/xPack actions and can be conveniently triggered via the VS Code graphical interface, utilising the xPack C/C++ Managed Build Tools extension.

Patches​

Starting with clang 18 there are no more patches.

Restart the Docker daemons​

If, for any reason, the Docker builds need to be cancelled, it is very likely that some processes will continue to execute in the background.

To completely terminate them, and ensure that Docker executes from a clean slate, restart the Docker daemons on all GNU/Linux machines:

sudo systemctl restart docker

Verify if both projects are pushed​

Verify that both the current project and the helper are on the xpack-development branch and the latest commits are pushed to GitHub.

caution

This is particularly crucial for large projects; otherwise, the build may proceed with the previous version.

Build the x64 macOS binaries​

For x64 macOS, first execute the build on the development machine (wksi, a recent macOS):

export XBB_ENVIRONMENT_SKIP_CHECKS="y"
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull

xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git

xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

xpm run deep-clean --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm install --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run build-development --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

For a debug build:

xpm run build-development-debug --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

The build takes about 100 minutes (1h40m) to complete.

When functional, push the xpack-development branch to GitHub.

Execute the native build on the production machine (xbbmi, an older macOS); start a VS Code remote session, or connect with a terminal:

caffeinate ssh xbbmi

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To update the build scripts and build the development binaries:

export XBB_ENVIRONMENT_MACOSX_DEPLOYMENT_TARGET='11.0' && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config darwin-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm install --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run build-development --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

About 107 minutes (1h47m) later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/darwin-x64/deploy folder:

  • xpack-clang-20.1.8-1-darwin-x64.tar.gz
  • xpack-clang-20.1.8-1-darwin-x64.tar.gz.sha

To update the build scripts and build the release binaries:

export XBB_ENVIRONMENT_MACOSX_DEPLOYMENT_TARGET='11.0' && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config darwin-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm install --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run build --config darwin-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

Build the ARM64 macOS binaries​

Execute the native build on the production machine (xbbma, an older macOS); start a VS Code remote session, or connect with a terminal:

caffeinate ssh xbbma

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To update the build scripts and build the development binaries:

export XBB_ENVIRONMENT_MACOSX_DEPLOYMENT_TARGET='11.0' && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config darwin-arm64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm install --config darwin-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run build-development --config darwin-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

About 44 minutes later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/darwin-arm64/deploy folder:

  • xpack-clang-20.1.8-1-darwin-arm64.tar.gz
  • xpack-clang-20.1.8-1-darwin-arm64.tar.gz.sha

To update the build scripts and build the release binaries:

export XBB_ENVIRONMENT_MACOSX_DEPLOYMENT_TARGET='11.0' && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config darwin-arm64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm install --config darwin-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run build --config darwin-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

Build the x64 GNU/Linux binaries​

Execute the Docker build on the production machine (xbbli); start a VS Code remote session, or connect with a terminal:

caffeinate ssh xbbli

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To update the build scripts and build the development binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config linux-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-link-deps --config linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

About 125 minutes (2h05m) later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/linux-x64/deploy folder:

  • xpack-clang-20.1.8-1-linux-x64.tar.gz
  • xpack-clang-20.1.8-1-linux-x64.tar.gz.sha

To update the build scripts and build the release binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config linux-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

Build the ARM64 GNU/Linux binaries​

Execute the Docker build on the production machine (ampere); start a VS Code remote session, or connect with a terminal:

caffeinate ssh ampere

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To update the build scripts and build the development binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config linux-arm64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-link-deps --config linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

About 178 minutes (2h58m) later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/linux-arm64/deploy folder:

  • xpack-clang-20.1.8-1-linux-arm64.tar.gz
  • xpack-clang-20.1.8-1-linux-arm64.tar.gz.sha

To update the build scripts and build the release binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config linux-arm64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

Build the x64 Windows binaries​

Execute the Docker build on the production machine (xbbli); start a VS Code remote session, or connect with a terminal:

caffeinate ssh xbbli

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

To update the build scripts and build the development binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config win32-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config win32-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-link-deps --config win32-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config win32-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

About 237 minutes (3h57m) later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/win32-x64/deploy folder:

  • xpack-clang-20.1.8-1-win32-x64.zip
  • xpack-clang-20.1.8-1-win32-x64.zip.sha

To update the build scripts and build the release binaries:

rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/clang-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/clang-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
\
xpm run deep-clean --config win32-x64  -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-prepare --config win32-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets && \
xpm run docker-build-development --config win32-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

Update the durations of the development builds​

In website/docs/_project/_development-durations.mdx, update the durations of the builds.

Commit with the message website: update development durations

Update the trees in the Installation Guide page​

Copy the trees displayed at the end of the test builds and paste to the Folder hierarchy section within the web file:

  • website/docs/_project/_folders-hierarchies-linux.mdx
  • website/docs/_project/_folders-hierarchies-macos.mdx
  • website/docs/_project/_folders-hierarchies-windows.mdx

If present, replace the actual version (such as 20.1.8-1.1) with the following expression:

${customField('releaseVersion')}
tip

macOS does not have a tree command; either compile it from sources, or install it via Homebrew, and create a link to /usr/local/bin.

Commit with the message website: update folder hierarchies

Copy and paste the complete list of links displayed at the end of the build, in sequence, for each platform (Windows, macOS, GNU/Linux), and check the differences compared to the repository.

Commit if necessary with the message package.json: update executables links

How to build a debug version​

In some cases it is necessary to execute a debug session with the binaries.

For these cases, the build script accepts the --debug options.

There are also xpm actions that utilise this option (build-development-debug and docker-build-development-debug).

Files cache​

The XBB build scripts utilise a local cache such that files are downloaded only during the initial execution, with subsequent executions being able to utilise the cached files.

However, occasionally some servers may not be available, and the builds may fail.

The workaround is to manually download the files from alternative locations (such as https://github.com/xpack-dev-tools/files-cache/tree/master/libs), place them in the XBB cache (Work/cache) and restart the build.

Execute the production builds​

The automation is provided by GitHub Actions and three self-hosted runners (the GitHub hosted runners do not have the necessary resources).

Start the self-hosted runners​

If the runners have expired (more than 2 weeks since last use), execute the github-runner-configure.sh script to reconfigure them. This script is available in the xbb-helper and requires GITHUB_API_XPACK_DEV_TOOLS_RUNNERS_TOKEN to be present.

  • On the development machine (wksi) open SSH sessions to the build machines (xbbli, xbbmi, xbbma, ampere):
caffeinate ssh xbbli
caffeinate ssh xbbmi
caffeinate ssh xbbma
caffeinate ssh ampere

Create a screen session, to provide a persistent standard output for the runners:

screen -S ga

To exit it, use # Ctrl-a Ctrl-d.

For xbbli & ampere, which have additional memory, start two runners:

~/actions-runners/xpack-dev-tools/1/run.sh &
~/actions-runners/xpack-dev-tools/2/run.sh &

For the macOS x64 runners, ensure the deployment target is defined:

unset WORK_FOLDER_PATH
export XBB_ENVIRONMENT_MACOSX_DEPLOYMENT_TARGET="11.0"
~/actions-runners/xpack-dev-tools/run.sh &

On all other machines, start the runner with:

~/actions-runners/xpack-dev-tools/run.sh &

To check the status of the runners locally:

ps -a | grep 'Runner.Listener'

To terminate them:

killall Runner.Listener

Check the status of the runners​

The status of all self-hosted runners is available at the GitHub Runners page.

Publish the helper​

Publish a new release of the helper on npmjs.

In the npm packages helper, update the dependency to the new helper​

In templates/_xpack-dev-tools/build-assets/package-merge-liquid.json, update the reference to "@xpack-dev-tools/xbb-helper":.

Generate the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/clang-xpack.git

Commit the top changes​

  • Stage GitHub workflows and package.json files (top and from build-assets)
  • Commit with the message re-generate top commons

Check for disk space​

Verify that the build machines have sufficient free space and perform cleanup operations as necessary:

xpm run check-space -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

This is equivalent to executing a df command.

df -gH /

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

This is equivalent to executing a comprehensive rm to remove the build folders:

rm -rf ~/Work/xpack-dev-tools/*/build-assets/build

Commit and push​

  • Push the xpack-development branch to GitHub
caution

This is particularly crucial for large projects; otherwise, the build may proceed with the previous version.

Verify if the helper is pushed and published​

The helper xpack branch must be updated and the npm package published.

Manually trigger the build GitHub Actions​

To trigger the GitHub Actions builds, utilise the xpm actions:

  • trigger-workflow-build-darwin-x64
  • trigger-workflow-build-darwin-arm64
  • trigger-workflow-build-linux-x64
  • trigger-workflow-build-linux-arm64
  • trigger-workflow-build-win32-x64

The scripts behind these actions require the GITHUB_API_DISPATCH_TOKEN variable to be present in the environment, and the organisation PUBLISH_TOKEN to be visible in the Settings → Action → Secrets page.

These commands utilise the xpack-development branch of this repository.

Durations & resulting binaries​

The builds take 3h30m hours to complete:

  • darwin-x64: 1h58m (nuc)
  • darwin-arm64: 0h49m (m1)
  • linux-x64: 2h22m (amd)
  • linux-arm64: 3h11m (ampere)
  • win32-x64: 3h23m (amd)

The workflow results and logs are available from the GitHub Actions page.

Update the values in website/docs/_project/_github-actions-durations.mdx with those shown by GitHub Actions.

Commit with the message website: update actions durations

The resulting binaries are available for testing from pre-releases/test.

Execute the tests​

Automated tests​

The automation is provided by GitHub Actions on GitHub-hosted runners.

To trigger the GitHub Actions tests, execute the xpm actions:

  • trigger-workflow-test-prime
  • trigger-workflow-test-docker-linux-x64
  • trigger-workflow-test-docker-linux-arm64
xpm run trigger-workflow-test-prime -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets
xpm run trigger-workflow-test-docker-linux-x64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets
xpm run trigger-workflow-test-docker-linux-arm64 -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

The scripts behind these actions require the GITHUB_API_DISPATCH_TOKEN variable to be present in the environment.

These actions utilise the xpack-development branch of this repository and the pre-releases/test binaries.

The test results are available from the GitHub Actions page.

After executing the prime tests, download the test results and compare with previous runs.

Manual tests​

To download the pre-released archive for the specific platform and execute the tests, utilise:

git -C ~/Work/xpack-dev-tools/clang-xpack.git pull
xpm run install -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets
xpm run test-pre-release -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

For additional tests, on each platform, download the archive from pre-releases/test and check the binaries.

On macOS, remove the com.apple.quarantine flag:

xattr -cr ${HOME}/Downloads/xpack-*

Running the tests with the system binaries​

To calibrate the tests with the system compiler:

bash ~/Work/xpack-dev-tools/clang-xpack.git/scripts/test.sh --system

On Ubuntu, to install clang and libraries, use:

sudo apt-get install --yes clang lld libc++-dev libc++abi-dev

On Manjaro, to install clang and libraries, use:

pacman --sync clang lld libc++ libc++abi

On Linux x64, the 32-bit clang libraries are not available, skip the tests:

bash ~/Work/xpack-dev-tools/clang-xpack.git/scripts/test.sh --system --skip-32

Publish the release​

Create a new GitHub pre-release​

  • In CHANGELOG.md, add the release date and a message such as * v20.1.8-1 released
  • Commit with the message CHANGELOG update
  • Check and possibly update the build-assets/templates/body-github-release-liquid.mdx
  • Push the xpack-development branch to GitHub
  • Execute the trigger-workflow-publish-release xpm action: 
    xpm run trigger-workflow-publish-release -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

The workflow result and logs are available from the GitHub Actions page.

The result is a draft pre-release tagged such as v20.1.8-1 (mind the dash in the middle!) and named such as xPack LLVM clang v20.1.8-1 (mind the dash), with all binaries attached.

  • Edit the draft and ensure it is attached to the xpack-development branch (important!)
  • Perform the final edits (maintenance vs. new release) and check if everything is satisfactory
  • Keep the pre-release button enabled
  • Do not enable Discussions yet
  • Click the Publish release button
info

At this moment the system should send an email notification to all clients watching the clang-xpack project.

Prepare a new blog post​

  • Check and possibly update the website/blog/_templates/blog-post*-release-liquid.mdx
  • Execute the generate-website-blog-post npm script; this will add a file to the website/blog folder: 
    npm run generate-website-blog-post -C ~/Work/xpack-dev-tools/clang-xpack.git/website
  • Fix the TODO in the front matter description property (select maintenance vs. new release)
  • Fix the TODO in the summary (select maintenance vs. new release)
  • Add entries in the Bug fixes; utilise the following syntax 
    - [[#N](https://github.com/xpack-dev-tools/clang-xpack/issues/N)]:
  • Copy/paste the folder hierarchies from the website/docs/_project/_folders-hierarchies-*.mdx files
  • Commit with the message website: blog post release 20.1.8-1.1 published
  • Open the top package.json file
  • Check if the links in the xpack.executables property cover the actual binaries
  • If necessary, also check on Windows

Update the package.json list of binaries​

  • Select the xpack-development branch
  • Ensure that the binaries were attached to the pre-release
  • Execute the update-package-binaries xpm action: 
    xpm run update-package-binaries -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets
  • Open the top package.json file
  • Check the baseUrl: it should match the file URLs (including the tag/version); no terminating / is required
  • From the release, check the SHA & file names
  • Compare the SHA sums with those shown by cat *.sha
  • Check the executable names
  • Commit all changes with the message package.json: update URLs for 20.1.8-1.1 release

Prepare the release​

  • Switch to the xpack-development branch
  • If there are changes, commit all changes to the xpack-development branch
  • Check the latest commits npm run git-log
  • Update CHANGELOG.md, add a line such as * v20.1.8-1.1 published on npmjs.com
  • Commit with the message CHANGELOG: publish npm v20.1.8-1.1
  • Check the list of files included in the npm package: 
    npm run npm-pack
  • Possibly update .npmignore

Set the new version​

Execute the following within the project folder:

npm version 20.1.8-1.1

The postversion npm script should also update tags via git push origin --tags; this should also trigger CI tests.

Publish the release to npmjs.com​

Execute the following command within the project folder:

npm publish --tag next
tip

When publishing for the first time, utilise --access public.

After a few moments, the version will be visible at npmjs.com, under the Versions tab.

Test if the binaries can be installed with xpm​

Execute the trigger-workflow-test-xpm xpm action:

xpm run trigger-workflow-test-xpm -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

This will test if the package can be installed via xpm install on all supported platforms.

The test results are available from the GitHub Actions page.

Copy the test results​

For the toolchain projects, utilise the Artifact download URL from the Generate tests report GitHub Action to download the test-report.zip file, unzip it and copy the markdown file to website/docs/tests.

Edit the website/docs/tests/index.md file to add the new page to the list.

Check the Known problems link in the blog post.

Commit with the message website: add test results

Check npm package tags​

Execute the following command in a terminal:

npm dist-tag ls @xpack-dev-tools/clang

Tag the npm package as latest​

When the release is considered stable, promote it as latest:

npm dist-tag add @xpack-dev-tools/clang@20.1.8-1.1 latest

Check the result:

npm dist-tag ls @xpack-dev-tools/clang

Should the release prove problematic and require unpublishing:

npm unpublish @xpack-dev-tools/clang@20.1.8-1.1

Build the static website locally​

Validate the website content by performing a local build using the npm build script:

npm run build -C ~/Work/xpack-dev-tools/clang-xpack.git/website

Resolve any broken links that may be identified during the build process.

Publish the website​

The website deployment is performed automatically when pushing to the website branch, utilising a dedicated workflow in GitHub Actions.

The website branch may be subsequently updated, provided the version in package.json remains unchanged.

Verify the result at https://xpack-dev-tools.github.io/clang-xpack/.

Update the stable branch​

In this Git repository:

  • Select the xpack branch
  • Merge the xpack-development branch into the xpack branch
  • Push the xpack branch to GitHub
  • Check out the xpack-development branch
note

Avoid further updates to the xpack branch until the next release.

Create the final GitHub release​

  • Navigate to the GitHub Releases page
  • Check the download counter; it should match the number of tests
  • Add a link to the Web blog [Continue reading »](); utilise the same blog URL
  • Remove the tests only notice
  • Disable the Set as a pre-release button
  • Enable the Set as the latest release button
  • Click the Update Release button

Check SourceForge mirror​

Clean-ups​

Remove the pre-release binaries​

Clean the work area​

Execute the trigger-workflow-deep-clean xpm action:

xpm run trigger-workflow-deep-clean -C ~/Work/xpack-dev-tools/clang-xpack.git/build-assets

This will remove the build folders on all self-hosted runners.

The results are available from the GitHub Actions page.

Close possible open issues​

Check GitHub Issues and Pull Requests:

Close those that have been addressed.

Share on X/Twitter​

  • In a separate browser window, open X/Twitter
  • Utilising the @xpack_project account
  • Paste the release name such as xPack LLVM clang v20.1.8-1.1 released
  • Paste the link to the web page release
  • Click the Tweet button

Analytics​

Credits to Shields IO for the badges and to Somsubhra/github-release-stats for the individual file counters.