Skip to main content

How to make new xPack Windows Build Tools releases

GitHub package.json version NPM Version GitHub issues GitHub pulls licence

This page is designed for maintainers of the xPack Windows Build Tools project and provides documentation on creating new releases.

Prerequisites​

@xpack-dev-tools/windows-build-tools is an xpm package that packages platform-specific archives, built with the xPack Build Box (XBB).

The Windows binaries are compiled on x64 GNU/Linux, utilising mingw-w64.

For details on installing the prerequisites, please read the Build Prerequisites page.

Obtain project sources​

The project is hosted on GitHub:

Branches

This project utilises multiple branches:

  • master, not actively used
  • xpack, containing the latest stable version (default)
  • xpack-development, containing the current development version
  • website, containing the current website content; pushes to this branch automatically trigger publication of the main website
  • development, containing the current preview website content; pushes to this branch automatically trigger publication of the preview website

All development is conducted in the xpack-development branch, and contributions via Pull Requests should be directed to this branch.

When new releases are published, the xpack-development branch is merged into xpack.

To clone the stable branch (xpack), execute the following commands in a terminal (on Windows use the Git Bash console):

rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack \
  https://github.com/xpack-dev-tools/windows-build-tools-xpack.git \
  ~/Work/xpack-dev-tools/windows-build-tools-xpack.git
For development purposes, clone the xpack-development branch.
rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack-development \
  https://github.com/xpack-dev-tools/windows-build-tools-xpack.git \
  ~/Work/xpack-dev-tools/windows-build-tools-xpack.git

Alternatively, if the repository has already been cloned:

git -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git pull
tip

To contribute Pull Requests, fork the project and ensure the Copy the master branch only is disabled.

Utilise the xpack-development branch and ensure you contribute the Pull Requests back to the xpack-development branch.

During development, it is convenient to maintain a writable instance of the library to enable changes in parallel with the parent project.

To facilitate the use of a writable instance of this library in other projects, add a link from the user's global xPacks store to this local development folder:

xpm link -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git

And in the projects referring it:

xpm link @xpack-dev-tools/windows-build-tools
Obtain the writable helper sources (optional, for development purposes)

The project has a dependency on a common helper, which is normally installed as a read-only dependency; for development purposes, to enable modifications to the scripts located within the helper, clone the xpack-development branch and link it to the user's global xPacks store:

rm -rf ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
mkdir -p ~/Work/xpack-dev-tools && \
git clone \
  --branch xpack-development \
  https://github.com/xpack-dev-tools/xbb-helper-xpack.git \
  ~/Work/xpack-dev-tools/xbb-helper-xpack.git

Alternatively, if the repository has already been cloned:

git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull

If a writable instance of this library is required in another project, add a link from the user's global xPacks store to it:

xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git

Other repositories in use are:

  • none

Release schedule​

This distribution generally follows the official GNU make releases, which are very rare, and the BusyBox tags.

Prepare a new release​

Before starting the build, perform some checks and adjustments.

Download the build scripts​

The build scripts are available in the build-assets/scripts folder of the xpack-dev-tools/windows-build-tools-xpack Git repository.

To download them on a new machine, clone the xpack-development branch, as described above.

Check Git​

In the xpack-dev-tools/windows-build-tools-xpack Git repository:

  • Switch to the xpack-development branch
  • Pull new changes
  • If necessary, merge the xpack branch
  • If necessary, merge the website branch
caution

This is critically important; otherwise you will release the previous content again!

The xpack branch should remain unchanged since the previous release and will be updated when the new release is published.

Update helper & other dependencies​

Check the latest versions at https://github.com/xpack-dev-tools/ and update the dependencies in build-assets/package.json.

Generate the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git

Commit the top changes​

  • Stage GitHub workflows and package.json files (top and from build-assets)
  • Commit with the message re-generate top commons

Generate the website commons​

Execute the generate-website-commons npm script from website/package.json within the project folder.

npm run generate-website-commons -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website

Commit the website changes​

  • Stage the website folder and top README.md
  • Commit with the message re-generate website commons

Check the latest upstream release​

GNU Make​

The latest releases must have been announced in GNU Make releases.

Compare the latest upstream release with the latest xPack release. If necessary, update the triplet, otherwise increase the fourth number, as below.

BusyBox (Windows fork)​

The Windows fork of the BusyBox project doesn't have full releases, but the latest versions are tagged in the GitHub project (https://github.com/rmyorston/busybox-w32/tags).

When updating busybox, increase the fourth version number.

Increase the version and update the top package.json​

As required by npm projects, this one also utilises SemVer.

  • Determine the next version (such as 4.4.1-3.1)
  • Update the version in the top package.json file
  • Use the new version, suffixed by .pre, such as 4.4.1-3.1.pre.

Update VERSION​

Update the build-assets/scripts/VERSION file; the format is 4.4.1-3. The fourth number is the xPack release number of this version. A fifth number will be added to the version in package.json when the package is published to the npm server.

Update the websiteConfig (if necessary)​

Check and possibly update the release-specific properties (like the release date) within website/package.json.

Update the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git

Update the website commons​

Execute the generate-website-commons npm script from website/package.json within the project folder.

npm run generate-website-commons -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website

Start the local web server​

Execute the npm script clear followed by start in the website sub-project, or execute the following within the project folder:

npm run clear -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website
npm run start -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website

Navigate to the Maintainer Info page, the Start the local web server section.

Update the version specific code​

  • open the build-assets/scripts/versioning.sh file
  • add a new if with the new version before the existing code

Fix possible open issues​

Check GitHub Issues and Pull Requests:

and resolve them; assign them to a milestone (such as 4.4.1-3.1).

Update CHANGELOG.md​

  • Open the CHANGELOG.md file
  • Check if all previously fixed issues are included
  • Check the latest commits npm run git-log; if necessary, copy/paste lines, group by dates and edit them using the regular expressions below
  • To convert the dates into headings, change from: 


    ([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]) [*]
    to: 


    ## $1

    *
  • To remove the remaining dates, change from: 
    ^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [*]
    to: 
    *
  • Add a new entry such as * v4.4.1-3 prepared
  • Commit with the message prepare v4.4.1-3

Push the changes to the xpack-development branch​

Ensure that the latest commit that updates VERSION is pushed to the xpack-development branch!

caution

This is critically important; otherwise the next steps will build the previous release again!

Execute the development builds​

The build currently executes on the x64 GNU/Linux machine.

Before the actual build, execute a test/development build.

Visual Studio Code extension​

All actions are defined as xpm/xPack actions and can be conveniently triggered via the VS Code graphical interface, utilising the xPack C/C++ Managed Build Tools extension.

Patches​

This project uses the upstream sources, without patches.

Restart the Docker daemons​

If, for any reason, the Docker builds need to be cancelled, it is very likely that some processes will continue to execute in the background.

To completely terminate them, and ensure that Docker executes from a clean slate, restart the Docker daemons on all GNU/Linux machines:

sudo systemctl restart docker

Verify if both projects are pushed​

Verify that both the current project and the helper are on the xpack-development branch and the latest commits are pushed to GitHub.

caution

This is particularly crucial for large projects; otherwise, the build may proceed with the previous version.

Build the x64 Windows binaries​

Execute the Docker build on the production machine (xbbli); start a VS Code remote session, or connect with a terminal:

caffeinate ssh xbbli

To check the available space:

xpm run check-space -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

To free the space utilised by all previous builds of all xpack-dev-tools packages:

xpm run clear-all-projects-builds -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

To update the build scripts and build the development binaries:

rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
git -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpack-dev-tools/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
\
xpm run deep-clean --config win32-x64  -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
xpm run docker-prepare --config win32-x64 -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
xpm run docker-link-deps --config win32-x64 -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
xpm run docker-build-development --config win32-x64 -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

Several minutes later, the output of the build script is a compressed archive and its SHA signature, created in the build-assets/build/win32-x64/deploy folder:

  • xpack-windows-build-tools-4.4.1-3-win32-x64.zip
  • xpack-windows-build-tools-4.4.1-3-win32-x64.zip.sha

To update the build scripts and build the release binaries:

rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/package-lock.json && \
rm -rf ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets/package-lock.json && \
git -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git pull && \
xpm run install -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
\
xpm run deep-clean --config win32-x64  -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
xpm run docker-prepare --config win32-x64 -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets && \
xpm run docker-build-development --config win32-x64 -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

Update the durations of the development builds​

In website/docs/_project/_development-durations.mdx, update the durations of the builds.

Commit with the message website: update development durations

Update the trees in the Installation Guide page​

Copy the trees displayed at the end of the test builds and paste to the Folder hierarchy section within the web file:

  • website/docs/_project/_folders-hierarchies-windows.mdx

If present, replace the actual version (such as 4.4.1-3.1) with the following expression:

${customField('releaseVersion')}

Commit with the message website: update folder hierarchies

Copy and paste the complete list of links displayed at the end of the build, in sequence, and check the differences compared to the repository.

Commit if necessary with the message package.json: update executables links

How to build a debug version​

In some cases it is necessary to execute a debug session with the binaries.

For these cases, the build script accepts the --debug options.

There are also xpm actions that utilise this option (build-development-debug and docker-build-development-debug).

Files cache​

The XBB build scripts utilise a local cache such that files are downloaded only during the initial execution, with subsequent executions being able to utilise the cached files.

However, occasionally some servers may not be available, and the builds may fail.

The workaround is to manually download the files from alternative locations (such as https://github.com/xpack-dev-tools/files-cache/tree/master/libs), place them in the XBB cache (Work/cache) and restart the build.

Execute the production builds​

The automation is provided by GitHub Actions.

Publish the helper​

Publish a new release of the helper on npmjs.

In the npm packages helper, update the dependency to the new helper​

In templates/_xpack-dev-tools/build-assets/package-merge-liquid.json, update the reference to "@xpack-dev-tools/xbb-helper":.

Generate the top commons​

Execute the top generate-top-commons npm script to update the build-assets/package.json and the GitHub workflow files.

npm run generate-top-commons -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git

Commit the top changes​

  • Stage GitHub workflows and package.json files (top and from build-assets)
  • Commit with the message re-generate top commons

Commit and push​

  • Push the xpack-development branch to GitHub
caution

This is particularly crucial for large projects; otherwise, the build may proceed with the previous version.

Verify if the helper is pushed and published​

The helper xpack branch must be updated and the npm package published.

Manually trigger the build GitHub Actions​

To trigger the GitHub Actions builds, utilise the xpm actions:

  • trigger-workflow-build-win32-x64

The scripts behind these actions require the GITHUB_API_DISPATCH_TOKEN variable to be present in the environment, and the organisation PUBLISH_TOKEN to be visible in the Settings → Action → Secrets page.

These commands utilise the xpack-development branch of this repository.

Durations & resulting binaries​

The build takes several minutes to complete.

The workflow results and logs are available from the GitHub Actions page.

Update the values in website/docs/_project/_github-actions-durations.mdx with those shown by GitHub Actions.

Commit with the message website: update actions durations

The resulting binaries are available for testing from pre-releases/test.

Execute the tests​

Automated tests​

The automation is provided by GitHub Actions.

To trigger the GitHub Actions tests, execute the xpm actions:

  • trigger-workflow-test-prime
xpm run trigger-workflow-test-prime -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

The scripts behind these actions require the GITHUB_API_DISPATCH_TOKEN variable to be present in the environment.

These actions utilise the xpack-development branch of this repository and the pre-releases/test binaries.

The test results are available from the GitHub Actions page.

After executing the prime tests, download the test results and compare with previous runs.

Manual tests​

To download the pre-released archive for the specific platform and execute the tests, utilise:

git -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git pull
xpm run install -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets
xpm run test-pre-release -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets
caution

Since the builds were performed on the latest GitHub macOS runners, it is strongly recommended to also execute the tests on an older macOS 11 machine.

For additional tests, on each platform, download the archive from pre-releases/test and check the binaries.

Publish the release​

Create a new GitHub pre-release​

  • In CHANGELOG.md, add the release date and a message such as * v4.4.1-3 released
  • Commit with the message CHANGELOG update
  • Check and possibly update the build-assets/templates/body-github-release-liquid.mdx
  • Push the xpack-development branch to GitHub
  • Execute the trigger-workflow-publish-release xpm action: 
    xpm run trigger-workflow-publish-release -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

The workflow result and logs are available from the GitHub Actions page.

The result is a draft pre-release tagged such as v4.4.1-3 (mind the dash in the middle!) and named such as xPack Windows Build Tools v4.4.1-3 (mind the dash), with all binaries attached.

  • Edit the draft and ensure it is attached to the xpack-development branch (important!)
  • Perform the final edits (maintenance vs. new release) and check if everything is satisfactory
  • Keep the pre-release button enabled
  • Do not enable Discussions yet
  • Click the Publish release button
info

At this moment the system should send an email notification to all clients watching the windows-build-tools-xpack project.

Prepare a new blog post​

  • Check and possibly update the website/blog/_templates/blog-post*-release-liquid.mdx
  • Execute the generate-website-blog-post npm script; this will add a file to the website/blog folder: 
    npm run generate-website-blog-post -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website
  • Edit the front matter properties (the description property)
  • Select the correct summary (maintenance vs. new release)
  • Add entries in the Bug fixes; utilise the following syntax 
    - [[#N](https://github.com/xpack-dev-tools/windows-build-tools-xpack/issues/N)]:
  • Copy/paste the folder structures from the installation guide
  • Commit with the message website: blog post release 4.4.1-3.1 published

Update the web install page​

  • Check and possibly update the output of the --version runs in
    • website/docs/install/_project/_automatic-install-quick-test.mdx
    • website/docs/install/_project/_manual-install-quick-test.mdx
  • Commit with the message website: update quick tests
  • Open the top package.json file
  • Check if the links in the bin property cover the actual binaries
  • If necessary, also check on Windows

Update the package.json list of binaries​

  • Select the xpack-development branch
  • Ensure that the binaries were attached to the pre-release
  • Execute the update-package-binaries xpm action: 
    xpm run update-package-binaries -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets
  • Open the top package.json file
  • Check the baseUrl: it should match the file URLs (including the tag/version); no terminating / is required
  • From the release, check the SHA & file names
  • Compare the SHA sums with those shown by cat *.sha
  • Check the executable names
  • Commit all changes with the message package.json: update URLs for 4.4.1-3.1 release

Prepare the release​

  • Switch to the xpack-development branch
  • If there are changes, commit all changes to the xpack-development branch
  • Check the latest commits npm run git-log
  • Update CHANGELOG.md, add a line such as * v4.4.1-3.1 published on npmjs.com
  • Commit with the message CHANGELOG: publish npm v4.4.1-3.1
  • Check the list of files included in the npm package: 
    npm run npm-pack
  • Possibly update .npmignore

Set the new version​

Execute the following within the project folder:

npm version 4.4.1-3.1

The postversion npm script should also update tags via git push origin --tags; this should also trigger CI tests.

Publish the release to npmjs.com​

Execute the following command within the project folder:

npm publish --tag next
tip

When publishing for the first time, utilise --access public.

After a few moments, the version will be visible at npmjs.com, under the Versions tab.

Test if the binaries can be installed with xpm​

Execute the trigger-workflow-test-xpm xpm action:

xpm run trigger-workflow-test-xpm -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/build-assets

This will install the package via xpm install on all supported platforms.

The test results are available from the GitHub Actions page.

Check npm package tags​

Execute the following command in a terminal:

npm dist-tag ls @xpack-dev-tools/windows-build-tools

Tag the npm package as latest​

When the release is considered stable, promote it as latest:

npm dist-tag add @xpack-dev-tools/windows-build-tools@4.4.1-3.1 latest

Check the result:

npm dist-tag ls @xpack-dev-tools/windows-build-tools

Should the release prove problematic and require unpublishing:

npm unpublish @xpack-dev-tools/windows-build-tools@4.4.1-3.1

Build the static website locally​

Validate the website content by performing a local build using the npm build script:

npm run build -C ~/Work/xpack-dev-tools/windows-build-tools-xpack.git/website

Resolve any broken links that may be identified during the build process.

Publish the website​

The website deployment is performed automatically when pushing to the website branch, utilising a dedicated workflow in GitHub Actions.

The website branch may be subsequently updated, provided the version in package.json remains unchanged.

Verify the result at https://xpack-dev-tools.github.io/windows-build-tools-xpack/.

Update the stable branch​

In this Git repository:

  • Select the xpack branch
  • Merge the xpack-development branch into the xpack branch
  • Push the xpack branch to GitHub
  • Check out the xpack-development branch
note

Avoid further updates to the xpack branch until the next release.

Create the final GitHub release​

  • Navigate to the GitHub Releases page
  • Check the download counter; it should match the number of tests
  • Add a link to the Web blog [Continue reading »](); utilise the same blog URL
  • Remove the tests only notice
  • Disable the Set as a pre-release button
  • Enable the Set as the latest release button
  • Click the Update Release button

Check SourceForge mirror​

Clean-ups​

Remove the pre-release binaries​

Close possible open issues​

Check GitHub Issues and Pull Requests:

Close those that have been addressed.

Share on X/Twitter​

  • In a separate browser window, open X/Twitter
  • Utilising the @xpack_project account
  • Paste the release name such as xPack Windows Build Tools v4.4.1-3.1 released
  • Paste the link to the web page release
  • Click the Tweet button

Analytics​

Credits to Shields IO for the badges and to Somsubhra/github-release-stats for the individual file counters.