Getting Started with the xPack flex
Overview
xPack flex is a standalone cross-platform (GNU/Linux, macOS) binary distribution of flex, aimed at reproducible builds.
What are reproducible builds?
To be reproducible, an operation must remain stable over time and across different environments. In other words, if builds are repeated after some time, possibly on a different machine or platform, the resulting behavior must be functionally equivalent.
The main target of this project is macOS 10.13, as the older version of this utility fails during some builds.
flex is a tool for generating scanners: programs which recognize lexical patterns in text.
This is a fork of the original flex, maintained up to 2.6.0 (2015).
flex is an open source project hosted on GitHub and maintained by Will Estes.
The open source xPack flex project is hosted on GitHub as
xpack-dev-tools/flex-xpack;
it provides the platform specific binaries as
release assets.
In addition to the binary archives and the package metadata, this project also includes the build scripts.
xPacks
xPacks (short for xpm packages) are general-purpose, language-neutral software packages.
What the heck are xPacks? Please, do not introduce another package format!
While the initial appearance may seem complex, utilizing xPacks (xpm packages) is, in fact, straightforward. The design rationale is to automate frequent operations that occur during software development, in this case the installation of dependencies, and to ensure reproducibility.
xPacks are managed by xpm (the xPack Project Manager), a program that complements the npm CLI (the popular JavaScript package manager), with new language-neutral features.
The xPacks Framework does NOT introduce a new package format; instead,
it uses the same format as npm packages, which is a collection of
files/folders and a package.json file with the package metadata.
xpm can install packages from the same repositories as npm, whether public or private.
The packages (usually regular archives, but also git repositories), are extracted into separate folders within the project.
Based on the content, there are two types of packets:
- source xPacks (that install source files, usually libraries) and
- binary xPacks (that install executables/binary files, usually tools).
The binary xPacks include references to archives with the platform specific
binaries (such as .tar.gz for Unix or .zip for Windows).
These archives are also expanded along the package metadata. Since they
include executables, links/forwarders to
these executables are created in a .bin folder,
eliminating the need to add multiple folders to the PATH.
Given that some binary xPacks, such as toolchains, can have very large archives, the packages are extracted only once into a user global location to conserve space. In projects, instead of duplicating the content of these archives, symbolic links are created.
Simply put, xPacks can be used to further automate the installation of source libraries and tools.
Features
All binaries are:
- self-contained (include all required libraries)
- file-system relocatable (can be installed in any location)
- built on slightly older systems (to make them run on both old and new systems)
Compatibility with older systems
Given that most operating systems maintain significant compatibility with older versions, building an application on an older system ensures that the same binary can run on newer versions. Conversely, building an application on a newer system may utilize library features that are not available in older versions, making backward compatibility less feasible.
Similarly to flatpacks or snap, but significantly simpler, xPacks include all dependent shared libraries within the distributed archives, making the binaries independent of any similar libraries installed on the system. This ensures they can run on any system without needing specific libraries to be installed.
Also the builds are configured so that the binaries do not depend on being installed in specific folders, and can be installed in any location, including in user folders.
Benefits
The main advantages of using the xPack flex are:
- a convenient, uniform and portable install/uninstall/upgrade procedure; the same procedure is used for all major platforms (x64 GNU/Linux, arm64 GNU/Linux, arm GNU/Linux, x64 macOS, arm64 macOS)
- multiple versions of the same package can be installed at the same time on the same system
- no need to worry about dependent libraries, they are all included
- not affected by system updates that might change the versions of the dependent libraries
- significantly lighter and easier to use than Docker images that provide similar functionality and are GNU/Linux centric
- projects can be tied to specific tools versions; this provides a good reproducibility, especially useful in CI/CD environments.
Compatibility
The xPack flex project is fully compatible with the upstream flex.
Install
The executables and other related files can be installed automatically with xpm or manually by downloading the platform specific archives.
The details of installing the xPack flex on various platforms are presented in the Install Guide.
Documentation
The original flex documentation is available from the project web site:
Release schedule
This distribution generally follows the official flex GitHUb Releases.
Support & feedback
The quick advice for getting support and providing feedback is to use the GitHub Discussions.
For additional information, please refer to the Help Centre page.
Change log
The release and change log is available in the repository
CHANGELOG.md file.
Maintainer & Developer information
For information on the workflow used to make releases, please see the Maintainer Info page.
For information on how to build the binaries, please see the Developer Info page.
However, the ultimate source for details are the build scripts themselves,
all available from the
flex-xpack.git/build-assets/scripts
folder. The scripts include common code from the @xpack-dev-tools/xbb-helper package.
License
Unless otherwise stated, the original content is released under the terms of the MIT License, with all rights reserved to Liviu Ionescu.
The binary distributions include several open-source components; the
corresponding licenses are available in each archive in the
distro-info/licenses folder.
Releases
The list of releases is available in the Releases pages.
Credits
Special thanks to MacStadium, for providing the Apple Silicon development infrastructure as part of their generous Free and Open Source Software program.
Enjoyed Using This Project? Let Us Know!
If you enjoyed using this project, please let us know! Here are some ways you can show your support:
