Documentation
RPM OSTree Engine simplifies the process of maintaining an rpm-ostree repository. It's an opinionated, containerized tool-set to build multi-architecture extensions of rpm-ostree based operating systems like Fedora IoT and managing remote repositories.
Features in short:
- Repository Deployment: local, sshfs
- Supported Sources: Fedora IoT 33
- Build caching
- Integration via gitlab CI templates
- Generation of Deltas
- Multiple Architectures
- Linters: json
- Build extension via script hooks
- Automated Signing (TODO)
- RAW image generation (TODO)
Parts of it's strength comes from using gitlab tools, hosting the project on a gitlab instance.
(Opinionated) Quick-Start
To get your first build running we'll assume gitlab as hosting instance with gitlab-runners of required architectures available. For actual deployment we further assume an sshfs target remote for dropping the repository and a webserver configured to serve it.
As bullet points:
- an sshfs remote setup to hold the repository
- an HTTP webserver in front of the repository to serve it
- gitlab runners available in your gitlab instance of the required hardware architectures
Apart from this really opinionated setup many more configurations can be built of course.
Let's get a quick overview of the steps to get going:
- Create a gitlab repository with the following structure
- Setup basic ostree repo configuration (see resources/test/test-repo for a starting point)
- Write a .gitlab-ci.yml file extending the provided template as shown below
Expected repository structure (see example in resources/test/test-repo):
/
├── ostree-files/ - Everything to be included into the image
│ ├── ostree.json - treefile
│ └── treecompose-post.sh - post-processing script (shell)
├── hook.d/ - Place custom pre-build scripts here
└── .gitlab-ci.yml - extends rpm-ostree-engine/gitlab-ci-template.yml
In order for the configuration to be used in a build use the job template build-ostree
. Create any number of extending jobs, depending on the number or architectures you want to maintain. Tag your jobs with e.g. tags: [arm64]
depending on the runners tags of your instance. If your instance does not feature aarch64 runners for example, you could run gitlab-runner on a RPi4 (RPI3 is too slow for builds) and register it in your project.
An example .gitlab-ci.yml
:
include:
- remote: https://git.shivering-isles.com/os-forge/rpm-ostree-engine/-/raw/main/gitlab-ci-template.yml
variables:
CI_OSTREE_REF_NAME: OSTreeBeard
build-ostree-amd64:
extends: .build-ostree
# skip sshfs mount
before_script:
- true
tags:
- amd64
In order to generate deltas use the job template build-deltas
. Create one per architecture here as well. This is technically not necessary, since delta-generation is architecture independent but this still needs to be generalised.
In order to extend the build process with custom scripts place any script files into the hook.d directory at the root of your repo. E.g. to sed a configuration file of the source repo before running the actual build.
Repository Management
Get your repository into the container
The container can either be used to manage a repository mounted as volume or synchronized via sshfs.
Volume Mount:
Just mount the repository into the container. E.g. within the repository on your local machine run podman run --rm -it $(pwd):/remote-storage quay.io/os-forge/rpm-ostree-engine:1.0.0 /bin/bash
to get a shell in the container with the repository mounted under /remote-storage
. You can now proceed with the build part.
SSHFS - Local:
Locally you can mount the remote via the following command within the running container (e.g. as a toolbox):
$ mount.sh sshfs --sshfs-target=example.com --sshfs-user=user --sshfs-path=/some/dir --sshfs-key=<private-key>
On success the target will be present under /remote-storage
and you can proceed with the build step.
SSHFS - CI:
In a CI pipeline the image will attempt to mount an sshfs remote to /remote-storage
if you don't override the before_script
with true
. This directory is then symlinked into the build environment and used as the builds target, adding a commit to the existing repository. If no repository exists a new one will be created.
Use the following CI variables to control the SSHFS mount:
CI_SSHFS_TARGET=
CI_SSHFS_PATH=
CI_SSHFS_USER=
CI_SSHFS_PRIVATE_KEY=
Build
The build task will use .cache
and .deploy-repo
in the current working directory as optional, pre-existing folders. It is expected to run the build script in the projects root directory.
Locally run a build as follows, most parameters are optional depending on your setup:
$ build.sh --ostree-files=ostree-files --spec=ostree.json --treecompose=treecomose-post.sh --name=nameIt --branch=develop
In CI use the following variables to control the build (subject to change):
# Mandatory
OSTREE_REF_NAME=
# Optional
OSTREE_FILES_DIR=
OSTREE_FILE=
OSTREE_FILES_TREECOMPOSE=
OSTREE_REF_BRANCH=
Guide: Setting up a Remote Repo
- sshfs
- ssh-user
- sshd-settings for sshfs access (security)
- authorized_keys
- nginx as webserver
- nginx ostree specific config (caching etc.)
Guide: Setup gitlab-runner on RPi4
tbd.