Notes on Release, Build and CI Engineering
(This is a repost of the article I wrote in 2017. I think it is as relevant now as it was then)
For any code you write, you need several steps to transform it from a set of text files to a certain release artifact or a running service. You go through these steps manually at first, but sooner or later you decide to automate them. And this is how the CI/CD pipeline of a project is born.
But there are different ways how you can organize the automation.
Or some ramblings on the difference between the mindset of developers and integration engineers, based on a totally unscientific observation.
Some time ago I worked as a Build/CI Engineer in a Java development team. I came there with zero knowledge of the Java ecosystem. I had to learn a lot, including at least three kinds of Java garbage collection and how many brackets one needs to write a “Hello world”.
I have several favorite stories from those times, and this one is about how I untangled the mess of a Gradle the Java developers managed to create in their repository.
There are several very generic mathematical abstractions, which I miss a lot in my daily life. The issue is that those concepts do exist, and I meet them all the time, but I do not know how to refer to them without using a mathematical term, that most people will not understand.
So, to celebrate the Pi Day, let's try to close the gap.
And today we are going to talk about the idea of a Manifold.
Once you get exposed to the Cloud, DevOps and Everything As A Service world, you realize that almost every piece of software you might need has already been written by someone (or provided as a service).
Why is it not all rainbows and unicorns then?
Because you do not need a tool, a software or a service. You need an infrastructure.
In this article we address another topic which appeared in multiple discussions recently. We take a look at the difference between the SRPM and the so called dist-git repository of a package. And why do we indeed prefer the dist-git.
I know people who imagine distribution development as the process of piling up the code in the git repository for 6 months and then building it all in one go at the end of those 6 months, so that it can finally be shipped. This is very far from reality. And it is impossible to explain things like CentOS Stream without addressing this confusion.
The Ultimate Source of Truth, Wikipedia, defines continuous integration as the practice of merging all developer working copies to a shared mainline several times a day.
My version goes a bit deeper.
Disclaimer: I am a Senior Principal Engineer in Red Hat. I was a member of the RHEL 9/CentOS Stream 9 Bootstrap team. Opinions are my own.
The chase for “bug-to-bug compatibility” hurts community, hurts RHEL customers and hurts the industry as a whole. The real innovation behind the CentOS Stream is the attempt to change it.