Entries tagged ci

I'd like to setup some auto-builders for some projects - and theese projects must be built upon Lenny, Squeeze, Lucid, and multiple other distros. (i386 and amd64 obviously.)

Looking around I figure it should be simple. There are a lot of continuous integration tools out there - but when looking at them in depth it seems like they all work in temporary directories and are a little different to how I'd expect them to be.

Ultimately I want to point a tool at a repository (mercurial), and receive a status report and a bunch of .deb packages for a number of distributions.

The alternative seems to be to write a simple queue submission system, then for each job popped from the queue run:

• Creates a new debootstrap-based chroot.
• Installs build-essential, mercurial, etc.
• Fetches the shource.
• Runs make.
• Copies the files produced in ./binary-out/ to a safe location.
• Cleans up.

Surely this wheel must already exist? I guess its a given that we have to find build-dependencies, and that we cannot just run "pbuilder *.dsc" - as the dsc doesn't exist in advance. We really need to run "make dependencies test build", or similar.

Hudson looked promising, but it builds things into /var/lib/hudson, and doesn't seem to support the use of either chroots or schroots.

ObQuote: "I feel like I should get you another sweater." - "Friends"

Tags: chroot, ci, debian, hudson, schroot | 8 comments

I've been overhauling the way that I build Debian packages and websites for myself. In the past I used to use pbuilder but it is pretty heavyweight and in my previous role I was much happier using Gitlab runners for building and deploying applications.

Gitlab is something I've no interest in self-hosting, and Jenkins is an abomination, so I figured I'd write down what kind of things I did and explore options again. My use-cases are generally very simple:

• I trigger some before-steps
  • Which mostly mean pulling a remote git repository to the local system.
• I then start a build.
  • This means running debuild, hugo, or similar.
  • This happens in an isolated Docker container.
• Once the build is complete I upload the results somehere.
  • Either to a Debian package-pool, or to a remote host via rsync.

Running all these steps inside a container is well-understood, but I cheat. It is significantly easier to run the before/after steps on your host - because that way you don't need to setup SSH keys in your containers, and that is required when you clone (private) remote repositories, or wish to upload to hosts via rsync over SSH.

Anyway I wrote more thoughts here:

• https://github.com/skx/thyme/

Then I made it work. So that was nice.

To complete the process I also implemented a simple HTTP-server which will list all available jobs, and allow you to launch one via a click. The output of the build is streamed to your client in real-time. I didn't bother persisting output and success/failure to a database, but that would be trivial enough.

It'll tide me over until I try ick again, anyway. :)

Tags: ci, ick | No comments

In a previous job I wrote some simple utilities which were helpful for continuous-integration pipelines. Most of these were very very simple, for example:

• Find all *.json files, and validate them.
• Find all *.yaml files, and validate them.
• Run all the tests in a given directory, execute them one by one.
  • But stop if any single test failed.
  • run-parts does this on Debian GNU/Linux systems, but on CentOS there is no -exit-on-error flag.
  • Which is the sole reason I had to re-implement that tool. Sigh.
• Parse PHP-files and look for comments that were bogus.
  • I mentioned this in the past.

Each of these simple tools were run of the mill shell-scripts, or simple binaries. I've been putting together a simple deployable tool which lets you run them easily, so here it is:

• https://github.com/skx/sysbox/

The idea is that you install the tool, then run the commands by specify the appropriate sub-command:

sysbox validate-yaml [path/to/find/files/beneath]

Of course if you symlink validate-yaml to sysbox you don't need to prefix with the name - so this is just like busybox in this regard.

Might be interesting to some.

Tags: ci, golang, syadmin-tools, utility | No comments