Entries tagged git

Related tags: chronicle, cvs, cvsrepository, github, lazyweb, mercurial, passwords, puppet, serverspec, ssh, todo, yawns.

Rotating passwords

Friday, 24 February 2017

Like many people I use a password-manage to record logins to websites. I previously used a tool called pwsafe, but these days I switched to using pass.

Although I don't like the fact the meta-data is exposed the tool is very useful, and its integration with git is both simple and reliable.

Reading about the security issue that recently affected cloudflare made me consider rotating some passwords. Using git I figured I could look at the last update-time of my passwords. Indeed that was pretty simple:

git ls-tree -r --name-only HEAD | while read filename; do
  echo "$(git log -1 --format="%ad" -- $filename) $filename"

Of course that's not quite enough because we want it sorted, and to do that using the seconds-since-epoch is neater. All together I wrote this:

# Show password age - should be useful for rotation - we first of all
# format the timestamp of every *.gpg file, as both unix+relative time,
# then we sort, and finally we output that sorted data - but we skip
# the first field which is the unix-epoch time.
( git ls-tree -r --name-only HEAD | grep '\.gpg$' | while read filename; do \
      echo "$(git log -1 --format="%at %ar" -- $filename) $filename" ; done ) \
        | sort | awk '{for (i=2; i<NF; i++) printf $i " "; print $NF}'

Not the cleanest script I've ever hacked together, but the output is nice:

 steve@ssh ~ $ cd ~/Repos/personal/pass/
 steve@ssh ~/Repos/personal/pass $ ./password-age | head -n 5
 1 year, 10 months ago GPG/root@localhost.gpg
 1 year, 10 months ago GPG/steve@steve.org.uk.OLD.gpg
 1 year, 10 months ago GPG/steve@steve.org.uk.NEW.gpg
 1 year, 10 months ago Git/git.steve.org.uk/root.gpg
 1 year, 10 months ago Git/git.steve.org.uk/skx.gpg

Now I need to pick the sites that are more than a year old and rotate credentials. Or delete accounts, as appropriate.



Generating fingerprints from SSH keys

Wednesday, 7 October 2015

I've been allowing users to upload SSH public-keys, and displaying them online in a form. Displaying an SSH public key is a pain, because they're typically long. That means you need to wrap them, or truncate them, or you introduce a horizontal scroll-bar.

So rather than displaying them I figured I'd generate a fingerprint when the key was uploaded and show that instead - This is exactly how github shows your ssh-keys.

Annoyingly there is only one reasonable way to get a fingerprint from a key:

  • Write it to a temporary file.
  • Run "ssh-keygen -lf temporary/file/name".

You can sometimes calculate the key via more direct, but less obvious methods:

awk '{print $2}' ~/.ssh/id_rsa.pub | base64 -d | md5sum

But that won't work for all key-types.

It is interesting to look at the various key-types which are available these days:

mkdir ~/ssh/
cd ~/ssh/
for i in dsa ecdsa ed25519 rsa rsa1 ; do
  ssh-keygen -P "" -t $i -f ${i}-key

I've never seen an ed25519 key in the wild. It looks like this:

$ cat ~/ssh/ed25519-key.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMcT04t6UpewqQHWI4gfyBpP/ueSjbcGEze22vdlq0mW skx@shelob

Similarly curve-based keys are short too, but not as short:

ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLTJ5+  \
 rWoq5cNcjXdhzRiEK3Yq6tFSYr4DBsqkRI0ZqJdb+7RxbhJYUOq5jsBlHUzktYhOahEDlc9Lezz3ZUqXg= skx@shelob

Remember what I said about wrapping? Ha!

Anyway for the moment I've hacked up a simple perl module SSH::Key::Fingerprint which will accept a public key and return the fingerprint, as well as validating the key is well-formed and of a known-type. I might make it public in the future, but I think the name is all wrong.

The only code I could easily find to do a similar job is this node.js package, but it doesn't work on all key-types. Shame.

And that concludes this weeks super-happy fun-time TODO-list item.



Validating puppet manifests via git hooks.

Monday, 27 April 2015

It looks like I'll be spending a lot of time working with puppet over the coming weeks.

I've setup some toy deployments on virtual machines, and have converted several of my own hosts to using it, rather than my own slaughter system.

When it comes to puppet some things are good, and some things are bad, as exected, and as any similar tool (even my own). At the moment I'm just aiming for consistency and making sure I can control all the systems - BSD, Debian GNU/Linux, Ubuntu, Microsoft Windows, etc.

Little changes are making me happy though - rather than using a local git pre-commit hook to validate puppet manifests I'm now doing that checking on the server-side via a git pre-receive hook.

Doing it on the server-side means that I can never forget to add the local hook and future-colleagues can similarly never make this mistake, and commit malformed puppetry.

It is almost a shame there isn't a decent collection of example git-hooks, for doing things like this puppet-validation. Maybe there is and I've missed it.

It only crossed my mind because I've had to write several of these recently - a hook to rebuild a static website when the repository has a new markdown file pushed to it, a hook to validate syntax when pushes are attempted, and another hook to deny updates if the C-code fails to compile.



Migration of services and hosts

Friday, 29 August 2014

Yesterday I carried out the upgrade of a Debian host from Squeeze to Wheezy for a friend. I like doing odd-jobs like this as they're generally painless, and when there are problems it is a fun learning experience.

I accidentally forgot to check on the status of the MySQL server on that particular host, which was a little embarassing, but later put together a reasonably thorough serverspec recipe to describe how the machine should be setup, which will avoid that problem in the future - Introduction/tutorial here.

The more I use serverspec the more I like it. My own personal servers have good rules now:

shelob ~/Repos/git.steve.org.uk/server/testing $ make
Finished in 1 minute 6.53 seconds
362 examples, 0 failures

Slow, but comprehensive.

In other news I've now migrated every single one of my personal mercurial repositories over to git. I didn't have a particular reason for doing that, but I've started using git more and more for collaboration with others and using two systems felt like an annoyance.

That means I no longer have to host two different kinds of repositories, and I can use the excellent gitbucket software on my git repository host.

Needless to say I wrote a policy for this host too:

#  The host should be wheezy.
describe command("lsb_release -d") do
  its(:stdout) { should match /wheezy/ }

# Our gitbucket instance should be running, under runit.
describe supervise('gitbucket') do
  its(:status) { should eq 'run' }

# nginx will proxy to our back-end
describe service('nginx') do
  it { should be_enabled   }
  it { should be_running   }
describe port(80) do
  it { should be_listening }

#  Host should resolve
describe host("git.steve.org.uk" ) do
  it { should be_resolvable.by('dns') }

Simple stuff, but being able to trigger all these kind of tests, on all my hosts, with one command, is very reassuring.

| 1 comment.


Updates on git-hosting and load-balancing

Monday, 25 August 2014

To round up the discussion of the Debian Administration site yesterday I flipped the switch on the load-balancing. Rather than this:

  https -> pound \
  http  -------------> varnish  --> apache

We now have the simpler route for all requests:

http  -> haproxy -> apache
https -> haproxy -> apache

This means we have one less HTTP-request for all incoming secure connections, and these days secure connections are preferred since a Strict-Transport-Security header is set.

In other news I've been juggling git repositories; I've setup an installation of GitBucket on my git-host. My personal git repository used to contain some private repositories and some mirrors.

Now it contains mirrors of most things on github, as well as many more private repositories.

The main reason for the switch was to get a prettier interface and bug-tracker support.

A side-benefit is that I can use "groups" to organize repositories, so for example:

Most of those are mirrors of the github repositories, but some are new. When signed in I see more sources, for example the source to http://steve.org.uk.

I've been pleased with the setup and performance, though I had to add some caching and some other magic at the nginx level to provide /robots.txt, etc, which are not otherwise present.

I'm not abandoning github, but I will no longer be using it for private repositories (I was gifted a free subscription a year or three ago), and nor will I post things there exclusively.

If a single canonical source location is required for a repository it will be one that I control, maintain, and host.

I don't expect I'll give people commit access on this mirror, but it is certainly possible. In the past I've certainly given people access to private repositories for collaboration, etc.

| No comments


Is there a ACL system for "all" revision control systems?

Sunday, 16 December 2012

Once upon a time a company started using distributed version control, and setup several project repositories using darcs.

Over time people became more sane and new projects were created in mercurial.

Later still Git became available, and was used by a few of the brave.

Sadly each of these projects is hosted on the same host, and in the home directory of the same user. This means these two commands work:

hg clone ssh://projects@dev.host/foo

git clone ssh://projects@dev.host/bar

I'm now wanting to setup per-repository ACLs and have hit a problem...

There are several git-wrappers such as gitolite and gitosis. There is also the excellent hg-gateway and mercurial-server for dealing with mercurial.

However I've yet to find a wrapper which will handle both git & mercurial repositories, under the same UID. (+ Darcs too, of course).

So my question - is there such a beast out there, or do we need to write it? I expect such a thing would be useful for many people, so I'm surprised I've not yet found it.



No, I don't want your number

Friday, 23 November 2007

I'm still in the middle of a quandry with regards to revision control.

90% of my open code is hosted via CVS at a central site.

I wish to migrate away from CVS in the very near future, and having ummed and ahhed for a while I've picked murcurial as my system of choice. There is extensive documentation, and it does everything I believe I need.

The close-runner was git, but on balance I've decided to choose mercurial as it wins in a few respects.

Now the plan. I have two options:

  • Leave each project in one central site.
  • Migrate project $foo to its own location.

e.g. My xen-tools could be hosted at mercurial.xen-tools.org, my blog compiler could live at mercurial.steve.org.uk.

Alternatively I could just leave the one site in place, ignoring the fact that the domain name is now inappropriate.

The problem? I can't decide which approach to go for. Both have plusses and minuses.

Suggestions or rationales welcome - but no holy wars on why any particular revision control system is best...

I guess ultimately it matters little, and short of mass-editing links its 50/50.

| No comments


I role and I tumble practically all night long

Wednesday, 10 October 2007

Today mostly consisted of a new release of the chronicle blog compiler. Interestingly this received several random mails today. I wonder what caused that all of a sudden?

The release of the compiler is timely, as it reminds me I've still not managed to find a decent gallery compiler. Although the thought of writing my own, rightly, fills me with depression.

I've been interested in reading more about both Git and SELinux upon Planet Coker Debian recently. I've switched several small projects over to GIT but I've not yet listed them publically. First of all I would like to see if there is a version of trac that I can install which supports git repositories. I guess that's a job to research tomorrow.

I wonder if I would confuse people by hosting GIT projects upon cvsrepository.org? ;)

| No comments


Recent Posts

Recent Tags