Entries posted in April 2008

Recently I took stock of my Javascript programming efforts, with the intention of simplifying things. To date I've put together three sites which make use of Javascript:

debian-administration.org

There isn't very much javascript in use upon this site, and you might be forgiven for thinking there was none at all if you don't have an account, or don't login.

There's a tagging system which is starting to creak under the sheer number of different tags, and several back-end parts of the site make use of AJAX calls.

Most of the script lives in a single file common.js which I cobbled via a process of trial and error, augmented with a little copy & paste coding.

It works. But I knew I could do better ..

ctrl-alt-date

This was my first attempt to make a site be truely dynamic and "pretty". It has succeeded in that respect, although the lack of members makes the site itself essentially a failure.

This time round I decided that I'd utterly refuse to write my own Javascript. So instead I used script.aculo.us (damn I hate those ugly URLS).

This library made it almost too easy to add flash. I liked it a lot.

Having said that though the sheer scope of the library and the way it didn't fit in the way that I coded made it painful to use at times.

It works, and it works well. Like it? Yes. Love it no?

mail-scanning.com

This site has a fair amount of javascript upon it, but that isn't obvious unless you're actually a user. The only piece that I can recall is the real-time count of spam caught upon the front-page. (Realtime stats are cool!)

Most of the code here is the simple kind, reverting back to the way I worked on the Debian Administration site; we're talking about basic effects such as:

• show/hide a div
• make an AJAX request every now and again.
• Do a bit of auto-completion.

To get more of a feel for whats out there I wrote this initially with my own code, then later migrated it to jQuery.

Quite frankly jQuery rocks. The way it works is a little strange at first, but it is so natural after a while. As an example:

// find the div called "foo" - hide it.
$("#foo").hide()

I'm liking this library a lot recently, but only time will tell if I use it more.

In conclusion I filed #473125: ITP jQuery failing to see the existing ITP already present.

Once the package makes it through the NEW queue I will update it to follow the skeletal javascript-policy.

I disagree about the naming scheme suggested by that policy primarily because we already have packages of several of the "big" javascript libraries, such as scriptalicious, and see no gain in renaming them. But otherwise the policy is sane enough; following it will cause no harm at the very least.

ObQuote: Stand By Me

Tags: javascript, jquery, prototype, scriptalicious | No comments

I've made a new release of the chronicle blog compiler, primarily to allow the "Subject:" header to be used for new blog subjects.

(That allows new entries to be automatically posted via email, with an appropriate procmail setup. I'll add one as an example shortly.)

Whilst on the subject of RSS creation (huh?) I've written a tiny utility which will create an RSS feed from a list of text files. It will also create an index.html file to match.

To see why this is useful you could view my recent changelog.

I think there is a need for a small tool to read files and create feeds from them - like mod_index_rss does, but without messing with Apache.

If there is any interest I'd be happy to release the code, as-is it doesn't use a template..

Online privacy is important. Mostly when this is discussed it is in the context of client-side anonymity.

Looking at it from the other side, though, How do you host a website anonymously?

You could register the domain via a proxy, or with bogus details. But if you host the site yourself the IP address may be traced to the hosting provider, and that may be used to trace back to you.

So, the alternatives? Well you could use a hosted site such as livejournal / wordpress / googlepages / etc. But pretty surely they'll be able to trace content back to you - and if you don't host it there's a high chance they'll just pull it if you talk about "bad things". (I guess you could use TOR for uploading / your connections there.)

So, going back to the question. How can you host something, easily accessible to the world, without risk of your identity/association being discovered?

---

I'm, obviously, ignoring FreeNet. Two reasons for that:

• It's slow, has no search-engine goodness, and is unproven.
• It requires an atypical client. Aunt Milly won't be able to surf Freenet...

I almost think the best way forward would be to write a site which was a proxy for a file-sharing protocol, then link people to items that way. Relying on the swarm to host the files..

The downside is that you'd have to have a convincing argument for when RIAA comes calling, suggesting that you're sharing their stuff too. If it wasn't a general purpose proxy then the deniability is gone, and if it is you're at risk of general copyright infringement claims.

Hard problem. Shame.

ObQuote: HellRaiser

Tags: chronicle, rss | 6 comments

François Marier has recently been posting some interesting entries about the Mutt mail client upon his blog.

His tips are pretty basic, but that doesn't make them less useful. So here's my tip of the day: reply_regexp.

When you're viewing a mail, and you choose to reply it the subject of that mail is the basis of your message's subject. For example given a message with "Subject: hello" your reply will typically have the subject "Subject: Re: hello".

This is real rocket science here, people.

Imagine you're using SPAM filtering which tags messages it isn't sure about with a prefix "UNS:". Suddenly things don't look so hot, as you might end up with a mail with the subject:

Subject: UNS: Re: Hello

Reply to that mail, whilst being half-asleep, and what do you get? You get this:

Subject: Re: UNS: Re: Hello

Bad. Ugly. Wrong.

The following snippet in my ~/.muttrc file correctly deals with this case:

set reply_regexp="^(((UNS:[ \t])|[rR][eE]:[ \t])*)+"

Neat. Cool. Nice.

I've paid for some advertising upon the LWN.net site. (No link, I'm not trying to game things here.)

I didn't know what to expect, but I was willing to risk the expenditure as a way of saying thanks to them for their great content. (Because of my Debian Project membership I get a free LWN subscription, as do all developers. see here for details if you're a developer without a subscription.)

I've paid $30 and that has given me a months run of my advert, with clickthroughs hovering around 1%. Not horribly bad at all!

I probably won't repeat the experiment for the forseeable future, but I'm glad that I did it at least once.

If you have something appropriate for a Debian-based audience don't forget you're welcome to advertise upon the Debian Administration website for free - See the Advert FAQ for details.

ObQuote: The Craft

Update: jquery was accepted yesterday. Today I uploaded a new version to more closely align it with the javascript-policy.

Tags: advertising, jquery, lwn, mutt, muttrc | No comments

One thing I do a lot is select a region of text, then have it replaced with the output of a command.

The most common job is sorting a number of lines, such as "use XX:YY;" lines in perl scripts.

Finally having gotten annoyed enough about how clunky shell-command-on-region was I wrote my own lisp function:

• replace-region-command-output

Only after that did I discover M-x sort-lines. D'oh. Still I guess my solution is more general, and less difficult to use. (I find the use of the Emacs prefix troublesome to type; since you have to do it in advance - I almost always forget.)

I also learnt of M-x list-matching-lines yesterday. Thats a new discovery which really rocks. (I can use "^sub " to find a list of subroutines, etc.)

This could be improved, and fleshed out a lot if there were any interest.

But its neat as-is:

#!/bin/sh
#
#  Dump packages in the NEW queue.
#
#  This could be improved, perhaps:
#
#  --show-names --show-dates, etc.  Or just show all info in a table.
#
wget --quiet -O-  http://ftp-master.debian.org/new.html | \
 perl -ne 'print $1 . "\n" if ( $_ =~ /^<td valign="top" class="sid">([^<]+)<\/td>$/ )'

I thought there was something similar in the devscripts package, or contained within debian-goodies but apparently not.

ObQuote:Battle Royale

Tags: debian, emacs, new-queue | 2 comments

So here's the hard part, if you start writing about people and relationships you have to continue, otherwise people who do follow will make assumptions, or not be aware of any potential gaffes they are about to make.

I've written in the past about some stuff which I now wish I hadn't because this post is suprisngly hard to compose.

I'm now single, and living alone.

No fiance, no partner, and no kitten.

The next time I write about friends I'll make sure it is for local people only. Unless I get carried away, in which case I should be stopped.

ObQuote: The Taming of the shrew; via 10 Things I hate about you.

Tags: megan, personal | No comments

http://10.print.debian.rocks.twentygototen.org/

ObQuote: Aliens

Tags: debian, links, random, tef | No comments

If you upload a new package to the Debian archive which contains a setuid or setgid binary please please ask for a security audit, or carry out one yourself.

I certainly accept that the security audit project webpages are not terribly current, and the mailing list is essentially dead, but there are people, such as myself, who would gladly look at your package. All you have to do is ask.

When I see two packages in testing with trivialy obvious security bugs it just makes me wonder why we bother.

• tss
  • tss: local root exploit
  • tss: Allows reading arbitary files
• acon
  • #475733 acon: local root exploit

I'm going to take this chance to restate my hardline position on package maintainence - even though it might not be directly applicable - If you cannot program/debug/handle the language a package is developed in you shouldn't maintain it.

Too often I've seen signs of this; somebody maintaining a C-based program but unable to program in C. Why?

I wonder if we could have a policy / guideline that any new setuid/setgid application must have at least two maintainers, or a documented audit prior to acceptance? Hard to manage but I think it would be useful even if it didn't catch everything. Some bugs such as #475747 (lovely number!) are trivial to discover.

ObQuote: Dangerous Liaisons

Tags: rants, security | No comments

I've made a new release of the chronicle blog compiler.

There are a couple of minor changes to the variables exported to the theme templates, as contributed by MJ Ray, and a new spooling system.

This works in a simple fashion, and allows you to queue up posts. For example If you write a new entry containing the psuedo-header "Publish: 20th April 2008" and you have a crontab entry containing this:

  chronicle-spooler
    --spool-dir=~/blog/spool/  \
    --live-dir=~/blog/data/  \
    --post-move='cd ~/blog && make upload'

It works as expected. When you call this on the 20th April the file will be moved from ~/blog/spool into ~/blog/data, and your blog will be rebuilt & uploaded.

The implementation was different than the original suggestion, but is nice and clean, and can be used to do other things with a little bit of effort.

Anyway if you see this entry the spooling system is working!

ObQuote: 30 Days of Night.

Tags: blog, chronicle | No comments

My blog compiler received a bit of love recently, primarily because MJ Ray wanted to use it.

As mentioned before I've added a simple spooling system, and the mercurial repository now contains a simple RSS importer.

In other news I've been working on various Debian packages, here is a brief summery:

bash-completion

After seeing a RFH bug I closed a few bash-completion bugs, and submitted patches for a couple more.

I was intending to do more, but I'm still waiting for the package code to be uploaded to the the alioth project.

javascript work

I've updated the jquery package I uploaded to follow the new "Javascript standard" - in quotes only because it is both minimal and new.

Once the alioth project has been configured I'll upload my sources.

Apache2

I've agreed to work on a couple of SSL-related bugs in the Apache 2.x package(s) - time disappeared but I hope to get that done this weekend.

Initially that was because I was hoping I could trade a little love for getting a minor patch applied to mod_vhost_alias - instead I've now copied that module into libapache2-mod-vhost-bytemark and we'll maintain our own external module.

I've been loaned a Nokia 770 which is very nice. Having used it with vim, ssh & etc I think that I'd rather have a device with a real keyboard.

The Nokia 810 looks pretty ideal for me. I'm going to be asking around to see if I can get a donated/loaned device to play with for a while before I take the plunge and pay for one of my own.

I've got a couple more things on the go at the moment, but mostly being outdoors is more interesting to me than the alternative. Hence the downturn in writing and releasing security advisories.

I'll pick things up more fully over the coming weeks I'm sure.

ObQuote: Shaun of the Dead

Tags: alioth, chronicle, debian, nokia, nokia 770, nokia 810, projects | No comments

So I've written a quick hack. A client-side filter/utility program for working against IMAP servers.

Consider it a general purpose system which is similar to Procmail, but applied after your remote machine has already done the sorting.

Here's a flavour:

<GMail>
  username somebody.like.me
  password yeah.right
</Gmail>

<Folders>
  <livejournal>
        unread exec /usr/local/bin/notify "Livejournal Comment"
        mark read
  </livejournal>

  <inbox>
        mark read
  </inbox>

</Folder>

What does that do? It first of all logs into GMail with the given username and password, then selects two folders:

=livejournal/

For each unread message in the folder it runs the specified command with STDIN being the message body.

Then it marks each new message as "read".

=inbox/

This simple rule just marks all messages as read.

Why? Well I have a bunch of folders on a bunch of gmail accounts and I don't pay attention to them - but some, specific, mails should result in an SMS being sent to me ... so I need to do something clever.

I'm sure with a bit of effort this could be made IMAP-server independent, and could have a more flexible matching system. The simplicity right now comes about primarily because i dont want to parse a config file.

Anyway, suggestions for potential features are welcome. It does what I need as-is, even if it isn't pretty.

ObQuote: Lost Boys

Tags: gmail, hacks, imap, perl, sift | 2 comments

I've updated the IMAP utility that I mentioned previously, which has now been given the name sift. It will accept, and process, a much simpler configuration file format keeping state as it goes.

Here's my updated sample file:

username: blah.bah
password: pas.word

#
#  Comments are fine.
#
folder:livejournal status:new subject:temp mark:read exec:~/bin/notify
folder:foo status:new mark:read
folder:bar status:old exec:/usr/local/bin/record delete

Each line consists of a set of tokens, split by whitespace, which is "executed" in order.

So the first line selects the folder "livejournal", finds messages which are "new", then each message containing "temp" in the subject is marked as read, and the program "notify" is executed once for each match.

Essentially we keep a list of messages as "current" as we process each line, that list of messages is then refined as we move through the line. (When a folder is opened all messages are selected by default.)

As a simple example to delete all the messages contained in a folder we'd use this:

folder:foo delete

To refine that to only delete messages from "fred" we'd say:

folder:foo from:fred delete

(If there were no matches the "delete" action wouldn't occur.)

Consider each line of input a collection of filters each operating on the previous result. Simple to understand, simple to extend with more operations, and simple for me to code!

TODO: Add a "move:xxx" to move a message to folder "xxx", and a bit more polish, then release.

ObQuote: Tron.

Tags: gmail, hacks, imap, perl, sift | No comments

OK I'm done with this now, the sift utility has been released.

I think that is a large overlap with imapfilter; but I win because I can write simple rules, rather than any actual code, to perform jobs.

In other news I flew my kite today, and I still like eating Pies: Thank God reading Debian Planet isn't mandatory.

ObQuote: The Godfather

Tags: gmail, hacks, imap, perl, sift | No comments

So I've made a new release of sift, the IMAP client/toy.

I think I named this one after looking up synonyms of "sieve", or "filter". Can't recall now.

The next time I release a toy application, tool, or hack, I will take the time to sit down and write namecheck. (Update: Too easy. Done.)

Here's what it looks like / how it should work:

skx@gold:~$ namecheck namecheck
Testing        freshmeat.net - Available
Testing      sourceforge.net - Available
Testing    alioth.debian.org - Available
Testing      code.google.com - Available
Testing     savannah.gnu.org - Available


Name is currently not claimed

I guess it just comes down to having a list of URLs and patterns to test for:

http://freshmeat.net/projects/%s     | We encounted an error
http://sourceforge.net/projects/%s   | Invalid Project
http://alioth.debian.org/projects/%s | Invalid Project

A tool like that would be useful if you're bad at picking names for projects and want a reasonable chance of uniqueness. Looking up sites by hand is just fiddly and repetitive.

Of course "namecheck" is a provisional title; until it may be used to test that its own name is not in use ;)

Today I shall be mostly smoking, drinking, and trying to fix a dying steam engine. Good times.

ObQuote: Brief Encounter

Update: Quick hack: namecheck | future home in ~/bin/.

Tags: namecheck, sift | 14 comments

I installed Debian upon a new desktop machine yesterday, via a PXE network boot.

It was painless.

Getting xen up and running, with a 32-bit guest and a 64-bit guest each running XDMCP & VNC was also pretty straightforward.

There is a minor outstanding problem with the 32-bit xen guest though; connecting to it from dom0, via XDMCP, I see only a blank window - no login manager running.

GDM appears painlessly when I connect via VNC.

The relevent configuration file looks like this:

# /etc/gdm/gdm.conf
[security]
AllowRoot=true
AllowRemoteRoot=true

[xdmcp]
Enable=true

The same configuration on the 64-bit guest works OK for both cases.

(I like to use XDMCP for accessing the desktop of Xen guests, since it means that I get it all full-screen, and don't have to worry about shortcuts affecting the host system and not the guest - as is the case if you're connecting via VNC, etc).

Weirdness. Help welcome; I'm not 100% sure where to look

Anyway, once again, a huge thank you to the Debian Developers, bug submitters, and anybody else involved peripherally (such as myself!) with Debian!

I love it when a plan comes together.

SSL

ObRandom: Where is the cheapest place to get an SSL certificate, for two years, which will work with my shiny Apache2 install?

Somebody, rightly, called me for not having SSL available as an option on my mail filtering website.

I've installed a self-signed certificate just now, but I will need to pay the money and buy a "real" one shortly.

So far completessl.com seems to be high in the running:

• 1 year - £26
• 2 years - £49

For double-bonus points they accept Paypal which most of my customers pay with ..

ObQuote: The Princess Bride

Tags: debian, gdm, meta, ssl, vnc, xdmcp, xen | 9 comments