Is there an existing system which will allow me to query Apache logfiles via an SQL string? (Without importing into a database first).
I've found the perl library SQL::YASL - but that has a couple of omissions which mean it isn't ideal for my task:
- It doesn't understand DISTINCT
- It doesn't understand COUNT
- It doesn't understand SUM
Still it did allow me to write a simple shell which works nicely for simple cases:
SQL>LOAD /home/skx/hg/engaging/logs/access.log; SQL>select path,size from requests where size > 10000; path size /css/default.css 13813 /js/prototype.js 71261 /js/effects.js 37872 /js/dragdrop.js 30645 /js/controls.js 28980 /js/slider.js 10403 /view/messages 15447 /view/messages 15447 /recent/messages 25378
It does mandate the use of a "WHERE" clause, but that was easily fixed with "WHERE 1=1". If I could just have support for count I could do near realtime interesting things...
Then again maybe I should just log directly and not worry about it. I certainly don't want to create my own SQL engine .. it just seems that Perl doesn't have a suitable library already made which is a bit of a shocker!
Tags: accesslogs, apache, lazyweb, perl, sql No comments