Older blog entries for Stevey (starting at number 108)

hank

 I like your idea of a good visualization tool for duplicate file finding.

 As you might have seen from my recent diary entry I spent a while working on a quick and dirty script for finding duplicate files.

 I'd love to see a screenshot if you could dig one up - as I have a hard time imagining a useful GUI for such a tool.

 Finding duplicate directories might be simple, but displaying partial duplications seems tricky to me - maybe I just don't have the eye for it.

XSS

 Spent a while investigating online presentation systems recently for managing a new website in a collaborative manner.

 I narrowed down the list of systems to a couple - then went looking through the code to see how secure/paranoid/flexible each one was.

 (Due to my mistrust of such systems - How many times have holes been pointed out in PHPNuke et al?)

 Depressingly in both cases I found exploitable weaknesses. To my shame I tried to demonstrate one in a non-malicious manner after the author(s) didn't seem to understand what I had discovered and reported ... it went wrong. The main site was borked for around 15 minutes.

 I guess there's a good side the admins now spot the problem, but the down side is that I may have inspired evil people to take advantage.

 It was a genuine error for which I can only apologise profusely; in my investigation I hadn't realised quite what effect I'd have.

 Ce la vie ..

 Based on early responses the sites/packages will both be fixed shortly so a "Name and Shame" is inappropriate - but I'll document the flaws which might encourage other authors to take more care and be more paranoid in the future...

Hacking

 Nothing much to report - I wrote some quick and dirty scripts today to find duplicate files as I'm bad at organizing MP3's.

 First we scan through a directory, recursively, writing out a temporary file containing MD5 hashes and filenames - then we use that to find duplicate files.

 Handy, but messy.

Security

 It looks like I was responsible for the following two Debian Security Advisorys:

 (Details here, and here respectively).

 I'm such a naughty boy ... ;)

salmoni - No I guess all cats are perfect, although some are more perfect than others ;)

Advogato

 I spent an hour or two working on the Advogato codebase last night - adding support for Article Editting.

 This isn't complete yet because I'm having issues with the way that articles are posted. What I have is an 'Edit' link displayed next to an article if you're the author.

 The edit link brings up the article, preamble, and title in a form which you may edit and submit.

 This is where it goes wrong - when the form is submitted a new article is posted with the changes applied. What should happen is the old article should be updated. I'll deal with this tonight if I get time.

 There are other issues to deal with - such as the forking of Advogato. There are many different versions of the code now. I think I'm correct in saying that Steven Rainwater's version is the most up to date - but there are different fixes and changes in each version.

 I have packaged a copy for Debian which is pretty standard with only the addition of my password emailing patch and no other changes. (I can make the .deb file available to the world if theres any interest - I didn't do this initially to avoid poluting the world with yet another codebase).

 The article editting I started just for fun - if it's complete I've no idea what to do with it. Keep it to myself? Add it to my .deb?

GNUMP3d

 Over the weekend I changed a lot of things in my MP3 streamer, rather than reading the tags from each audio file as needed there's an indexer script which builds up a database of all the files and tag information.

 This "database" is used throughout the code which provides a huge performance win - at the expense of potentially out of date information.

 So far I'm assuming that the indexer will be run from cron, but I'm experimenting with the auto-rebuilding of the index whenever the machine is "idle"... We'll see how that goes.

 A new release is going to arrive soon - I'm determined to get it out before I drop offline during my housemoving.

13 Jun 2003 (updated 13 Jun 2003 at 23:30 UTC) »
Article Feedback

 I've been swamped with feedback over the recent article I posted, both as comments and as personal mail.

 Firstly thanks to everybody for taking the time to offer their thoughts, and secondly sorry for my delay in replying to you all.

 Partly I wanted to digest all the information, and partly because I'm just gearing up to move house. Three weeks and counting!

 Two quick non-specific comments:

  • It looks like an idea repository appears to be a useful idea; but the site would have to be very flexible to be worthwhile.
  • I'm suprised lots of people jumped upon the Exchange mention - it was just a random example of a piece of software that I would like to so, not something I want to write myself. (To my mind any exchange replacement will not be a PHP groupware system, that may be great - but if it's not a server that an unmodified copy of Outlook can connect with it's not an exchange server..)
Coding

 I've been writing code sporadically recently, working upon my MP3 streamer and trying to get it to pre-generate a cache of all the song tags.

 This is a fairly nice thing to work upon, it's simple, self-contained and doesn't involve any tricky coding. A job that I can divide into discrete parts and implement and test fairly easily.

 Hopefully this will be complete over the weekend, subject to my erratic sleeping pattern.

Life

 Did I say I was moving soon? OK I am, and I'm alternating between wishing it wasn't necessary and hating the fact it's not happening yet.

 The reason for the move is entirely my own - until last September I had a cat who lived in my flat with me.

 I've always been a cat lover, and they really do suit my moods and temperement.

 However I grew up spending my summers upon my grans farm - surrounded by Alsations.

 When I lost my cat I was determined that he couldn't be "replaced", and I had the thought of making my next pet a dog.

 My flat being too small to lock a dog inside alone whilst I work I decided to move house - taking advantage of the fact that the Edinburgh property market is obscene, and I could double the money I'd paid for this property only 3 years ago.

 So now I've got a nice new flat all lined up - top floor overlooking a park. Twice the size of my current place and very very sexy.

 As I'm getting closer to the entry date I've been looking at lots of 2 or 3 year old Alsations and Labradors .. but I'm increasingly thinking cat-ness is the pet for me.

 Wouldn't I feel stupid essentially having moved for no reason if I end up with another perfect cat?

Advogato

 I posted my second ever article - this was really the result of some discussions in the pub with some friends over the weekend.

 Be gentle with me ..

Telephones

 The company I'm working for is looking to upgrade the telephone system it uses, as sysadmin I'm in charge of this by default.

 Whilst networking is interesting I find I'm learning more than I really care to about telephone systems, voicemail, ddi, and switchboard systems.

 The first presentation I received was interesting, I felt curious about the technology involved to route calls from the exchanges down to our switchboard and from that to our individual telephones - but now I'm getting blase about the whole thing.

 I can't help thinking this is a bad thing in a way that I cannot articulate appropriately. I know that I'm not turned on by all technology, and indeed I'm not a fan of the telephone in general - but even so I feel I should be interested.

 Maybe it's another artifact of my increasing sloth and apathy... as the temperature rises every summer I get less motivated and more sleepy. Always.

 Wintertime is my time .. in the same way that night time is mine.

 It's at times like this I think I should have continued spending my life working in nightclubs; lets face it it's got appeal:

  • Beautiful people (tm).
  • Free beer.
  • Keeps you fit (lifting all the crates around).
  • Surrounded by your friends.
  • Good Music

 To be fair I know I could never really go back to it, and there were downsides too - being busy from 10PM-5AM every Friday and Saturday puts a damper upon your social life - and dealing with drunken people doesn't make you feel good about the human race, or customers in general.

 Life's not so bad .. I just need something to motivate me - either computery (see that article I mentioned) or outdoorsy.

 (Either that or jwz should offer me a job in his club ;)

groom

 That's a neat POP3 library, any thoughts about IMAP access too?

 One minor quibble: Your sample code contains a couple of exploitable buffer overflows.

 I guess this isn't a huge problem as the code is clearly sample code, and isn't installed setuid() or anything silly like that. I just have a reflex action of trying to persuade people to always be more careful...

int main(int argc,char** argv){
        int mysock;
        char myservername[64];
        char username[64];
        char password[64];

// Time passes ...

strcpy(myservername,argv[1]); strcpy(username,argv[2]); strcpy(password,argv[3]);

// Thorin sits down and starts singing about gold ..

Rating

 chalst: I don't regard the ratings of diarys as anything more than how interesting I find them.

 This means I rate people higher if they cover things that interest me, and down if they have wildly heretical views.

 As it's used to filter the entries I see this is a fine way of using it; but I don't think that article rating should be based upon interest.

 They should be based upon the quality of the writing and their appropriateness in this forum...

Gifts + Clothes

 Discovered that I'd had some new gifts donated from my wishlist yesterday which was pleasing, especially as this coincides with the delivery of some new t-shirts.

 I like to design my own t-shirts, and this week I've recieved five new ones. Simple. Plain. Each bearing a day of the week upon the chest.

 Today I'm being subversive, wearing 'Friday' despite it being only Thursday... :)

 I wonder how many people here wear jeans + t-shirt to work every day, as I do..?

Updated Articles Idea
johnnyb - yes I agree. Sorry to be so unclear.

 Clearly the articles must be in some public area so that they may be voted upon. Comments however shouldn't be allowed.

 The tricky part is deciding when an article has received enough votes to "go live".

 Either you have to decide there must be some arbitary number of votes - of which above half must be positive, or you introduce weighting of positive and negative votes.

 I don't think it's reasonable to weight the votes by seeing who they are from...

IMAP

 I have my remote mailbox stored as IMAP, which is filtered at the server side with procmail.

 When I login I run 'mailstat' to see how many messages I have in my folders, but this doesn't work remotely, and mutt doesnt allow me to see the mailboxes with new mail.

 So .. we have a new tool imapls for listing total and unread mails in each mailbox to which you're subscribed.

 It's a small perl script, and considering it only took an hour or two to put together I like it already, maybe somebody else will too?

2 Jun 2003 (updated 2 Jun 2003 at 23:25 UTC) »
Articles

 I agree with raph that the most recent article wasn't appropriate for the site .. but it's hard to see how this situation could be improved.

 My initial thought was to have the articles only visible to people of "Master" rank or above(?), at which point it could be rated.

 If there were a voting that was positive, in some way, it could go live.

 This is similar to the way that I belive Kuro5hin works though I've never studied the mechanics - and vaguely Perl Monks-like.

 Sadly despite my hacks on Advogato's code, (probably causing more harm than good), I'm still constantly confused by the multiple repositories and the XML stuff.

 Every now and again I think about writing a similar construction in Perl which I could understand - but that would be a worthless duplication of effort. If it worked it would steal viewers from here, and if it didn't without an audience it'd be pointless.

 Never enough hours in the day .. that's my trouble!

MPIO

 The MPIO is a simple parallel port MP3 player with 64Mb of onboard memory.

 Once upon a time I owned one, and couldn't find any driver details. Eventually I got it working under Linux, but it was a nasty hack.

 I'd forgotten all about it, since being given a Muvo, (usb based), player which required no effort to get working with Linux.

 Yesterday I received an email from somebody who'd done a Google search for details and come across my Advogato diary describing my progress.

 He asked, politely, if I could share my code - but sadly I could not... Having changed computer in the meantime I was sourcless.

 Nonetheless I remembered an Australian site that contained some code.

 In case anybody is in a similar situation again:

 It's interesting that the code talks about a device called a Jazz Piper; but it works perfectly with the MPIO device.

 (I guess for completeness MPIO USB Linux Driver is held upon SourceForge).

Google

 The power of google to allow people to help themselves never ceases to amaze me.

 It was only a couple of weeks ago that I got an email from somebody in Russia who wanted to know about the patch I'd made to allow my Logitech PCVC720K/40 camera to run under Linux.

 He had the same model, so I shared the code.

 I feel humbled by the sheer size of the world, and the unlikely coincidence that one person working alone can produce exactly what somebody else in another continent desires, and they can both find each other by magic.

Code-Fu

 I've not done much real work recently, I toyed with packet capture and HTTP sniffing. Nothing really new was produced - A C++ Plugin based network sniffer, with a couple of plugins for displaying POP3/FTP/CVS logins, etc.

 The intention is to handle form POSTS and attempt to analyse their contents for passwords - but I've lost interest for the moment.

 From the mails I've received I (perhaps unsuprisingly) appear to have only received interest from the blackhats. That's enough to make me feel slightly uneasey.

99 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!