Older blog entries for Artimage (starting at number 3)

Obligatory statement: It has been a while since my last diary entry.

Mnet

We are dangerously close to releasing 0.5.1-Stable. I did some one line Perl chicanery to parse the CVS logs and make a changefile. Basically this needs to be released so we can start in on the fun stuff. v0.6 has a totally new block fetcher that is far superior and it will use distutils. If those two major advantages don't turn you on then how about Zooko's new hack that will allow people to hide their IP behind a relay. In theory this gives you the same anonymity you get using something like anonimizer. (In truth this is not true because the MetaTracking system we use, but it is a pretty good start.) As for me, I am re-working the bootpage loader code, not very sexy, but it will help me get a better feel for EGTP, Mnet, and the MetaTracker system.

I also took a look at khashmir which is a distributed hash table library of the Kademlia flavor implemented in Python (Definition stolen from SF page.) It looks like Drue, its creator, is going to port it to Twisted reactor async io core. If this works out then, like EGTP, it should be able to route past Firewalls and NAT's using relays in a dynamic fashion. That is pretty dope. Its big advantage is that it is completely decentralized, unlike EGTP which uses a semi-centralized MetaTracker system.

The Bunker

Not much to tell really. It looks like Lock, my friend who works at Dell, has tracked us down a 1u server. I believe it will get shipped out to England some time next week. Woot!

Job

So I am flying off to Boston again on Tuesday, this time sign the contracts making me part of the working world again. Though I am really looking forward to starting to work again, not to mention being paid, I will miss having unlimited time to code my own projects.

Randomness

So EGTP gives this warning under windows: This OS needs a better source of entropy (or something similar). So I thought hey, this is something I should look into. Google, Oracle for all things geeky, of course shown a light on the path of wisdom. This post to a mailing list seemed to have a cookie cutter answer for what I was looking for. (Side note: this should work on all windows with IE3 or later, or 95 OEMsp2 or later.)

So I quickly got it working, and spewing out nice random output. But I wondered how good the output was. Which led to David Wagner's page on randomness. What a treasure trove this is. So I grabbed Diehard, a series of tests for RNG's (Random Number Generators), and read up on how to use it. Thirty minutes later I have generated an eleven meg data file for it, and it is churning away. (Small side note, due to an error in the DOS version I was forced to enter the filename each time.)

This turned out to be disappointing, though not surprising. Since I was really using code to grab a seed for an RNG, it turned out the data wasn't passing the tests. Which means what? I am not sure. I know that on linux /dev/random will run out if you if you don't give it time to replenish. The cookbook answer didn't tell me how to check to make sure my data was good. In the end, this is still better than the way we were doing it. (Using time and mouse position. This short post by Don Davis tells why mouse position is not a good source.)

With all of this done, I talked to the other EGTP hackers about where to put my code. First reaction, add a patch to crypto++, Wei Dai's C++ crypto package that we use. Now, if you have been following my links, you will notice that the first recipe I found was also posted by Mr. Dai. Wouldn't he have used that snippet in his own library you ask? Oh, yes, he did. But when the code was written it either wasn't there, or didn't get researched. Wei has a great class called AutoSeededRandomPool that plays nicely with /dev/random (Linux), /dev/urandom (*BSD), and Windows.

I talked with Zooko, and it turns out that embedding C++ code into python isn't the easiest thing for a Python novice to be doing. It will be much simpler to just create a small file that simply does our entropy gathering. So that is what I will do now, and we can come back and do a major overhaul later.

Anyway, I have got the code compiling and working in Python using distutils. There is still some testing ahead, but it feels good to have gotten it working and to have a better understanding of distutils and the C/Python API.

Job hunting

I went to Boston on an interview yesterday for a consulting firm. I have never thought that consulting was up my alley, but after talking to them I must say that I am quite excited by the prospect. They all seem smart and the work seems interesting, I have a good feeling about it.

EGTP

I didn't get to code at all yesterday, and today I have just been futzing around. I wrote my first python class, a dummy verify function that always returns true, and used it to fix the calls in TristeroLookup. Of course it's really trivial, but it seems to work. Now if I can just figure out why the local tests are failing....

14 Aug 2002 (updated 14 Aug 2002 at 16:28 UTC) »

So this is my first entry here. I was very dubious of this entire blog thing until I started reading them. Now I must admit, I am hooked.

EGTP

Two days ago I finally got EGTP to compile on Windows. There were a bunch of little issues with Makefiles and directories, as well as one 'bug'. It turns out that you need the b flag (for binary) when dealing with windows but not with Unix. So the code worked on Unix, but under windoze it broke. The lesson here is to never leave Unix. :)

I am now trying to get the tests to work. This is a bit hard since I really don't understand ... well.. there are a lot of things.

  • Python - I am trying to learn Python. It still surprises me when it doesn't act like Java or Perl.
  • P2P concepts - Distributed programming is not the same as client server. I keep rediscovering this fact.
  • EGTP/MNet - There isn't a ton of documentation on how things are supposed to work or why they are designed the way they are.
  • Windows - I miss Unix.

As you can see, I am in a little deep. But the people on #mnet have been really helpful.

The Bunker

Hopefully we will have our 1u shipped to The Bunker early next week. I am looking forward to moving to a more reliable mail host. I am also going to put up a projects page there for myself.

WishList

Last week I re-engineered the Amazon Wish List parsing Perl module and sent the diffs to the maintainer. He had been using a regexp that was just huge. I decided to use the Perl package HTML::TokeParser. It is smaller, and each regexp is much easier to deal with. Now I need to start saving the data to XML. In the end I want to be able to track prices in Amazon. When a price drops by more than 10% I will send an email. Kinda like a stock watcher.

Remailers

I talked with Len Sassaman at Def Con. He wants me to put a remailer in The Bunker. He actually knows one of the founders and is talking to him about it. (I know the other founder.)

During our chat he brought up the fact that most remailers are run either in the US or in Germany. I find this to be very interesting. Apparently, because of the Nazi's and their Gestapo Germans are very careful about privacy. The argument often heard here in the US: "You only have to fear the government if you are doing something illegal" just doesn't wash in Germany. I mentioned this to a friend who pointed out that we had McCarthy and his communist witch-hunt. Why have we forgotten this lesson so soon? And what about Nixon and Watergate? John Ashcroft scares me more than Iraq.

Anyway, back to tech. This led me to look into where remailers are. I found The Remailer Geographical Mapping Project, but they don't have an actual map. So I did a bit of looking, and there are some Perl packages that will do just fine. One that will generate a map with points given lat/longitude and the other that will return a lat/longitude given a postal address. So I want to put that up on the bunker as well.

Wow, this has gotten pretty long. I guess I should go code now. :)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!