Older blog entries for gnutizen (starting at number 7)

9 Jun 2003 (updated 28 Sep 2003 at 23:33 UTC) »

Well, those guys at Nullsoft have released another p2p application, W.A.S.T.E., and I've been checking it out. They released the C++ code for it, but I don't know C++, so I've begun hacking out the protocol in C.

The documentation says that, when you have a potential host to connect to, WASTE connects, and sends 40 bytes to the other host, which consists of (if there is no network name), 16 random bytes, as well as 24 more bytes which is a blowfish of the random bytes with a 20-byte SHA-1 of my public key plus 4 pad bytes. If the remote host knows my public key, it sends a 40 byte response.

I used Ethereal to sniff a send to a host running WASTE that has my key to get a 40 byte send. I sent that send to the host and also get a 40 byte response. In the C++ code, I see that g_pubkeyhash is the 20 bit public key hash encrypted, and that WASTE uses (a possibly modified version of) Paul Kocher's Blowfish code. So I'm messing around with this for now. The source I'm reading is in C++, a language I don't know, so I guess I can be forgiven for not totally understanding it. I often wonder how much faster I would be at writing code if I had years of experience under my belt. Much faster, I guess.

Well in the C language Gnutella clone that I'm developing, gnutizen, the SIGPIPE from the send() is now caught (although not in CVS), but now it seems in the same relative place that a SIGSEGV (segmentation violation) happens. Most of what I've read says that the SIGPIPE is fairly normal and catching it, or turning it into an EPIPE with a signal from send - the remote end simply breaks the connection and there's not much you can do about it. The segmentation fault looks like something I have to track down more though.

I had access to systems running Solaris, OpenBSD, and Windows previously, now I have access to a FreeBSD and Linux. I noticed while demonstrating to my friend that there is a problem with the file download in Linux. So that's another bug I'll have to fix.

I got a copy of Microsoft Visual C++ 6.0. I guess it's pretty old (1998). That's fine though, I'll run it on my Windows 98, and get ahold of whatever Developer Studio thing compiles C nowadays afterwards. I ran gnutizen through it and encountered a bunch of problems. It choked on "#include <sys/time.h>", as well as an include of unistd.h. It didn't know the call strncasecmp(). More importantly I made a character array with a constant size of 0 that it barfed on. The others I fixed in CVS, the single character "array" I didn't yet, that's next on my plate but I haven't run into any walls...yet. MSVC++6 has some nice features like seeing what calls are made inside a function, and where else in the program the function itself is called, as well as other features. Once I clean up the array thing I guess I'll see if it has any more useful stuff aside from making solely Windows installation easier (I made a nifty icon for Gnutizen for Windows).

So when I've had time to work on this program since early April, that's mostly what I have been doing...dealing with bugs and making sure things run cross-platform. There's not much point in adding new features if there are big bugs around that crash the program or prevent downloading.

I figured out (with a little help from gdb) why gnutizen was dying after a few minutes of running on it's new high speed FreeBSD and Linux development boxes - during a send it would get a SIGPIPE signal, because it would be writing to a socket/"pipe" after the other end had already broken the connection and was not reading. I put in a hack that catches the SIGPIPE signal, so now the program keeps running. I see that send has flags on some systems that turns SIGPIPE into EPIPE. I can attempt a better fix for this later, for now, the hack is OK. I will put it up on CVS, hopefully tonight.

Ah..my first Advogato posting in over six months...well, I just got access to some more machines, Linuxes and FreeBSDs, and I wanted to run gnutizen on them, maybe even have it run 24/7 on some of them...well, I noticed, now that I finally have my hands on a normal networked Linux account, which box I even have root on, I see that gnutizen does not work once compiled! I tracked down the problem, apparently the byte order/endianess/whatever of the port is wrong, so I did a hack, which is not in CVS yet, and it worked, so that's good. The program seems to die after a few minutes of running on FreeBSD or Linux, as was the case with the high-speed OpenBSD it was on, I'll have to check into that with gdb, strace and all that.

I opened an account on Sourceforge and put Gnutizen up there. Although Gnutizen is functional in terms of being able to connect to, search, find and download from the Gnutella network, we have not yet implemented functions like sharing, traffic routing and so forth, and thus are still in "alpha" mode as we are not ready to be a good Gnutella citizen yet, so I slapped on about 1000 messages everywhere regarding that (on the home page, in the UI, by not releasing the code so it is only gettable by CVS etc.). Of course, developers looking at the code, trying to compile and run it and so forth would be helpful at this point, so I put it up on SF. Mondragon has been going through the code and has made some improvements.

Setting up WinCVS, plink, and all of that was a hassle. Yes I use WinCVS...if Debian had an installer as good as FreeBSD does I'd probably be developing on Debian. Doing an init of the project was a hassle so I winded up init'ing it from my sourceforge shell. After that I got WinCVS/plink working (although I'll have to see if there's a flag for plink to close the window it pops up when it's done, otherwise I get a new pop-up window that sticks around with every commit). Then SourceForge's CVS server went down for about an hour this morning. But otherwise it's pretty good. I'd prefer not to release a tgz of the program before beta but I might wind up doing that. We'll see.

I went to the NYLUG meeting for the first time yesterday. The meeting was two people doing a presentation about PHP. There were over 50 people there but I didn't know anybody, I figured I might see at least 1 person I knew. Nothing I was that interested in, I've used PHP on web sites, with MySQL connections and so forth, but nothing I was that interested in. I've been thinking for a while it might be cool to have a "P2P/distributed computing/whatever" group in New York. Not one that met every week, more like every few months, unless people were otherwise interested. I already know OF some p2p developers that live in New York, although I do not "know" them - that's why to have the meeting though, right? I already have in mind some people I'd invite to a possible meeting in a few weeks (or months), if you live near NYC, and are interested in P2P or distributed computing, and would be interested in a meeting, send me e-mail. My e-mail is at the bottom of my advogato profile - gnutizen at yahoo dot com.

I put up an updated alpha version of Gnutizen - alpha 0.07. I implemented the threads functionality so it can have multiple connections simultaneously. I also put in code so that it can download the files it finds. So now it's actually a useful Gnutella servent, although I have to put in sharing and routing and so forth before a version 1 release, Gnutizen needs to be a good Gnutella citizen (that's where the gnutizen name is derived from after all).

I've been working on 0.08 since. I decided to change from version 0.4 Gnutella handshaking to version 0.6 handshaking. Now I see what version of clients I'm connecting to which is handy, and I also have a better shot at connecting supposedly.

Yesterday and today I've been grafting a Windows GUI onto the program. I see that GTK-Gnutella says it's aiming to separate it's core and interface functions. I've been planning that since day one, as well as being cross-platform, which saves me time in the long run. I never programmed a Windows GUI before aside from a skeleton - I've gotten it so that the connected hosts as well as a tally of how many queries they've seen show up on the GUI though. I realized I needed more room so started looking at tabs or MDI - this seemed like a lot of code changes so I put it aside for today. I like having real-time updates of host connections. In the future I'll probably do an X-Windows GUI.

I just started working on message passing. I'm putting query GUID's and what connection they come from in a table. Soon I will pass on queries, and if queryhits come back I'll look in the table to see where to forward them.

Gnutizen was crapping out on Solaris, but then I saw that my big-endian autoconf marker wasn't showing up. I'll have to fix that. I ran splint (lint) on gnutizen - it marked gethostbyname() as a memory leak but then I saw it was static, so it's not really a leak. Splint found some things to fix though, like unused int's to remove and so forth.

I just released another gnutizen alpha version - 0.06. Yay. I had the coding done and thought I was going to release it but then autotools (autoconf/automake) started barfing on me. So then I had to slug through autotools stuff until I had it ready for dist. It was a pain, but I learned some stuff about autotools and some shortcuts that will make my life easier henceforth so it had a bright side. Aside from the autoconf and automake manuals, there's a book online that is somewhat helpful regarding this.

Anyway, I'm happy that I finished the release. Now I have to decide what to do next. Probably something from my TODO file. There are three things to do next - allow multiple threaded connections instead of one at a time, put caught pongs in an array so I don't write duplicates, and begin work to allow downloading. The beginning of the downloading work will be putting caught queryhits in an array, so in some ways it will be similar to the pong catching. Cool. I'm happy I released 0.06!

Compaq has a program where you can get a free shell account on a bunch of Compaq machines running Linux, FreeBSD and so forth. Sunday night I logged onto a few machines to see if the latest alpha of Gnutizen compiled and ran. I then went back to development and added some #include statements, and changed some stuff around. Monday morning I try to log on - bam , locked out. The second time this happened, I guess some op thought I was up to something. I shot them off a mail explaining what I was doing.

I perused the Gnutella forums. They talked about various stuff, the Savannah versus SourceForge question which I had just heard about (SourceForge's backend is propietary while Savannah's is open). Some Germans are developing a Gnutella server/client called GPU which can do distributed calculations. One of the cool things is the people running it not only do calculations but can send their own out. I had tried it before but it crashed, not only the program but my system. I replied with that and he said he'd fixed a lot of the socket problems up with the latest version. I downloaded the new one and tried it out, it worked OK and I replied he should put some number threory functions. He replied some were already in there, and that anyone could write a module for it. The program is in Delphi Pascal, but he said modules didn't have to be written in Pascal, they could be in other languages like C. I also wrote a post discussing cross-compatibility, and the need to be able to compile on different platforms, specifically that one needed access, or feedback from those who had access to platforms, in order for programs to be cross-compatible as I bemoaned that Compaq locked my account

Well Compaq unlocked the accounts and I tried the compile out. FreeBSD seems to have the same problem with a send() that OpenBSD has. The send() doesn't respond with an error, but dies on a SIGPIPE. The same send() works in a previous version. In fact the same send() works in the current version some percentage of the time. I'm going to try and isolate the error. There's been enough re-writing, and once I get the BSD error knocked out, enough automagic cross-compatibility as well as the program running without major errors to do a new alpha release. I guess I can do that right after I knock out this send() error.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!