Older blog entries for niksilver (starting at number 27)

We're looking for another developer to join our open source project, Jtrix! If you know of a great Java developer, then ask them to contact me at recruitment@jtrix.org with a plain text CV. There's a full ad on our site. An in-depth knowledge of Java (not J2EE, but the real Java J2SE and the JVM) is very important.

In other news, much of the work started before the holidays is coming together. And I need to ensure that it's okay to run a GPL'd program on Jtrix, which is LGPL'd. The idea is that Jtrix is an operating system, like Linux, so the licencing of the program you run on Jtrix should be irrelevant, just as you can run GPL'd, LGPL'd and proprietary applications on Linux.

Some discussion yesterday about documentation. All the usual things, but with a twist given by the open source nature of the work. The question is how should we document a sub-project or other piece of work in such a way that is reliable (doesn't get out of date) and leads new developers (or interested hackers) through how things work.

The answer we arrived at is to use the magic of wiki. The jtrix.org site uses devwiki to help contributors manage project pages. We'll use this to post rough plans of work *before* we embark on it. It's easy to access and easy to update as the work proceeds. Of course, there are other levels of granularity to. After a wiki design/overview doc comes Javadoc, and then comes reading the source itself (last resort!). But wiki is a good step from very broad concepts to Javadoc.

We'll try it out first with our hosting efficiency project.

Just started on a mini-project to get the hosting service, Hospitality, to scale up.

The hosting service allows a user to launch their code into a remote location --- the remote location runs the hosting service. Of course it's all secure and audited. You've got to have a right to run your code there, and it's all nicely sandboxed when it gets there.

The problem is that one hosting service may span several servers ("nodes"), and the more nodes, the longer it takes for them to communicate with each other. The efficiency gain we're aiming to introduce is to control one node's view of another, so not all the data comes down the line, and to have tools to select data segments by particular criteria.

The other advantage of this system is that we expect to apply it to other services, such as the Indexing Service. Should be done by end of Jan.

As always, the theory is right if you can apply it right. Our (well, Filippo's and Andy's) extremely fine-grained planning of the Mediator is proving to keep Filippo on track.

To summarise: the Mediator is a central part of a Jtrix node implementation. It prevents code sharing between two netlets which use each other's services, but still allows them to communicate as if unhindered. It's magic (almost). It is also fiercely complicated, having to ensure that one netlet's dodgy coding doesn't bring down the other, ensuring you can safely run untrusted code, and so on.

Filippo and Andy worked out a required rewrite to the half-day level of granularity. It took two days. But it's keeping Filippo on course. Contrast that with any other work in the same area which has gone over time by approximately three-fold. It's early days, but the moral of the story is that good planning, if you can bring yourself to do it, pays off.

Fingers crossed for the rest of Filippo's work.

Phew! Off to Devon (Southwest UK) to meet with the bosses. Jim, our CTO, had this rather nice idea of how to help open source people... the problem is that although people want to help, taking on a big task is daunting, especially when there are as many new concepts to learn as in Jtrix. Our roadmap contains lots of really excellent things, but they're quite big.

Jim suggested we write these things very roughly ourselves so they kind-of work, and where you can see what needs to be done very clearly. Then a developer can take this and improve it. Everyone like to design, but this is a great way to get started and still have plenty of creative input.

I wonder what other people think of this idea...?

More joy, as we have recieved our first submission of actual Java code. It extends the help system on the launcher to include full pages of text. A small thing, but not only is it a great help to people, but it's been written in a simple but extensible way. It shows that input from outside can be very effective - I don't think we'd have thought of that technique, and it's a good idea to remember. We've incorported it into the main codeline and it will appear in the next release.

Well, we finally got 0.14.0 out the door. Quickly followed by 0.14.1 and 0.14.2 as we found some very minor errors. That was Thursday. Today we've discovered another small problem and hope 0.14.3 will be released shortly. However, all these are minor in comparison to the achievements made by the team. The list of new features is impressive.

The main changes ran deep (hence more than a month without a release) and mean that a single application can run across many machines, even though they may be owned by different providers. So this means I can buy hosting from two companies independently, and have my application run on *both* of them simultaneously. A giant blow for small consumers! No more monopolies! As soon as one company increases their prices too high, I just shift the balance to the other company. And no downtime.

On other fronts, there is increasing interest among open source people about integrating SOAP into Jtrix. This really is open source at its best - people want it, they can add it. And we'll help, of course.

Yet more Windows test failures. All trivial, but it shows to go you that testing must be thorough. About 80% of our user base is Windows, judging from incoming mail, and 30% are solving the same problems. Which takes us back to the testing.

Another issue is that so many corporate firewalls restrict their users from accessing non-Web ports (!=80), even if the connection originates from the inside. One of our developer community is working to solve this, giving Jtrix arbitrary network protocol options (SOCKS, SOAP, etc). But even if Jtrix can manage it, they still cannot access CVS and so on.

Today I hope to continue my documentation of the Jtrix mediator and threading, which I find endlessly fascinating, believe it or not. But then I used to find database schema design endless fascinating, too...

Deep joy. I posted a slashdot comment about the Anthill project and got moderated up to 3. Just imagine... if I'd added some irrelevant and uninformed words I might have got to 4 ;-)

Two new developers started yesterday, one recommended by MikeCamel to work on migrating our Web site into Jtrix. There is much to do here, but mostly (hopefully) it's about learning our JSPs and servlets.

Our release of 0.14 continues. Some basic tests are failing on Windows. Nothing fundamental, just paths etc, but it all needs checking and correcting.

Stuggling yesterday with Java threads. Although I'm starting to understand the details, I'd like a nice visual way of representing the thread/object usage in your program and being able to spot if and when deadlocks occur. For example, I have two threads which call methods on the same object. If A is called before B then it's all fine. If B happens to be called before A then it deadlocks. I should be able to draw out my program and say "Aha! I can see that in the following circumstances I have a problem...".

Sadly it seems this is a universal problem. Found a couple of ways of representing threads formally, courtesy of Google. For example this project at UKC which includes this paper. But a formalism is far from an at-a-glance aid.

The reason for all of this is that Jtrix has some pretty sophisticated threading tools, and I need to add discussion of them into the "How to write netlets" document.