Older blog entries for raph (starting at number 164)

1 Mar 2002 (updated 1 Mar 2002 at 06:29 UTC) »
C

Thanks to tk for the response to my inlining query. "static inline" does indeed look like it might be the right answer. I'll dig into portability issues more and let dear diary know...

inkjets

I am obviously very heartened to hear that the Gimp-Print project is having good success with Even Toned Screening. As it happens, I spent some time today tuning the algorithms for a paying customer. The result is here, in case anyone is brave enough to dig through the code.

The biggest advantage of EBS is the new "tandem screening" mode, in which all planes are screened at the same time. If you're making light blue, this means more pixels covered by cyan or magenta dots, and less either white or both. The result is even better smoothness, and also a slightly expanded gamut.

I've looked at the Adaptive Hybrid screening in Gimp-Print, and find it to be quite excellent. In addition, there's been a lot of attention paid to doing the 6-color separation to minimize the patterning from darker inks in lighter regions - it's a delicate balance between those patterns and making the page sopping wet with light inks. The result is that Gimp-Print's 1440x720 mode is overall a bit smoother than my rinkj prototype in solids and gradients. I bet that a large part of the reason why they're seeing an improvement in going to ETS is the inherent advantage of error diffusion over blue noise masks for highly detailed areas such as line work.

Color is a lot trickier. I've found Gimp-Print to be about as well tuned as one can expect from hand-tuning and visual inspection. However, I think moving to "real" color management using a spectrophotometer will result in significant improvements. One area, in particular, that I think will improve dramatically is the consistency between the various modes and resolutions. If you look at Gimp-Print's 1440x720 and 720x720 modes on an Epson 870, they don't match very well. The latter also has some color management anomalies: the transition between light and dark inks seems to have notably less color saturation than it should.

My experiments with Argyll have been very positive, but more work is needed, especially to make the process easier for mortal users (my experimentation involves lots of arcane command lines and hand-editing of raw spectro data files). As is common for me, I feel like I have a pretty good idea how to do it, but it's hard to find the time. Hopefully, we will be able at some point to justify more inkjet work as part of the Ghostscript project, but until then it will go at the more usual free software pace :)

One solution to this problem would be to recruit an apprentice. I'm open to this idea, but haven't done much in the way of active recruiting. It would be a fairly good deal - the job of "color apprentice" would bring with it lots of goodies, including toys (printers and color measuring equipment), a damn good education in color science, help with getting academic recognition for the work (published papers and credit towards a degree), and a modest stipend. If you know of anybody, put us in touch :)

In any case, I'm very impressed with gimp-print's accomplishments, and am pleased that we're able to share code and other results. This is the true spirit of free software.

Python

I'm starting to really like Python. In fact, I've basically decided that all my "fun" programming is now going to be in Python, or C with Python wrappers.

Python, I think, is one of the few languages that lives up to the promise of huge productivity gains. One of the major wins is having lists, maps (hash tables), and strings right there under your fingertips. Another major win is a reasonably clean object framework. Yet another is a sane module and namespace organization.

At the same time, Python still has some serious practical problems. Among them is the fact that there is no one killer GUI toolkit. If you want stability and portability, there's Tkinter, which aside from those two virtues sucks pretty hard. wxPython seems to be picking up momentum, but it's still fairly immature, and it's also not clear that you get as good access to the underlying toolkit when there are three layers involved. PyGtk seems quite cool, but is not exactly portable. There are also the usual fit and polish problems of trying to run Gtk+ apps on other desktops, but those are by no means Python's fault.

Even so, there is no other language out there with a better GUI story than Python's, and some quite a bit worse (cough, Java, cough). With luck, at least one of the Python GUI toolkits will mature to the point where it becomes a truly compelling application platform.

Dare I mention it, but there's also the speed issue. By my quick benchmark, Python is about 200 times slower than optimized C for vanilla integer-and-array work. There's something more than a bit disconcerting about realizing that your 900MHz laptop has just become the equivalent of a 4.5MHz machine. Obviously, if you really care, you code the speed-critical bits in C, but I find this less than fully satisfying.

What I think has gotten me newly excited about Python is the realization that a lot of these problems simply reflect the relative immaturity of the platform, and will almost certainly be fixed over time. The core language is simple enough that it is realistic to expect that it will be implemented at least modestly well. If Python continues to mature as I expect it will, then it will become a powerhouse of free software.

If you're a Lispish type, you'll find this essay to be quite enlightening. In it, author Erann Gat describes how he "lost his faith" in Lisp, and is now happily hacking Python at Google.

One final thought: the primary reason that Tcl became popular is the fact that it was packaged with Tk. The fact that this was even possible at the time is a glowing testament to how far we've come since then.

extern inline

I just found out that "extern inline" is a complete disaster. It's present in both gcc and C99, but with opposite meanings. See the gcc status page, this bug-hurd ml post, this lkml post, and this bug-glibc post for more info.

I really want to have a portable way to specify inlining. Currently, the Ghostscript codebase uses macros heavily, to the detriment of readability and debugability. Inline functions would be a great way to unsnarl some of this mess, without sacrificing speed.

We can't possibly be the first project to have run into this problem. I know the Linux kernel uses "extern inline" extensively, but it's fairly nonportable to non-gcc compilers. Has anyone out there solved the problem of making inline functions a viable, portable alternative to macros? Anyone trying to read the GS sources will thank you!

trust

I went to a talk by John Mitchell on his trust management work on Monday. It's somewhat interesting stuff, but very different from my own work on trust metrics. Basically all of the "trust management" literature is vulnerable to a failure of any single "trusted" node. To me, this means that the security of any trust management deployment scales inversely with the number of users. Since a lot of the motivation behind trust management over simpler systems such as access control lists is to make things more manageable as they scale, I have a feeling that "trust management" will remain primarily interesting to academics for some time yet.

In any case, the problems they're struggling with now are pretty much the same as the ones I struggled with during my internship under Matt Blaze at AT&T in the summer of 1996 - discovering credentials, being able to analyze the algorithms, managing hierarchical namespaces. It's important to publish your work, dammit! Thus, I have some new motivation to finish my thesis.

rebar

I've put the sources up on casper. Run "./rebar testproj/" to test. It will build an executable stored in /tmp. The code should be interesting to read, but it isn't functional enough to use yet. (no pun intended, of course)

Thanks to the people who responded to my post, and sorry if I haven't replied. Yes, one of the key ideas is memoization across invocations. This is indeed similar to what compilercache does, but I believe that the idea goes back farther, at least to Vesta and probably before.

Anyway, it continues to be interesting, and I wish I had more time to work on it.

Ghostscript

I have nice happy feelings about Ghostscript these days. One of the things I realized lately is that a lot of our recent improvements, while not rocket science, are things that will make things much nicer for users. On this list I'd include autoconf, IJS, the recent fix for the bug which caused almost all Type1hints to be trashed in pdfwrite, replacing the completely broken x11alpha device with a mode that does real antialiasing (which has existed for a while in Ghostscript, but was apparently almost never used), accepting more broken PDF's, etc.

Of course, I love the rocket science, and am really looking forward to doing some hardcore 2D graphics work over the next few months, but this other stuff is important too. Ghostscript has long had a reputation for being difficult and unfriendly. I think that's about to change.

jbig2dec

I did some fun work on jbig2dec over the past few weeks, as well. It now passes an important milestone: it decodes some of the freely available test files.

There's a wierd bug somewhere, though. The code exists to decode symbol dictionaries, but on all the test files, it seems to be able to decode a few hundred symbols, then goes haywire. It's very, very difficult to track down: it is impossible for a human to look at an arithmetic encoded bitstream and make any sense of it. I sent an email to the authors of the test files asking for help (traces from their encoder or decoder would help enormously) but haven't heard back from them yet.

Anyway, it's great that the project is live.

rebar

Another really fun project is rebar. Here, the goal is to make the build process for software a thing of beauty. The auto* toolhodgepodge gets the job done most of the time, for most Unix users, but nobody could ever accuse the thing of being beautiful.

So instead of thinking about features and also backwards compatibility with existing things such as make, I'm trying to distill the build process to its essence. One very powerful inspiration is the Vesta project, which takes the position that the build process is a functional program - the arguments are the source files and configuration parameters, and the result is the built package. This is a very different way of thinking about the world than make, which treats the build process very procedurally, basically scribbling in the directory where the source, object, and binary files are kept.

There are a ton of build system designs out there. One of the most important axes is: which language are the build scripts based on? In many, many cases, this language is one that already exists: scons and distutils use python, the original cons uses perl, ant uses xml+java, the original make uses Unix shell, auto* layers m4 and some other stuff on top of the shell, and so on. Some systems actually invent their own little language, such as jam.

What language is best? Obviously, using a real language such as python gives you a lot of power, but I also feel that it tends to bind the specification of the build process too tightly to the implementation. An important rebar goal is to decouple these as much as possible.

So the rebar design seems to be converging on a really simple functional programming language. Lists, maps ("hash tables"), strings, and files are first class data types. Your typical tasks of compiling and linking are built-in functions that map files to files. The language is dynamically typed. The fun part is that the implementation is lazy and implicitly parallel. This means that files don't get compiled unless they need to, and that you can get the equivalent of -j without having to do anything fancy in the build script.

The current prototype implementation is written in Python, and is now capable of interpreting the core language and compiling simple projects, with autogen'ed code and autoconf-style tests, as well. I've implemented -j, but there are two big pieces that haven't been done yet: first, assigning reasonable names to the generated files, and second, skipping the compilation if it's already been done (in a previous invocation). But neither of these is fundamentally difficult.

Python amazes me for its concision. The current prototype is all of 900 lines of code, yet it contains a lexer, parser (recursive descent), core language interpreter, and parallelizing process spawner.

At some point, I'll want to include more people. The free software mantra is, after all, "release early and often". But at this point, I'm much more interested in attracting people with similar goals regarding the beauty of the end product, and the knowledge and experience to understand what I'm trying to do. My choice of a lazy functional language as the basis for rebar is extremely unlikely to be appreciated by the pimply-faced Slashdot hordes, and I don't want to be deluged with "how to I get program X to compile with library Y on Z Hat Linux?" questions.

Friends and family

I spent a few hours on Monday with zooko, his wife Amber, and their child Irby. It was great fun - we talked about lots of interesting things. Heather and the kids got a chance to meet them as well, which was great.

Alan continues to thrive on the Singapore Math workbooks. Reading them over, I can see why; they're really well done. I can recommend trying them to any parent of a gifted child.

Max's language development is proceeding at an incredible pace. A few weeks ago, I noticed that he was conjugating verbs: "jump, jumping, jumped". Over the past two weeks, he's mastered possessive ("Maxie's, Daddy's"), compound nouns ("barn owl", as opposed to simply "fire" for fire engine last month), complete sentences with subject and object ("put it here"). And he takes such delight in mastering every new thing!

Max is also really into Blue's Clues. I noticed today that one of the videotapes is starting to wear out. It's an interesting show for a "grow-up" to watch, because most of it is animation that's been composited with footage of just Steve, alone, in front of a bluescreen. (who's creen? blue's creen!) You imagine Steve staring and talking into space, pretending that's there's actually a blue dog, a talking salt and pepper shaker, and similar other stuff there. He does a good job of it, though :)

Found at the always interesting Hack the Planet: an essay by Paul Graham (of "On Lisp" fame) on taste.

12 Feb 2002 (updated 12 Feb 2002 at 08:29 UTC) »
Font

Not to keep people in too much suspense, the missing glyphs are J, U, W, and w. nomis came closest, and I tend to agree - the U could stand a little improvement.

In any case, it's deeply satisfying to hack on this font.

Singapore Math

Heather got the Singapore Math workbooks for Alan, on the recommendation of several other parents of gifted children. Well, for whatever reason, he loves them. This evening, he used wanting to do more Singapore Math as an excuse for pushing back his bedtime. I felt I was in one of those cheesy infomercials.

GPL viols

Now I've experienced firsthand the unpleasant, but all too common, practice of repackaging free software as Win32 shareware. This might even be legal if it were done with some care, but apparently the culture of scam artistry is too deeply ingrained for people to even try.

Yes, it pisses me off.

/.

The recent thread on "what makes a powerful programming language" was one of the dumbest ever. Characteristically, the dumber the article, the more response. This phenomenon isn't unique to Slashdot, of course, but they do seem to choose this path fairly often, or else be awfully sloppy. (and what's the difference, really?)

Of course, everybody still reads Slashdot. One of these days, I'm going to get around to tweaking Advogato to try to encourage many more people to post news articles. Maybe there is hope for a reasonable alternative.

CodeCon

It looks fun, but I'm not sure I'll have time to attend. It is disappointing that the Chord guys don't seem to be represented. That's without question one of the most interesting things happening in that space.

LeBe

I've always wanted to design a font. That's actually why I started gfonted. That project is now dormant, but it spawned gdkrgb and libart, so it was certainly interesting.

Anyway, when pfaedit crossed my radar screen, one of the first things I tried is tracing the font a specimen sheet I had fallen in love with. The program is crude, let me tell you, but strangely usable anyway. Before too long, I had all of the letters in the original sample, uppercase and lowercase, drawn.

There are some letters which are standard in the alphabet now, but weren't standard when the specimen sheet was printed (only a bit more than a hundred years after Gutenberg). I would have to draw these myself.

So over the past few days, I did. I'm pleased with the results. None of the people I've showed it to have been able to guess which four letters I drew from scratch. Can you?

Assemblies

I had a dream last night that I was asking miguel about assemblies. Inspired by the dream, I stopped by #gnome and had a very nice chat with miguel, bjf, and others about the topic. I still haven't found a good document that explains them - bjf's advice was to play with Microsoft's implementation. In any case, I'm convinced that there are some very good ideas there. Most of the time, when we build larger systems out of components, we just throw the pieces together and cross our fingers. Sometimes it works, sometimes Murphy's Law reigns. The idea of assemblies is apparently much more systematic.

As an old-timer, the name tickles me a bit. The name "assembly" used to refer to assembly language, which is basically obsolete these days, so I guess Microsoft decided that the slot in the namespace was free. In any case, it evokes a much older time.

Miguel is certainly right about one thing: Microsoft has a lot of very smart people who are paid to think about difficult problems. To ignore their work, for any reason, is stupid. Embrace and extend!

jbig2

Jbig2 implementations are starting to dribble out into the real world. We had a customer file come in recently that was created by Cvision's Cvista product. I hacked around a bit yesterday on jbig2dec, which is our (GPL) decoder project. It's fun stuff, and I hope we decide to ramp up the development on it. rillian is the main developer on it, but he hasn't had too much time for it either.

7 Feb 2002 (updated 7 Feb 2002 at 07:22 UTC) »

Anyone who is alarmed at goingware's latest diary entry is advised to take a close look at the URL.

Nice spoof, though. I'm kinda planning something like this for 4/1. bbs.porncity.net indeed.

The rest of my diary (worth reading imho) is here

6 Feb 2002 (updated 7 Feb 2002 at 00:04 UTC) »
Ghostscript

Well, there are a couple of new releases of Ghostscript out now, for both GPL and AFPL branches. There's a fairly significant security fix. You should upgrade.

These releases also contain IJS and autoconf. I haven't gotten much feedback yet from actual IJS users - I know this stuff takes time.

Mono

It's really good that we're having this flamewar^W discussion. I can definitely sympathize with the desire for better languages and tools, but adopting new stuff being driven by a highly proprietary company comes at a significant cost. I am not convinced yet that C# is dramatically better than choose-one-of C++ and Java. Add to the mix the fact that both languages are at least modestly mature and have half-decent implementations.

In a few years, when C# is also modestly mature and is half-decently implemented, it might be worth reconsidering. Of course, by that time, C++ and Java will be a lot more mature and well implemented :)

I found this article to be quite good, providing solid technical refutation of the clr's language neutrality claims. Despite the javalobby.org url, I found the article to be not especially biased towards Java either.

C# ultimately has the same problem as Java: the standard libraries are dictated by a proprietary company. It's very likely that Microsoft won't fuck them up as badly as Sun has, but it's also certain that the standard libraries won't be tuned for the needs of free software developers either.

SVG

The ksvg project is starting to come along pretty nicely. They use libart for the rendering, which is a good choice :)

It's something of an NIH project, given that there's a reasonable start at an SVG implementation in Gill and rsvg (which are based on the same codebase). There are quite a few things that rsvg does correctly (like opacity in groups), but ksvg doesn't do yet. It's sometimes irritating to see people struggle with problems you've solved a long time ago.

On the plus side, the ksvg developers have the Gill/rsvg code available as a reference, and also frequently invite me to their irc channel. NIH does not always mean a total waste of duplicated effort :)

Rebar

I've been hacking a little on a project which has been dormant for a while: rebar, my NuPerfect autoconf/make replacement.

Rebar is, of course, also very much an NIH project, given that the autotools teetering-house-of-cards already exists (the word "toolchain" conveys much more dignity than auto* really deserves), and there are plenty of other projects aiming to replace it as well. I'll try my best to read and understand the related work before releasing rebar upon the world. So far, my biggest inspirations have been auto*, Vesta, Python distutils, and jam. I've also looked at Ant, SCons, and OS X's ProjectBuilder. There are a bunch more I need to study, I know. At the top of the list is .NET's "Assembly" mechanism. (note: does anyone have a reference to a reasonable document on this mechanism? thanks in advance)

In any case, the current rebar snapshot currently compiles a hello world example :)

Thesis

I've also got a bit more written on my thesis. It's challenging finding time for it on top of everything else, but I'm determined to finish it sooner or later.

IJS

I just posted a snapshot of the new Ghostscript IJS client. If you have a printer with an IJS driver available, I encourage you to test it. Currently, that's HP printers, and you can also try Russell Lang's patch for gimp-print.

In other news, hpijs is now free software, under an BSD-minus-advertising license no less. This is very cool. I provided some nice quotes that will probably be used in their formal announcement.

Microsoft and security

The comments to the recent Slashdot story that Microsoft was going to focus on security had a childish, bickering tone. Fact is, this is a good thing. Microsoft's weakness in security has caused a lot of suffering, and it's just luck that it hasn't been worse. The free software community can take a lot of credit, I think, for inspiring Microsoft to take this initiative. Now they can chase OpenBSD's taillights for a while :)

NIH

NIH is just as rampant in the free software community as anywhere else, if not more so. The threadlet here on vector graphics libraries is certainly a case in point. I hadn't heard of the anti-grain geometry library until now, but it looks vaguely interesting. In any case, I guess a lot of times it's simply more fun to start a new project of your own than to contribute to one that already exists.

I'm not sure whether this is a good thing or a bad thing. I think NIH is responsible to some extent for the relative abundance of half-finished components, and the relative paucity of finished systems. If we're to improve on this, I think it would be a good idea to think about how to make collaboration more fun.

Life

Life is good!

Another quick update:

I just finished reading The Brothers Lionheart to Alan aloud. I broke into tears while reading the final chapter. This isn't something that happens a lot, mind you.

It's preposterous that one of the most beautiful children's book ever written, by a well known author no doubt, is not in print. Someone has posted a text copy online, and I consider this to be an "ethical copyright violation", specifically that the public good of making this work available outweighs the loss of incentive for copyright owners to reap profits from their legal protection, in this case utterly squandered.

You may find that the above link fails. Fortunately, archive.org preserves the text, though the broken HTML format makes this version quite unusable. Even so, it brings up a question: can archive.org continue to exist in this age of huge liability for copyrights?

Ghostscript

We're closing in on a new trio of releases. For one, I have the gdevijs driver successfully printing with hpijs 1.0 server. I'm looking forward to getting this into people's hands.

The free software community

I'm feeling more positive about the community than I have in months. Of course, there's a lot of stupidity and childishness, but there's also some serious wisdom, and some good work being done. I want to write about what I see, but my thesis comes first.

155 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!