Advogato blog for emk

27 Apr 2004

Tue, 27 Apr 2004 03:20:27 GMT

Finally, a working GDB. After much pain and suffering, I've upgraded to GDB 6.1, which appears to be reasonably compatible with gcc 3.4. No more bizarre RTTI errors, and symbol lookup is now blazingly fast. Cool.

Precompiled headers with GCC 3.4. Microsoft is still ahead here, I think--even with precompiled headers and a well-designed header layout, GCC 3.4 build times are painfully slow compared to MSVC 6. (I'll know about MSVC.NET when port this new code over.)

It seems that GCC does a lot of work at template instantiation time, which means that every std::map<T> in your non-precompiled headers hits you where it hurts.

C++ feelings. My love-hate relationship with C++ has been leaning towards "hate" recently, as I've been doing lots of complicated data structures and learning where shared_ptr just doesn't cut it. But I've laid the groundwork for a simple garbage collector (which I needed to write anyway, for use with the code I'm generating), and I feel about 500% better about my code base.

19 Apr 2004

Mon, 19 Apr 2004 01:22:05 GMT

Kernel 2.6 impressions. I've just upgraded my laptop from kernel 2.4 to 2.6, and finished screwing around with various configuration files. So far, everything seems to work very nicely, including features which were hopelessly broken in 2.4. Combined with a good hardware autoprober, this could be a pretty user-friendly kernel.

The new scheduler has improved interactive responsiveness tremendously, too. I can actually use my computer while exim delivers a large mail queue through expensive procmail filters.

So far, it's a winner. Does anybody know of a decent hardware autoprober for Debian?

17 Apr 2004

Sat, 17 Apr 2004 15:36:41 GMT

C++ Frustrations. Normally, I can get some enjoyment out of programming in C++ (especially with a working STL and the boost libraries), but I've been suffering a fair bit of frustration lately.

Smart pointers are almost good enough, but fail in interesting little ways just when you need them the most. In my Type::GetResultType function, for example, I need to return this for all types except function types, and there's no way to get a shared pointer to this. So I need to hack around this somehow.

I miss Dylan.

16 Apr 2004

Fri, 16 Apr 2004 21:46:33 GMT

I just installed Gnome 2.6 out of Debian experimental, and I'm pleased by several things.

It's faster. GTK seems to be faster--menu redraws are quick, and the whole interface feels pretty good. Most impressive, though, is Nautilus, which has gone from dog slow to downright snappy.

It reminds me of the Mac. I'm an old Mac user, so I'm delighted by the "spatial" version of Nautilus. Essentially, there's a one-to-one mapping between directories and folder windows, and folder windows have persistent properties, including location. This was a really robust UI design, and I'm glad to see Nautilus borrowing it. I just wish they'd simplify the screen even further, and add support for those nice Finder triangles.

It has good blogging tools. I'm writing this post using a panel applet. It only takes me two clicks to post to Advogato. I've also set up the Straw news aggregator, which is thoroughly reasonable.

All in all, the UI situation is very good in Gnome 2.6. I'm impressed at how quickly the Linux desktop is becoming truly usable.

(On an unrelated note, Blender, MakeHuman, and Wings3D make a darn sweet 3D modelling setup. You can get better, but not without getting well into the 4-figure range.)

16 Apr 2004

Fri, 16 Apr 2004 19:20:54 GMT

I'm banging away on a native code generator. Basically, we needed a bytecode interpreter, and I had written one too many bytecode VMs. Remembering a comment by David Simmons on his SmallScript work--he argued that JITs perform 10 times better than VMs for about the same amount of work--I decided to hack up a native code generator instead.

Sure enough, it's easy, at least once you figure out the platform's calling conventions and what a ModR/M byte actually is.

Now, generating good code is an entirely different matter. But I'm not trying to do that. :-)

27 Aug 2001

Mon, 27 Aug 2001 14:45:05 GMT

Python Iterator Bug. Python 2.2 supports CLU-style iterators (they call them generators). They're essentially a kind of co-routine.

But there's a subtle bug in the Python design. Consider tree traversal:

def inorder(t):
    if (t.left != None):
        for (node in inorder(t.left)):
            yield node
    yield t
    if (t.right != None):
        for (node in inorder(t.right)):
            yield node

If you study this carefully, you'll see that (unless the optimizer intervenes), Python has turned a perfectly good O(N) tree traversal into an O(N log N) traversal.

How to Fix It. I'm probably going to write a paper with the details, and try to get it published, but here's the crux of the matter.

Python insists on performing the two subiterations manually, and on returning their output unchanged. But if you stack enough of these iterations on top of each other, you can kiss your performance goodbye (both asymptotically and cycle-wise).

Instead, you need to transfer control to your subiterations:

def inorder(t):
    if (t.left != None):
        yield_all inorder(t.left)
    yield t
    if (t.right != None):
        yield_all inorder(t.right):

Now the compiler has a fighting change to optimize a deep stack of iterators into reasonable code.

It turns out that you can formulate such concepts as self-recursive iterators, tail-called iterators, etc., and apply optimization techniques analogous to those used by LISP compilers to optimize function calls. What's worse, you can actually implement all of this using a portable C back-end to your compiler. :-)

26 Aug 2001

Sun, 26 Aug 2001 19:01:54 GMT

My Favorite Toy Language. Oh, dear. My design efforts have gotten too far ahead of my coding efforts, opening me to charges of bogosity. :-/ An overview of what I'm up to...

Soft Typing. MFTL is softly typed. This means that (1) everything, including primitive types, is a subclass of Object, (2) type declarations all default to Object and (3) downcasting is automatic. For example, the following program is legal:

def fact_untyped (i)
  if (i <= 1)
    1
  else
    i * fact_untyped(i-1)
  end
end

But if the programmer types the following, they get much better performance and compile-time warnings about type violations:

def fact_untyped (i: int): int
  if (i <= 1)
    1
  else
    i * fact_untyped(i-1)
  end
end

Everything's An Expression. In the above example, 'if' is an expression, like the ternary operator in C. So you could write:

var x = if (y) 1 else 2 end

Newlines End Statements. Just like in sh, bash, JavaScript, and Ruby, newlines in MFTL are statement terminators. (This is slightly funky, but it allows 'if' to be an expression without massively cruftifying the semicolon-placement rules.)

Keyword-Based Initialization. Classes generally don't need constructors:

class Sample ()
  var x, key: x
  var y = 10
end

s = new(Sample, x: 5)

Generic Functions. You can do the equivalent of operator-overloading at run time, not just compile time.

abstract class Thing () end
class Rock (Thing) end
class Paper (Thing) end
class Scissors (Thing) end

def defeats? (a: Thing, b: Thing) false end
def defeats? (a: Paper, b: Rock) true end
def defeats? (a: Rock, b: Scissors) true end
def defeats? (a: Scissors, b: Paper) true end

defeats?(new(Paper), new(Scissors)) // false
defeats?(new(Rock), new(Scissors))  // true

Yes, we know how to make this go fast.

Getters and Setters. No more endless 'getFoo' and 'setFoo' functions! Just write:

public class Example ()
  public var foo, key: foo
end

If you later decide that you need a getter and setter function, just write:

public class Example ()
  var real_foo, key: foo
  public def get foo ()
    real_foo
  end
  public def set foo (value)
    real_foo = value
  end
end

The users of the class will never know the difference.

Other Stuff. Things I want, but which I know will require extra work to do right: inlineable iterators, design by contract, integrated unit tests, simple templates.

Performance. With full type declarations, it should be possile to compile MFTL to run at speeds approaching that of C. Softly-typed languages are a solved problem.

Implementation Status. The parser and the VM are about 50% complete. The compiler was about 2% complete before I threw it out and started over. :-( If I had the luxury of working on this project full-time, I could ship a demo interpreter in about three months, and an MFTL-to-C compiler not long after.

Feedback. Comments, suggestions to eric.kidd@pobox.com.

3 Jul 2001

Tue, 3 Jul 2001 22:15:32 GMT

Certification. I'm curious about the ongoing saga with aaronsw's certification. He's proving suprisingly hard to suck into the local web of trust.

So... I read up on his projects, looked as his work, reviewed the certification guidelines, and tried to give him an appropriate certification (based on what I could quickly learn).

Still no change. Very weird.

I'm quite bewildered by this trust metric. One very generous (but, IMHO, undeserved) master-level certification was enough to drag me up from observer to master. But the combined efforts of quite a few people aren't helping aaron at all.

MFTL. Wow! Between my short-term job at MIT (helping implement a funcional programming language) and my hobby (designing a functional/imperative/OO language), I'm beginning to find a whole bunch of good answers to design questions. It's nice to work with people who are much smarter than you are. ;-)

Poll: Do I dare to open up port 80 on my cable-modem system, and let people access my language design Zwiki? Or is that just begging for script kiddies to open fire at Zope and various Apache modules?

4 Feb 2001

Sun, 4 Feb 2001 21:10:04 GMT

XML-RPC: I finished merging the Windows patches from several contributors; these will all go out in the next release.

I've also started work on xml-rpc-api2cpp, which is essentially an IDL compiler for XML-RPC. Instead of using IDL files, however, it directly queries the remote server for API information.

My XML-RPC hacking time is down to a few hours here and there. This will change in six or seven weeks. Yay!

27 Jan 2001

Sat, 27 Jan 2001 05:23:46 GMT

XML-RPC Acceleration: Spent this morning hacking mod_gzip to to understand "deflate" compression. If you drop this hack into your webserver (and use a smart XML-RPC client), you'll probably cut your outbound XML-RPC network traffic by a factor of 10.

It's very experimental--I can make it dump core--but it's the start of something moderately nifty. Combine this with boxcarring, and you're beginning to get some decent scalabity and throughput.

Binary Data: I also spoke with several of Flight Gear developers (all very cool folks), and discussed the possibility of XML-RPC without any XML. Basically, a client and server could negotiate away the XML layer, and transmit raw binary data structures to each other. You'd keep all the fun features of XML-RPC (the introspection, the dynamic data, the 750-line clients), but get enterprise-grade RPC when you really needed it.

The hard part of this would be the design, not the implementation. How do you hide the funky new features from the less-advanced clients, and how do you activate the new protocol?

Introspection: I wrote a script called xml-rpc-api2txt. Give it the URL of a server, and it will print out a nicely-formatted interface specification, complete with documentation. I fixed it to play nicely with Meerkat, too--O'Reilly was preformatting some of their documentation strings, which was messing up Perl's formatting commands.

Now, who wants to hack this script to automatically generate C++ and Java classes for a given server? :-)

Community: Yikes! Things are really starting to move. I got piles of e-mail today, half of which contained patches and the other half of which contained great ideas.

SourceForge: Is full of bugs.