Older blog entries for osullivj (starting at number 53)

Insane programming: there's a myth about investment banking software commonly voiced by coders outside the pressure cooker world of trading systems development. It goes something like this: huge amounts of money are at stake, so the software must be super reliable and robust. Very often, the opposite is true. Time to market is the driving factor. Traders want functionality yesterday. If a bank can execute profitable business, it can afford to throw support staff at a flaky system. Bugginess will be tolerated if it means executing business that may have gone elsewhere.

In an earlier entry I mentioned that I don't blog about my day job, since wholesale banks are secretive and paranoid. Most other techies working on Wall St and the City of London keep quiet too. But some are starting to lift the lid. If you're curious about what it's like to code on a trading floor, you should be reading this blog.

Generators: are doing my head in ! Been coding up the generator based async IO approach I mentioned recently this afternoon. I want real, non blocking, multi threaded concurrency. I use generators to yield after sending down a socket. And then I resume when I get a Twisted callback telling me a response has arrived. That way no thread blocks waiting on a response. Less threads, more busyness.

Figuring out where to yield and when to return, and thinking about when I want resumable stack state, and when I don't, has been a head twisting experience. Which is great, because that means I am acquiring a non trivial design idiom. Last time I had this experience was my first multithreaded code in the late 90s. Before that it was OO in the early 90s. All good stuff.

27 Feb 2004 (updated 27 Feb 2004 at 06:52 UTC) »

Plan to throw six away: more levitt hacking last night. Really happy with the way the codebase is coming along at the moment. I've been working on this system since 2001. I've restructured and refactored so many times. For server infrastructure I started off with a plain medusa front end, and ZODB at the back. Then I switched to a Zope product approach, to leverage Zope's UI. Then I switched to a browser plug in implementation. Now I'm doing a standalone impl, that will probably use Twisted in a personal server deployment. And there were minor refactorings within those larger implementation decisions. I know Fred Brooks enjoined us to "plan to throw one away", but maybe I'm going too far. However, the upside is the implementation feels smaller and cleaner and far better factored.

clarkbw: my fave Skynyrd tune !

Generators: Python has them, and C# is proposing to add them. What are they ?

def GeneratorFunc( n):
    for i in range( n):
        yield i

for i in GeneratorFunc( 3): print "i=%d" % i

The call to GeneratorFunc does not execute any of the code inside the GeneratorFunc. Instead it returns an iterator object that we can use in the for loop at the bottom. The for loop implicitly calls the iterator's next() method, which in turn executes the code inside GeneratorFunc. Every time this happens, the yield keyword supplies the value to be returned by the iterator's next() method. The important thing to remember is that all the local variables, and stack state, inside the GeneratorFunc body are preserved between invocations of the iterator's next() method. All this may be a little clearer if we invoke the iterator directly rather than relying on for to do it...

def GeneratorFunc( n):
    for i in range( n):
        yield i

iter = GeneratorFunc( 3) while 1: print "i=%d" % iter.next()

All well and good, you may say. But where can I apply this idiom ? I can see tremendous leverage in two common coding areas: recursive tree walks and blocking IO handler code. More later...

Financial technology blogging: I ply my trade as a software developer in the City of London, working for wholesale banks. So do many of the developers I know. Since wholesale banks tend to be secretive and paranoid, most of us who develop trading systems don't talk about it much in public forums. I don't talk about my professional work at all on line. But a few people are starting to put their heads above the parapet...

A highly talented colleague Quant dev in SF

Hacking: rewriting the core system for my main project -levitt - for the fourth or fifth time. I had the original idea for this stuff in early 2001, and I've been tinkering ever since. It started off as a Zope product, coded in Python. I'm sticking with Python, naturally, but have dropped Zope, and I'm moving away from object orientation too. Zope has a lot to offer in terms of server architecture and ready made UI. ZODB persistence, transactions, threading model and the management UI give a lot of leverage. But to exploit them you have to buy into the Zope Object Model - including acquisition - and that's too big an imposition. Especially since I'm now moving away from OO.

Why move away from OO ? Because it introduces too much coupling between data and behaviour. Mainstream OO, with single inheritance and strong compile time type checking, as exemplified in Java and C#, binds your method body code very tightly to your data, making it less reusable. Dynamic languages like Python and Smalltalk introduce less coupling between data and behaviour, so code is more generic and reusable. I aim to loosen that coupling even further in levitt.

Seeing an Advogato link to Xerblin got me really excited the other day. There are some similarities to levitt: it's a programming system project, coded in Python. Xerblin's developer has used Forth ideas in his kernel. I have been using a Forth like stack based core in levitt too. I'm now moving to something different, since I've come to think there's too much state on the stack. Oberon ideas are there in Xerblin too. I'm far less familiar with this system than Forth and Python, but it sounds cool. It's good to see some fresh approaches to programming systems, rather than the same tired old ideas from Java and .Net.

Books: war reading continues with Hitler and the Power of Aesthetics. The obvious and pressing question on Nazism is: how could it happen ? Most answers are framed in terms of brutality, coercion, desperation, economic catastrophe. These explanations have seemed incomplete to me for some time now. Last year I read Sebastian Haffner's Defying Hitler, which gives one a tremendous sense of how the Nazis skillfully played on the psychology of loyalty, camaraderie and patriotism to sweep people into complicity. Spotts book explains in compelling fashion how seductive the aesthetics of Nazism were: the torchlight parades, the SS uniforms, the swastika banners, Wagner's music, Speer's monumental neo-classical architecture, and Hitler's theatrical sense of neo-religious ritual. The book reminds me again of Paul Feyerabend commenting on how he nearly joined the SS instead of the German Army, because the uniform was so cool.

Another thing that made Nazism possible was Hitler's charisma. Our image of Hitler now is largely determined by Allied wartime propaganda, which portrays him as a ranting demagogue, foaming at the mouth. That can't have been how the German populace percieved Hitler in the 30s. Spotts has a lot to say on how Hitler didn't engage in traditional political debate, but made an emotional appeal to his audience. By all accounts, his personal presence was hugely charismatic and overpowering. So we should distrust charisma. Personal experience has led me to this conclusion too - when I've made mistaken career choices its been because I've fallen prey to charismatic and persuasive personalities. Charisma can hide a multitude of sins. I must do some more reading on the mechanisms of charisma.

Looks like Jeff Vroom has broken cover. Good luck with the quest Jeff !

Hacking: did a minor release of templ last night.

Quicksilver: started reading the follow up to Cryptonomicon. I'm only 20 pages in, but already I'm transported, enraptured and enthralled. I'll be heading on over to metaweb to log my thoughts as I read.

Work: the new project is great - the best thing I've worked on professionally for several years. I'm getting to do some hardcore multi-threaded C++, which is a joy.

Blogging: two entries in two days !

Spam: got my first piece of snail mail spam yesterday. Of course, junk mail is nothing new. But this was a hand addressed letter to me, containing a hard copy of a Nigerian scam. No amount of Bayesian filtering is going to stop this !

BookCrossing: murrayc's recent post alerted me to BookCrossing. In an act of Python advocacy I released one of Sean McGrath's books today.

44 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!