Advogato: Pyjamas - Python Applications for Desktop and Web

What is pyjamas-desktop?

based on webkit, it's a cross-platform application development API. like pygtk2, like python-qt4 and wxWidgets, only much cooler because you can, if you want, load *complete* html pages, *complete* stylesheets, and even execute bits of javascript if you're feeling particularly obtuse.

wow. how?

because it's based on webkit. so you get access to the DOM model, you get full HTML compliance, wickedly-fast javascript execution, media plugins, CSS stylesheets - everything.

whoaaaa - bbb..back up a bit: what's pyjamas?

based on google webkit, pyjamas is a cross-browser _web_ application development API. it looks and smells like a desktop API, but underneath, pyjamas is an AJAX library and a comprehensive widget set (implemented as AJAX but you *never* go anywhere near javascript, because at the core of pyjamas is a python-to-javascript compiler).

the only actual exposure you really have to have to your app being a Web application is the initial ten line "loader" HTML page and an optional CSS file. even the CSS file is optional because you are provided via the Pyjamas API with access to some of the more useful CSS features such as setting the width and height, and you can if you wish directly manipulate the CSS style properties from your application.

in short, you get to write apps that _look_ like they ought to be running on a desktop, and pyjamas takes care of all the nasty browser tricks that you would normally take _months_ to code up - if you bothered at all - for safari, opera, IE7, IE6, firefox, mozilla, midori...

.. even konqueror nearly works if you twist its arm hard enough.

what's the fuss, then?

pyjamas-desktop is a project to port pyjamas - the toolkit that was ported to java - the one that looks like it _ought_ to run on the desktop - to the desktop.

i still don't get it. so what??

you get to run the SAME application source code (written in python) as EITHER a web application - on 99% of browsers in use today - OR as a desktop application - on as many environments as webkit (well, actually, pywebkitgtk at the moment) will compile on.

effectively, pyjamas becomes a "standard" for application development. cross-browser. cross-platform. even cross _widget_ set, if someone wants to create pywebkitqt4 and the associated qt DOM model webkit bindings. and a python-wxWidgets one.

yeah - and? i've been able do DOM model manipulation with Konqueror for years

yes... and you get all the KDE baggage, too. and KDE's multi-million lines of code is being very slowly ported to Win32. and there's no serious possibility of running KDE on an embedded platform, which is a tantalising possibility for webkit and pywebkitgtk, using gtk-directfb.

you... huh?? i'm having difficulty getting my head round the significance

free software from ground up home-grown alternative to silverlight.
free software from ground up home-grown alternative to adobe AIR.

except more than that - you can't get silverlight or AIR to run on anything but the platforms that those proprietary vendors choose to run it on (yes, i know there's Moonlight/Mono).

why??

because, as it's web-based, being based on webkit, any media plugins that webkit support, now and in the future, you can "embed", control, and interact with, through the webkit API.

e.g. flash plugins.

what else? got a kitchen sink to throw in with that?

this isn't burger king, you can't have fries - but you _can_ have an event sink, rather than a kitchen sink. and yes, you _can_, through the newly-created glib-gobject bindings, access and control webkit's DOM model through perl, *mm, or whatever obtuse language you choose.

my obtuse language of choice is python; h2defs.py / pygtk-codegen made short work of turning the glib header files into python bindings, and we're off.

(btw - yes, actually, there is an example in Pyjamas and GWT called "kitchen sink").

basically, if you're a perl-lover, you get to write your own kitchen sink, if you haven't already done so.

what's the catch?

[update: pyjamas-desktop 0.1, which is useable, was released 06sep2008]

the catch is: pyjamas-desktop is in development. the version known as "pyjamas-desktop/pywebkitgtk" is about... 16 hours old! don't be fooled, though: in python terms, that's an awful long time. the DOM model support went from 0% to 5%, enough to run Hello.py, in an hour. 5% to 20%, enough to run GridTest.py, in about 30 minutes. 20% to 40%, enough to run Mail.py, took another half hour, and so on.

i'm up to KitchenSink.py and it's pretty much been a walk in the park. i've a little bit of thinking to do, to handle events properly: the only type of event supported at present is mouse "click" but pywebkitgtk now has support for all the browser event types: it's just i haven't added them in to pyjamas-desktop yet.

also, i'm adding in glib bindings that i missed out and put on the "TODO" list as i go along. when i encounter a feature that i need for pyjamas-desktop, "TODO" becomes "Today".

where on earth did you get this idea from?

it seemed obvious.

i remember seeing - it could have been a trick of my imagination - back in 1992, a Windows NT 3.51 application toolkit where you could run Win32 apps - unmodified - through a web browser. exactly how this was done i really don't know, but the idea stuck.

so, last month, i made three separate attempts to get started. the first attempt was with GTK-Sharp and IronPython. It was the most promising of the attempts, but, missing a JSONRPC Proxy, and not wishing to get involved in Mono / .NET socket and XML at this early stage, i quickly skipped to pygtk2 (where it was easier to create a client-side JSONRPC proxy).

i did a part-port of pyjamas to gtk2 in a few days - enough to tell me that i was wasting my time ... ( pyjamas-pygtk2.tgz if anyone's interested) .. due to the lack of support for libgtkhtml3 which has a crucial feature i needed (for Pyjamas "FlowPanel"). i did a part-port of pyjamas to qt4 in about _two_ days... (pyjamas-pyqt4.tg z if anyone's interested) ... enough to know that, despite the rich-text support, the layout mechanism in Qt, and the lack of support for being able to remove layouts from grid layouts.... i just... gave up and made a beeline for webkit...

now, in under 8 hours, with the DOM model access glib bindings to webkit, pyjamas-desktop has 80% of the functionality completed!

what's the other possibilities?

there exists some _really_ exciting tantalising possibilities with this, which takes a bit of explaining.

the first time i saw pyjamas, i went "cool! a web-based widget set that looks like a desktop widget set. i wonder if it can be _made_ to be a desktop widget set?".

then, i saw that llampies - one of the pyjamas developers - had ported pyjamas to gtk. i got really excited, only to find that _actually_ what he'd done was the OTHER WAY ROUND: he'd ported _gtk_ to pyjamas!

GsocLlpamies

llampies, basically, has written "wrappers" - alternative implementations of gtk.py, glib.py and gobject.py, which get compiled with the python-to-javascript compiler, pyjamas, along with your pygtk2 app, to run your pygtk2 apps UNMODIFIED as *web* applications.

so, you write your app as a python-gtk app, you compile it up using pyjamas, and it runs as a web application. even though it's actually python-gtk-compliant source code.

bear with me whilst i outline why that work is so significant. take a python-GTK application, and run it through pyjamas, and it's a web app... but if you run it under pyjamas-desktop, it runs ... as a web app.

that doesn't sound significant...

(...in fact it sounds mad. you're taking a desktop app and running it... err... on a desktop, adding massive overhead and missing out bits of the pygtk2 API as well, _and_ you're limiting me to the subset of python that the pyjamas compiler supports!!!)

whooa, hold your horses - it sounds mad... until you remember earlier that i mentioned the possibility of running webkit under python-qt4 if someone writes the bindings. and running webkit under python-wxwidgets if someone does the python bindings for that, too.

so... you get to write an app as a python-gtk2 app, conforming to the pygtk API, and... through pyjamas-desktop/webkit with llampie's gtk2 wrappers, you get to run your python-gtk2 app UNMODIFIED in a qt4 environment. or a wxwidgets environment. or... any-other-kind-of-environment. including a web app, remember, if you dump pyjamas-desktop and just use llampie's compiler.

wow!

yeah.

it's still madness

yeah. quit bitching about it.

oh - and you _should_ be writing your apps to conform to a MVC framework _anyway_, so there's no need to complain about being limited to a subset of the python library. you have JSONRPC (yes, i've written a JSONRPC proxy, which is a useful JSONRPC client library in its own right), you have pimentech's JSONRPC server-side django plugin, so you can write the front-end in a limited subset of python (it's not _that_ limited) and get it to talk to the back-end, which happens to be running on loopback HTTP as a twisted app or a django service, where you have full and complete access to the entire range of python libraries.

and, then, if you're ready to make it a web app, compile it to AJAX with pyjamas and... well... that's it. you're done.

and your code is nicely subdivided into front-end, back-end, just the way it should be.

gimme some examples!

okaaay...


from ui import Button, RootPanel, Label
import Window

 
 def greet(sender):
    Window.alert("Hello, AJAX!")

 
 class Hello:
    def onModuleLoad(self):
        b = Button("Click me", greet)
        l = Label("hello world")
        RootPanel().add(b)
        RootPanel().add(l)

one button. one "hello world" label. one alert popup. one vision. (just gimme gimme fried chicken)

that's it??

yeah.

well.. ok... here's the associated HTML, too.


<html>
    <head>
        <meta name="pygwt:module" content="Hello">
        <title>Hello</title>
    </head>
    <body bgcolor="white">
        <script language="javascript" src="pygwt.js"></script>
    </body>
</html>

strictly speaking, you don't need the pygwt.js library in there, but you might as well leave it there. sure, webkit will try to load it, but if it doesn't exist, so what.

but.. but... that's just the pyjamas "hello world" example!

yes, and it's unmodified, too, that's the whole point, and it _still_ runs as a desktop application under pyjamas-desktop, using the modified pywebkitgtk (see code references below).

basically, i'm taking all the "javascript" code fragments in the pyjamas ui.py and other library functions, and replacing them line-for-line with pure python, calling pywebkitgtk DOM bindings functions! (ideally, of course, the work would go a lot quicker if i had a javascript-to-python compiler.)

so, for example, in pyjamas, there's some voodoo trickery going on to walk the Hello.html document, looking for that special "meta" tag with the name "pygwt:module". i've replaced __pygwt_processMetas javascript function with one that accesses the DOM model directly in python rather than in javascript, and, once the names of the modules are obtained, this is done:


for m in pygwt_processMetas():
    exec """\
from %s import %s
m = %s()
m.onModuleLoad()
""" % (m, m, m)

which, for each of the modules found, directly runs the onModuleLoad() function. so, in the Hello.html example, you get this being executed:


    from Hello import Hello
    m = Hello()
    m.onModuleLoad()

and thus your app gets prepared, ready to run the gtk main loop.

can i _really_ still run bits of javascript even though it's a desktop app?

yes.


    ctx = main_frame.gjs_get_global_context()
    doc = ctx.get_by_name("document")
    ctx.eval("console.error('hello');")
    doc.execute("""
node = document.createTextNode('Some content.'));
document.getElementById('body').appendChild(node);
""")

that's disgusting!

yes it is, isn't it? even more disgusting is that after you've done that, you can go and access the DOM model directly through the glib-object python bindings and mess with it some more!


    gdoc = main_frame.get_gdom_document()
    body = gdoc.get_element_by_id('body')
    node = gdoc.create_text_node('some more content')
    body.append_child(node)

of course, that's using the new pywebkitgtk bindings directly (just like in KDE except without... KDE: see example and bugreport context). but - you do NOT have to do this - that's pyjamas-desktop's job, to take you entirely away from the "mess" of performing DOM model manipulation, and providing you with a neat widget-set instead. just like it's pyjamas's job to take you entirely away from the mess of AJAX programming.

like they say on Braniac - "STOP! DO NOT TRY THIS AT HOME! We do these experiments, so you don't have to!"

what... what _is_ this!!!

it's a new widget set API, effectively. it's like python-qt4, it's like python-gtk2, it's like python-wxWidgets, except without the limitations of those widget sets. gtk2 has the look-and-feel of an abandoned urinal. qt4 i love but it still has limited "rich text" - nothing like the power and flexibility of CSS stylesheets.

when i or someone gets round to it, there will even be DOM bindings to the SVG document features of webkit. so, you will be able to create very powerful and flexible applications, drawing direct to a canvas. but, that might take a few days of work, so i'm putting it on the "TODO" list.

i want one. how can i get it? now!

right now, you'll need patches to webkit and pywebkitgtk (see below). then you get to play. in the meantime, you can always write your apps as web apps, using the pyjamas compiler, confident in the knowledge that you will be able to run them as desktop apps in the very near future.

code references and patches: