oubiwann is currently certified at Journeyer level.

Name: Duncan McGreggor
Member since: 2001-11-21 22:13:44
Last Login: 2008-10-23 14:55:14

FOAF RDF Share This

Homepage: You're looking at it.

Notes: If you would like to leave comments on the blog posts, visit the blogger version.


  Electric Duncan
  Earth. It's where I keep my stuff.



[del.icio.us] [project pages] [hacking society] [LinkedIn] [sourceforge] [orkut] [Last.fm] [openid]

blog roll:

[python] [zope] [politics] [friends] [physics] [math] [madness]

I can be reached at the following address:
echo "wngvtg@twrmnf.nl" | tr a-z h-za-g
(thanks for the tip, fxn !)

Projects

Recent blog entries by oubiwann

Syndication: RSS 2.0

26 Oct 2009 (updated 27 Oct 2009 at 07:05 UTC) »

Recent Work on Various Open Source Projects


Over the last few months, I've been doing lots of work on various open source projects. I've been so burried in them that I haven't blogged (or microblogged) much about them. So much has been happening, though, that I needed to take a break from the coding and communicate some of this :-)

txAWS


Over the last few months, Robert Collins, Thomas Herve, Jamshed Kakar and I have been putting lots of effort into improving cloud support in the async (Twisted) Python Amazon EC2 library. It's been a lot of fun to see that part of the library take shape and start getting production use from Canonical. We have implemented the following functionality groups and their associated API methods:
  • Instances
  • Key Pairs
  • Security Groups
  • Elastic IPs
  • Availability Zones
  • EBS
There is a ticket for the following two groups of functionality, and branches in progress for both:
Once those are merged, EC2 support in txAWS will be complete.

As a bonus, we've added support for arbitrary endpoints and with that in place, have successfully tested txaws.ec2 against Eucalyptus clouds :-)

txJSON-RPC

There's a new release of txJSON-RPC out now, downloadable from PyPI. Work on the next version has been a great deal of fun. What started out as a conversation on IRC with Terry Jones, ended up as spec-driven doctest work on trunk for implementing support for multiple versions of the JSON-RPC spec (pre-1.0, 1.0, and 2.0).

With these changes in spec support, txJSON-RPC has really started to mature, and that's been fantastic. Even more, the jsonrpclib module that's included in txJSON-RPC (and can be used with non-Twisted projects) is getting spec version support as well.

SOOM in txULS

As some may know, one of my computing passions is ultra large-scale systems. After a phone conversation with Jamshed Kakar and some nice exchanges on the Python ULS-SIG mail list with Alex Drahon, I started working on a set of coding experiments in self-organizing objects. The Google Doc informally outlines the various stages and goals.

For now, the code is living in a txULS series on Launchpad. The reason for its inclusion in txULS is that ultimate goal of the SOOM (self-organizing object meshes) code is to produce an async API for building Twisted services that provide behaviours as outlined in the Google Doc (linked above).

I would to emphasize the networking-library-agnostic nature of the ULS-SIG: Twisted comes up since I spend a lot of time with Twisted, but ever networking library is welcome. I'm personally interested in exploring (or watching other developers explore) various Stackless Python experiments in the ULS systems space.

txSpore

This project was a spontaneous effort resulting from an evening of code review when I first discovered the official Python API from EA/Maxis for the game Spore. It's been a blast and something that Chris Armstrong and I have been working on together. The API is currently feature-complete, but Chris has some excellent ideas about improving usage as well as some additional API augmentations that will make life easier for game developers.

Already more featureful and usable than the official Spore Python API, there are great things in store for this library. Chris has come up with several very cool demo ideas that take advantage of the new API and will push it to the limits. We're both pretty excited :-)

Isomyr

I love isometric games. I'm a freak for the classic look. One night about a month ago, Chris and I discovered Isotope, an isometric Python game engine by Simon Gillespie. It was last updated in 2005 at version 0.9, so I started working on a branch that could be released as 1.0. I never heard back from Simon after an inquiry for his permission to release as 1.0, so I forked the code to a new project: Isomyr. I released the code rewrite work I had done to that point (plus some changes such as replacing some old code with Numpy) as 0.1. At which point things just went nuts...

Isomyr now has support for multiple worlds, customizable (per world) in-game time and calendars, and basic interactive fiction development. The latest chunk of code (that hasn't been pushed up to Launchpad yet) is adding support for general planetary simulation (e.g., axial title, varying daylight hours, seasons, and weather). As you might imagine, this has been a great deal of fun to work on!

PyRRD

PyRRD has gotten some recent community love, with requests for a mail list, new developer-oritnted features, etc. Currently at version 0.0.7, the 0.1.0 release isn't to far away. Folks have been using trunk for a while, which added support for the RRDTool Python bindings back in March of this year. (PyRRD's focus has primarily been for users/developers who didn't have the RRDTool Python bindings installed). In the next couple weeks or months, I expect that we'll be adding a few more features, and then preparing the new release.

PyRTF

Another fixer-upper project, PyRTF (mirroed on Google code as pyrtf-ng) has been on hiatus for a while, due to my diminished need to manipulate and interact with RTF files. However, a new developer has joined the project and the code-cleanup and unit test development now continues. Thanks Christian Simms!

A while ago, Simon Cusack
(the original author of PyRTF) and I had some great discussions about the future development of PyRTF and his interest in merging the recent changes into trunk on SourceForge. I deferred on that action, wanting to wait until the code cleanup, unit tests, and API changes had been completed. With Christian's help, we may get there now :-)

Wrap-up

It's been about a year since I've been so active in open source development, and it feels really good to be at it again :-) Being back in Colorado seems to have helped in subtle ways, but mostly it's been the increased interaction and interest from developers in the community that I can thank for my increased activity (and thus enjoyment). You guys are awesome. You're the reason for any code I produce.


Syndicated 2009-10-26 14:16:00 (Updated 2009-10-27 06:20:50) from Duncan McGreggor

17 Sep 2009 »

PyCon 2010 Talks Neeed!


Hey folks,

At my last count, we've only received 20 talks so far for PyCon 2010! There are only 14 days remaining for talk submissions... if you've had a great idea about a talk for PyCon, now's the time to make it happen!

Below I've pasted some links that folks might find helpful. The first one has pretty much everything you need to know about submitting a talk for PyCon.



Syndicated 2009-09-17 17:12:00 (Updated 2009-09-17 17:22:44) from Duncan McGreggor

14 Sep 2009 (updated 17 Sep 2009 at 06:03 UTC) »

txSpore: Twisted Spore



I just had a delightful weekend of coding :-) I spent the past two days porting the Spore Python API to Twisted. You can now incorporate Spore data (from static XML as well as REST requests) into your non-blocking Python applications/games!

This was a pretty easy task, really. The API just makes HTTP requests with twisted.web.client.getPage. There was a little bit of work involved in creating object models for the XML, and some head-scratching for the error-catching deferToThread unit test I tried to write (it's still buggy... need to figure that one out). Everything else was pretty much cake.

In fact, it was so much fun to kick back and write some playful code that I might overhaul the sync Python code as well and incorporate both into txSpore.

Do be aware, however, that the code still has some big improvements coming. The first thing I want to hit is actually create a client object. Right now, the client module contains a series of functions (since state's not currently needed). However, I want to start doing some basic object caching in order to limit the number of requests made to spore.com and increase the response time. That's the big item for 0.0.2. Update: 0.0.2 is now released!

Next I'd like to create some more demo apps that show off the API usage better. Right now, there's one demo (a .tac file). All it does is ask for a user name, renders a user page, and then links to a user "Spore assets" page (that's the thumbnail image above).

One thing that might be fun to do is write a script that checks for the latest achievements and publishes them to various microblog/status sites with the Twisted PyngFM client :-)

There's a project page up on Launchpad for txSpore, and I've posted a notice and some updates to the Spore developer forums. It's also been published on PyPI.

Enjoy!


Syndicated 2009-09-14 05:05:00 (Updated 2009-09-17 05:38:20) from Duncan McGreggor

8 Sep 2009 »

Windows Media to MP3 Conversion for Mac OS X and Linux


For the past couple years, my girlfriend has been amazingly (astonishingly) patient about a whole slew of .wma files that we've got on the network drive... backups of her CD collection made when she was a Windows user. We managed to save them right before the computer died, but she hasn't been able to listen to them when she's booted into Ubuntu or Mac OS X.

Late last month, after getting back from two weeks abroad, Marjorie said that she'd really like to have access to her music collection again (the CDs are cumbersome and stored away in boxes for our impending move back to Colorado). With that said, I did some digging around, and found some immediately helpful links (two years ago, a few google searches had turned up results that indicated too much effort was involved).

I started out by trying a couple free Mac OS X GUI applications, but these ended up being quite horrible: either they did not offer the functionality I desired, they were buggy to the point of being unusable, or they rendered audio with unlistenable artifacts.

In the end, I had to use mplayer and lame in combination. After googling around and some trial and error, I discovered the combination of mplayer options that would successfully extract the audio data from .wma files and dump them as .wav files.

I started with a shell script, but quickly changed to Python, since there were several locations for the .wma files, and none of them on nice paths. I've used this script several times since then, when more .wma files were discovered, and have yet to encounter any issues in sound quality. Once nice-to-have would be to extract .wma metadata and save it in the new .mp3 files as id3 tags...

Anyway, here's the code:

#!/usr/bin/python
import os
import re
import subprocess
import sys


# script configuration
if sys.platform == "darwin":
MPLAYER_PATH = os.path.join(
"/Applications/Non-Standard/Audio and Video",
"MPlayer OS X 2.app/Contents/Resources/mplayer.app/Contents/MacOS")
# lame was manually installed into /usr/bin
LAME_PATH="/usr/bin"
elif sys.platform == "linux2":
MPLAYER_PATH = "/usr/bin"
LAME_PATH="/usr/bin"
MPLAYER = os.path.join(MPLAYER_PATH, "mplayer")
LAME = os.path.join(LAME_PATH, "lame")
DUMP_FILE = "audiodump.wav"
BACKUP_DIR = "wma"
WORKING_DIR = "/tmp"


def make_audio_dump(filename):
"""
Use mplayer to dump the audio contents of the .wma files as .wav files.
"""
command = ("\"%s\" -nosound -vo null -vc dummy -af resample=44100 -aid 1 "
"-ao pcm:waveheader \"%s\"" % (MPLAYER, filename))
subprocess.call(command, shell=True)


def convert_wav_to_mp3(input, output):
"""
Use lame to convert the .wav files to .mp3 files. Remove the raw .wav file
when done.
"""
command = "\"%s\" -b 256 -h \"%s\" -o \"%s\"" % (LAME, input, output)
subprocess.call(command, shell=True)
os.unlink(input)


def convert_wma_to_mp3(wma_filename):
"""
Given a .wma filename, get a filename for the new .mp3 file based on this,
convert the original to a .wav and then that to an .mp3 file.
"""
mp3_filename = re.sub("\.wma$", ".mp3", wma_filename)
make_audio_dump(wma_filename)
convert_wav_to_mp3(DUMP_FILE, mp3_filename)


def has_wma_files(filenames):
"""
Given a list of filenames, check to see if any of them have the .wma file
extension. If so, return a true value; otherwise, a false one.
"""
for filename in filenames:
if filename.endswith(".wma"):
return True
return False


def convert_wma_files(path):
"""
Walk a given file system directory and all its child directories in order
to find .wma files. If found, convert them to .mp3 files and backup the
originals.
"""
for dir, subdirs, filenames in os.walk(path):
# we don't want to convert files that have already been converted
if os.path.basename(dir) == BACKUP_DIR:
continue
# if there's nothing to do, move on
if not has_wma_files(filenames):
continue
# define and create the backup dir, if it hasn't been already
backup_dir = os.path.join(dir, BACKUP_DIR)
if not os.path.exists(backup_dir):
os.mkdir(backup_dir)
for filename in sorted(filenames):
# on mac os x samba shares, sometimes ._*.wma files are present;
# skip these
if filename.startswith("."):
continue
if filename.endswith(".wma"):
print "Dumping audio for %s ..." % (filename)
wma_filename = os.path.join(dir, filename)
wma_backup = os.path.join(backup_dir, filename)
convert_wma_to_mp3(wma_filename)
os.rename(wma_filename, wma_backup)


if __name__ == "__main__":
path = sys.argv[1]
os.chdir(WORKING_DIR)
convert_wma_files(path)



Hope someone else finds this useful and their significant others don't have to wait 2 years for their music!


Syndicated 2009-09-07 22:47:00 (Updated 2009-09-07 23:02:06) from Duncan McGreggor

7 Sep 2009 »

Twisted Ping.fm Client


I just merged async (Twisted) support into the Python ping.fm library today and have just taken it for a test drive. I do love Twisted :-) The Twisted pyngfm API usage is identical to the synchronous API, with the usual exception of using deferreds and callbacks.

Here's some example usage, the client I now use for command-line updates to Twitter, Identi.ca, Tumblr, Facebook, LinkedIn, Jaiku, and even Flickr (note that the keys are stored in an .ini-style config file):


#!/usr/bin/python
import sys
from ConfigParser import SafeConfigParser

from twisted.internet import reactor

from pyngfm.client import PingFMAsyncClient


def checkMessage(message):
if len(message) > 140:
print "Message is too long! (%s chars)" % len(message)
sys.exit(1)


def getKeys():
cred_file = "/etc/ping.fm.creds"
config = SafeConfigParser()
config.read([cred_file])
api_key = config.get("default", "api-key")
user_app_key = config.get("default", "user-app-key")
return api_key, user_app_key


def pingIt(message):

def check_result(status):
print status

def check_error(error):
print error.getErrorMessage()

def finish(ignored):
reactor.stop()

pinger = PingFMAsyncClient(*getKeys())
deferred = pinger.user_post("status", body=message)
deferred.addErrback(check_error)
deferred.addCallback(check_result)
deferred.addErrback(check_error)
deferred.addCallback(finish)


if __name__ == "__main__":
message = " ".join(sys.argv[1:])
checkMessage(message)
pingIt(message)
reactor.run()



There's another example in the README that iterates through the recents posts made to ping.fm. If you do manage to use it and come across any issues, be sure to file a ticket.

Enjoy!



Syndicated 2009-09-07 22:20:00 (Updated 2009-09-07 22:45:33) from Duncan McGreggor

241 older entries...

 

oubiwann certified others as follows:

  • oubiwann certified fuzzyping as Master
  • oubiwann certified oubiwann as Journeyer
  • oubiwann certified nornCabor as Apprentice
  • oubiwann certified moshez as Master
  • oubiwann certified itamar as Master
  • oubiwann certified amk as Master
  • oubiwann certified effbot as Master
  • oubiwann certified Uche as Master
  • oubiwann certified calderone as Master
  • oubiwann certified spiv as Journeyer
  • oubiwann certified glyph as Master
  • oubiwann certified mesozoic as Journeyer
  • oubiwann certified ChrisMcDonough as Master
  • oubiwann certified fdrake as Master
  • oubiwann certified ianb as Master
  • oubiwann certified tav as Journeyer
  • oubiwann certified jafo as Master
  • oubiwann certified Namosui as Apprentice
  • oubiwann certified rbhalotia as Journeyer
  • oubiwann certified essallen as Apprentice
  • oubiwann certified roozbeh as Journeyer
  • oubiwann certified titus as Master
  • oubiwann certified pvanhoof as Journeyer
  • oubiwann certified cactus as Master
  • oubiwann certified wingo as Journeyer
  • oubiwann certified etrepum as Master
  • oubiwann certified gstein as Master
  • oubiwann certified eikeon as Journeyer
  • oubiwann certified faassen as Master
  • oubiwann certified jml as Master

Others have certified oubiwann as follows:

  • fuzzyping certified oubiwann as Master
  • oubiwann certified oubiwann as Journeyer
  • nornCabor certified oubiwann as Master
  • lerdsuwa certified oubiwann as Journeyer
  • Namosui certified oubiwann as Master
  • rbhalotia certified oubiwann as Master
  • pvanhoof certified oubiwann as Journeyer
  • fxn certified oubiwann as Journeyer
  • mirwin certified oubiwann as Master
  • Omnifarious certified oubiwann as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

FOAF updates: Trust rankings are now exported, making the data available to other users and websites. An external FOAF URI has been added, allowing users to link to an additional FOAF file.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page