Older blog entries for connolly (starting at number 64)

23 Feb 2009 (updated 24 Feb 2009 at 23:07 UTC) »
Family business blog: drupal or wordpress? self-managed or hosted?

The Kansas City Web Professionals meet this Wednesday. I started to post to their linkedin group about my recent trip to Web Directions North, but I figured hey: why drive ad revenue on linkedin rather than my own consulting business site, Midwest Web Sense?

But that site needs some maintenance. I set up drupal on nearlyfreespeech.net and tried to get OpenID working with it... when I ran into glitches, I learned I'm already a major version behind. And the other drupal site in my life, breadcrumbs is falling apart because the drupal guy in our research group moved on.

Yesterday, Kyle, my 9 year old son, agreed to help. So I explained about hosted blogs vs running your own, and where the ad revenue goes, and such.

We had a fun hack session in my office; you can see his work on choderbug and as wsouski on twitter. I think he's starting to grok the community values:

@wsouski don't be a spammer, wsouski. don't be a spammer.

Update: the WP domains clue from mhausenblas in #swig was just the ticket to set up notes.midwestwebsense.com: Web Directions North ski trip was a blast!

I found something I was looking for via ohloh tonight... the only thing it knows about me is that I made a handful of commits to the W3C validation service in 2001. I started a profile and registered cwm as a project, but there wasn't enough instant gratification to keep me at it...

2 Jan 2009 (updated 11 Jan 2009 at 20:20 UTC) »
XO-1 as music front-end?

Latest mt-daapd transcodes. Sweet. No more ripping flac to mp3 just for stupid iTunes -- ndw 21 Dec 2008

Perfect; now that I have a new big disk, I can take our iTunes libraries, which have gotten sorta mixed up, and merge them into one big honkin networked library. mt-daapd Ubuntu quickstart worked fine.

But... how to access the songs from the living room? The roku soundbridge and stuff look nice, but I can't justify the cost. Likewise airport express.

It just hit me: my xo-1 has wifi and a headphone jack.

update: it works! I used mpd with output to icecast2. The documentation could use some diagrams, but once I got the picture, it was reasonably straightforward:

  • mpd indexes the music, takes commands over a network protocol from various clients such as pympd (linux) and theramin (mac), queues songs, and decodes and streams them

  • icecast takes streams and buffers and multiplexes them to multiple clients
  • mplayer on the XO-1 listens to the stream and plays it thru the headphone jack
  • powered speakers in the living room play the results

For bonus points: I'd like to control it from my sidekick (or g1). That should be just a matter of setting up a web-based mpd client and exposing it thru the firewall.


My related work:

Related work by others:

1 Jan 2009 (updated 1 Jan 2009 at 21:49 UTC) »
Still struggling to catalog CDs

I'd like to catalog a bunch of CDs. My cuecat isn't working today; I'm not sure why not. And even if it were, alexandria fails to make heads or tails of the list of ISBNs that I captured last time it worked.

I'm not quite ready to pay $40 for Delicous Library 2, but I'm getting there.

zebra looks interesting; I like the idea of scanning barcodes with a webcam.

I couldn't get the firewire DV camera that works great with kino to work with cheese or ekiga any other v4l app; dunno why.

I broke out the old USB webcams; they still don't work. lsusb shows them as:


046d:08b2 Logitech, Inc. QuickCam Pro 4000
0545:8333 Xirlink, Inc. Veo Stingray/Connect Web Camera
046d:0870 Logitech, Inc. QuickCam Express
0733:0401 ViewQuest Technologies, Inc. CS330 WebCam

lots of this:


$ cheese
libv4l2: error dequeuing buf: Input/output err

The Ubuntu community seems to track this as Bug #260918; I just subscribed.

p.s. quicken records show:

  • 2001/12/09 Best Buy Xirlink VEO PC Camera $26.74
  • 2001/12/29 Best Buy veo pc camera 26.74
  • 2002/12/21 COMPUSA #141 quickcam 59.99
  • 2003/11/29 MICRO CENTER #191 quickcam express rev2 49.99
1 Jan 2009 (updated 8 Jan 2009 at 15:29 UTC) »
Reviving Home Movies with kino and ffmpeg

I've tried digitizing movies before, but the codec puzzles were overwhelming. This year, Kino pretty much Just Works. After one permissions issue with /dev/raw1394 , it ate hours of video and produced nice collections of DV files with SMIL wrappers (hmm... I wish SMIL were an XHTML microformat... more on that another time). And it exports not only ogg but also consumer technology: youtube-style .flv and XVid that works with the $20 DVD player I got via craigslist. (Thanks for the clues, longtail video.)

One reason I swapped this task back in was that we ran across home movies on decaying analog media when cleaning up the basement. Another reason is all the storage space I have since I couldn't pass up a Micro Center hot deal: $70 for 640GB of disk. That's 11 cents/GB.

update: kivo is flaking out, as is dvgrab.

It's sorta silly to use DV format to capture video off VHS. It's also silly to capture all the uninteresting bits at high bit-rate. Better to make a cheapo FLV of the whole thing and then go back over the interesting parts.

ffmpeg to vcd works; to do svcd, need unstripped libs per medbuntu bug info


$ ffmpeg -t 2:00:00 -f dv -i /dev/dv1394/0 -target ntsc-svcd
russia-vhs-capture.mpg

p.s. note Video Pain by Tim Bray, May 2008.

30 Dec 2008 (updated 30 Dec 2008 at 18:39 UTC) »

woot! It works! Youtube videos on a $30 media player!

The AMV format was new to me; I found a HowToConvertToAMV recipe, but that lacks support for mp4a, the audio codec used by youtube (at least the videos I tested).

Then I found adding --enable-faad to ffmpeg (don't forget to make distclean) and presto:


$ clive
'http://www.youtube.com/watch?v=3pYN42YyWp0&feature=related'
clive 0.4.19 20080722  [Linux]
http://www.youtube.com/watch?v=3pYN42YyWp0&feature=related&fmt=
           100%
BrianReganWalkieTalkie.mp4                                 
              2.6MB
=> 1 (2.6MB), failed: 0, skipped: 0.
BrianReganWalkieTalkie.mp4                    100%    2.6MB
 109.2KB/s 00:00:24


$ ~/src/amv-codec-tools/AMVmuxer/ffmpeg/ffmpeg -i BrianReganWalkieTalkie.mp4 -f amv -s 128x90 -r 16 -ac 1 -ar 22050 -qmin 3 -qmax 3 BrianReganWalkieTalkie.amv FFmpeg version SVN-r589, Copyright (c) 2000-2007 Fabrice Bellard, et al. configuration: --enable-gpl --enable-libfaad libavutil version: 49.5.0 libavcodec version: 51.47.1 libavformat version: 51.17.0 built on Dec 30 2008 12:07:41, gcc: 4.3.2 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'BrianReganWalkieTalkie.mp4': Duration: 00:01:13.7, start: 0.000000, bitrate: 291 kb/s Stream #0.0(und): Audio: mpeg4aac, 44100 Hz, stereo Stream #0.1(und): Video: h264, yuv420p, 320x240 [PAR 0:1 DAR 0:1], 30.00 fps(r) PIX_FMT_YUV420P will be used as an intermediate format for rescaling Output #0, amv, to 'BrianReganWalkieTalkie.amv': Stream #0.0(und): Video: amv, yuvj420p, 128x90 [PAR 0:1 DAR 0:1], q=3-3, 200 kb/s, 16.00 fps(c) Stream #0.1(und): Audio: adpcm_ima_amv, 22050 Hz, mono, 64 kb/s Stream mapping: Stream #0.1 -> #0.0 Stream #0.0 -> #0.1 Press [q] to stop encoding frame= 1172 fps=343 q=0.0 Lsize= 3878kB time=73.2 bitrate= 433.7kbits/s video:3062kB audio:802kB global headers:0kB muxing overhead 0.376436%

Merry Christmas, Kyle!

It's not as nicely packaged as the Mac app Justin uses with his ipod, but it's cheaper and more free (I'm not sure if it's quite open source; I don't know the license details of the AMV code.)

See also: #swig chat, including notes on a mythv box.

tags: media, video

struggling to get my data out of Zope

I found a recipe for iterating over the contents of a Zope store. It looks promising, but I'm not wining yet. My zope instance was built with zope 2.7 but the debian server has 2.9 now; 2.7 doesn't even show up in an apt-cache search.

This job of migrating dm93.org from Zope was pending an alternative to Zope for family calendar storage. Now that my wife's laptop runs Leopard, google calendar works as a host.

Then I somehow lost root when poeple.w3.org was upgraded or something, and I just recently got around to asking thru channels to get it back.

Time to swap this job out, I think; here's my current state:


connolly@flanders:~$
INSTANCE_HOME=/var/lib/zope2.7/instance-dm93-disabled
PYTHONPATH=/usr/lib/zope2.9/lib/python python
Python 2.4.4 (#2, Oct 22 2008, 19:52:44) 
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> from Zope2 import app
>>> root=app()
/u2/connolly/dm93/Products/LocalFS/LocalFS.py:51:
DeprecationWarning: Using OFS.content_types is deprecated
(will be removed in Zope 2.11). Instead use
zope.app.contenttypes.
  from OFS.content_types import find_binary
No handlers could be found for logger "Zope"
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/zope2.9/lib/python/Zope2/__init__.py", line
51, in app
    startup()
  File "/usr/lib/zope2.9/lib/python/Zope2/__init__.py", line
47, in startup
    _startup()
  File "/usr/lib/zope2.9/lib/python/Zope2/App/startup.py",
line 46, in startup
    OFS.Application.import_products()
  File "/usr/lib/zope2.9/lib/python/OFS/Application.py",
line 685, in import_products
    import_product(product_dir, product_name,
raise_exc=debug_mode)
  File "/usr/lib/zope2.9/lib/python/OFS/Application.py",
line 723, in import_product
    raise sys.exc_info()
KeyError
>>> 

Have I tried this gnome blog thing before?

hmm... no blockquote? lame.

what about images/photos?

tried BloGTK; couldn't get it connected to the W3C Q&A blog.

I think the last blog editor I enjoyed was flock; maybe I'll give that another try.

25 Oct 2007 (updated 25 Oct 2007 at 01:16 UTC) »

Remembering Modula-3

Systems Programming with Modula-3 has been on my wishlist for years, but after reading the feedback from Tim Bray's Wide Finder project, I finally got my very own copy.

One of the reactions, Finding Lisp, observes:

Most popular programming languages have the same simple threads+locks paradigm that was popularized with pthreads and Java.

But Java made everything a monitor.

I learned pthreads while working on a big horrible C++/DCE project, Dazel (later bought by HP). One of the guys there (Jim W?) loaned me his copy of Systems Programming with Modula-3, where chapter 4 is a copy of An Introduction to Programming with Threads by Birrell. It's probably a good thing that we never followed thru on our dreams to rewrite the whole project in Modula-3, but it was good to know about partial orders on locks and such while taming the pthread libraries (... and the C++ exception runtimes; what a nightmare!)

And anybody who had studied the Modula-3 Thread and IO design would know better than to make everything a monitor.

A lot of good stuff from Modula-3 lives on in python, and some in Java, but DEC got bought by Compaq which got bought by HP, and a lot of the DEC SRC goodies seem to be disapearing from the net.

The wikipedia article on Modula-3 has a "This article does not cite any references or sources" tag since July 2006. That looks silly, since a number of books and articles are listed. I can see some unsupported claims, though, so I ordered my own copy of SPwM3 so I can separate some of the verifyable claims from the speculation.

I'd also really like to find a host (other than the wayback machine) for the hypertext version of the SRC Modula-3 sources; it's a gold-mine of software engineering theory and practice; for example, from Fingerprint.m3:

The original fingerprint interface offered at SRC did not include the procedure Combine. The Vesta configuration management project built a system that cached intermediate results for large software builds. Abstractly, this is a special case of the common subexpression problem mentioned previously, and the project used fingerprints as keys in the cache. It is instructive to learn what happened.

You might think that a simple way to solve the common subexpression problem without Combine would be to fingerprint the texts that result from printing the expressions represented by the nodes of the DAG. But if the DAG is not a tree, this is a serious error, since the length of the strings produced by printing a DAG can grow geometrically with its size, and therefore the probabilistic guarantee becomes useless even for quite small DAGs.

Avoiding this error, the Vesta group computed the fingerprint of a node by concatenating the node's label with the {\it fingerprints} of its children---treating these fingerprints as 8-byte texts--- and fingerprinted the resulting text. With this strategy, the number of texts fingerprinted is proportional to the number of nodes of the DAG, and the total length of these texts is proportional to the number of edges of the DAG. Thus the method appears efficient and sound.

Alas, the method is not sound. Recall that the probabilistic guarantee is valid only if the strings being fingerprinted are independent of the magic number. But fingerprints themselves are dependent on the magic number, so the probabalistic guarantee is invalid whenever fingerprints are fingerprinted. The Vesta group was soon debugging an unexpected collision.

The moral is simple: the procedure Combine is a convenience, but it is also much more than a convenience. It should be the only way that you ever generate a fingerprint from another fingerprint. In particular, never treat a fingerprint as text to be passed to FromText.

Maybe the python foundation would like to host it? I'm pretty sure Guido has a fondness for Modula-3.

I just discovered SocksiPy - A Python SOCKS client module via a httplib2 - 0.4.0 announcement.

I wonder how SocksiPy relates to the socksForPython module that I hacked together back in 1994. I suppose the clue Guido gave me might have been our first meeting.

55 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!