Advogato blog for mjw

Fosdem SystemTap Interview

Fri, 22 Jan 2010 00:08:56 GMT

Did an interview for FOSDEM about SystemTap. It discusses a wide range of topics. About when I got involved with Free Software, working for Red Hat, how FOSDEM helped the libre Java community, getting Fedora more observable by adding static markers into programs, the history of observation tools (tracers, profilers, debuggers) on GNU/Linux, comparisons to other tools like DTrace, GUI frontends, Eclipse integration, the future of SystemTap and of course why you should come to FOSDEM.

Fosdem talk – Full System Observability with SystemTap

Sun, 3 Jan 2010 21:08:01 GMT

Really looking forward to Fosdem next month. This year I will be giving a talk What is my system doing – Full System Observability with SystemTap during one of the main tracks. There will be some demos of the new systemtap java and python tracing support that I blogged about earlier.

Michael Meeks on Copyright Assignment

Tue, 22 Dec 2009 22:13:01 GMT

Michael Meeks published a though provoking essay called Some thoughts on Copyright Assignment. It is a must read when working with Free Software projects that request legal assignment of contributions to a corporation.

It contains sane recommendations both for individual contributors, project leaders and corporations seeking assignments.

Next step – SystemTap java hotspot jstack() support

Mon, 14 Dec 2009 23:08:56 GMT

Just checked in support for getting java backtraces from hotspot through systemtap. This is the next step in making not just the kernel and native programs observable, but also runtime based languages with SystemTap. And it is pretty powerful. It allows you to answer the question “How did I get here?” in combination with any of the SystemTap tapsets for VM level tracing, Java method entry/exit tracing, Native methods (JNI) tracing, or any other basic SystemTap function, statement or process probe that points into your java process. Just add one of the print_jstack() variants to your code (possibly including native frame and method signatures) and suddenly it is really easy to see what triggered a particular code path:

$ stap -e 'probe hotspot.jni.CallObjectMethod { log (probestr); print_jstack() }' -c 'java ModalTest'
CallObjectMethod(env=0x7f9d1000a1b8,obj=0x7f9d73efcaf8,methodid=0x7f9de4088b70) sun/font/FontManager.getFont2D sun/java2d/SunGraphics2D.checkFontInfo sun/java2d/SunGraphics2D.getFontInfo sun/java2d/pipe/GlyphListPipe.drawString sun/java2d/SunGraphics2D.drawString sun/swing/SwingUtilities2.drawString sun/swing/SwingUtilities2.drawStringUnderlineCharAt javax/swing/plaf/basic/BasicGraphicsUtils.drawStringUnderlineCharAt sun/awt/X11/XButtonPeer.paintText sun/awt/X11/XButtonPeer.paint sun/awt/X11/XButtonPeer.paint sun/awt/X11/XRepaintArea.paintComponent sun/awt/RepaintArea.paint sun/awt/X11/XComponentPeer.handleEvent java/awt/Component.dispatchEventImpl java/awt/Component.dispatchEvent java/awt/EventQueue.dispatchEvent java/awt/EventDispatchThread.pumpOneEventForFilters java/awt/EventDispatchThread.pumpEventsForFilter java/awt/EventDispatchThread.pumpEventsForFilter java/awt/Dialog$1.run java/awt/event/InvocationEvent.dispatch java/awt/EventQueue.dispatchEvent java/awt/EventDispatchThread.pumpOneEventForFilters java/awt/EventDispatchThread.pumpEventsForFilter java/awt/EventDispatchThread.pumpEventsForHierarchy java/awt/EventDispatchThread.pumpEvents java/awt/EventDispatchThread.pumpEvents java/awt/EventDispatchThread.run
You currently need both IcedTea6 from mercurial and Systemtap from git to play with it. But both projects are planning new releases soon.

FudCon Success – Systemtap meets Python

Mon, 7 Dec 2009 07:09:25 GMT

At FudCon, David Malcolm, Jon VanAlten, Will Cohen and I sat down, had some fun and made tracing python methods through systemtap possible:
0 python(20122): => search_function in Lib/encodings/__init__.py:71 15 python(20122): => normalize_encoding in Lib/encodings/__init__.py:49 37 python(20122): <= normalize_encoding 170 python(20122): => <module> in Lib/encodings/utf_8.py:8 193 python(20122): => IncrementalEncoder in Lib/encodings/utf_8.py:18 206 python(20122): <= IncrementalEncoder 251 python(20122): => IncrementalDecoder in Lib/encodings/utf_8.py:22 264 python(20122): <= IncrementalDecoder 310 python(20122): => StreamWriter in Lib/encodings/utf_8.py:25 323 python(20122): <= StreamWriter 340 python(20122): => StreamReader in Lib/encodings/utf_8.py:28 353 python(20122): <= StreamReader 367 python(20122): <= <module> 391 python(20122): => getregentry in Lib/encodings/utf_8.py:33 410 python(20122): => __new__ in Lib/codecs.py:77 429 python(20122): <= __new__ 440 python(20122): <= getregentry 462 python(20122): <= search_function

The coolest part is that it works through the existing patch to python for adding dtrace support. Some small tweaks to the autoconf detection was needed, but the rest was used as is.

If you want to learn how to add static user space probes to your program/package please see Will’s excellent guide. Adding User Space Probing to an Application: A simple example adding markers to a user-space application with SystemTap.

Going to FudCon

Wed, 2 Dec 2009 20:12:04 GMT

Currently stuck at heathrow airport… But I will make it to Canada somehow for FudCon. Hopefully with some nice systemtap jstack java hotspot backtrace examples.

They finally actually started to work. Weeee!

GPLed JDK 7 M5 + Jigsaw binary packages

Thu, 19 Nov 2009 22:13:46 GMT

After my somewhat cranky post about the JDK 7 M5 binaries being distributed under not very free and social terms. I was really happy to see Mark Reinhold’s announcement of the JDK 7 M5 + Jigsaw binary packages:

These binaries are licensed under the GPL (v2) with the Classpath Exception, plus the Assembly Exception.

Thank you Mark! Thank you Sun! You did make my day.

Trusting companies with your code…

Sun, 15 Nov 2009 12:20:09 GMT

Mark Reinhold announced JDK 7 Milestone 5 which is derived from OpenJDK7. Lots of people contributed to that. But sadly Sun decided they weren’t going to release this as Free Software. I wanted to research this JDK7 M5 and compare it to the work we did on OpenJDK and IcedTea, but apparently I am not allowed to…

Sun might have the right to do this given they only use contributions granted to them under their SCA terms. But the following clauses from the JDK7 license seem somewhat unfair given that we all contributed parts to OpenJDK.

“Licensed Software is “Confidential Information”. Licensee may not disclose or use Confidential Information”
“Licensee may not modify or create derivative works of the Licensed Software, or reverse engineer, disassemble or decompile binary portions of the Licensed Software, or otherwise attempt to derive the source code from such portions.”
“Licensee shall have no right to use the Licensed Software for productive or commercial use.”
“LICENSEE DUTIES Licensee agrees to evaluate and test the Software for use in Licensee’s software environment and provide feedback to Sun in a manner reasonably requested by Sun. Any and all test results, error data, reports or other information, feedback or materials made or provided by Licensee relating to Software (collectively, “Feedback”) are the exclusive property of Sun and Licensee here by assigns all Feedback to Sun at no cost to Sun.”
“This Agreement will commence on the date on which Licensee receives Licensed Software (the “Effective Date”) and will expire ninety (90) days from the Effective Date, unless terminated earlier as provided herein.”
“Upon termination or expiration of this Agreement, Licensee will immediately cease use of and destroy Licensed Software, any copies thereof.”
“It is understood and agreed that, notwithstanding any other provision of this Agreement, Licensee’s breach of this Agreement will cause Sun irreparable damage for which recovery of money damages would be inadequate…”

There is no reason for Sun to act in this anti-social way. There are free replacements for the previously encumbered binary blobs (thanks IcedTea!). So anybody that wants to can distribute a completely free version. And redistributing other peoples contributions under such draconian one-sided terms is totally unnecessary (and rude!). If someone wants to add the previously proprietary binary blobs one can do that under the terms of the OpenJDK Assembly Exception to the GPL that was specially drafted to allow this. So there is no need to not to redistribute the free parts under the GPL as everybody else is doing.

Come on Sun, tear down this proprietary legal wall that divides users from the developers that contributed all this code to be free and open.

Trusting companies with your code…

Sat, 14 Nov 2009 12:20:31 GMT

“Licensed Software is “Confidential Information”. Licensee may not disclose or use Confidential Information”
“Licensee may not modify or create derivative works of the Licensed Software, or reverse engineer, disassemble or decompile binary portions of the Licensed Software, or otherwise attempt to derive the source code from such portions.”
“Licensee shall have no right to use the Licensed Software for productive or commercial use.”
“LICENSEE DUTIES Licensee agrees to evaluate and test the Software for use in Licensee’s software environment and provide feedback to Sun in a manner reasonably requested by Sun. Any and all test results, error data, reports or other information, feedback or materials made or provided by Licensee relating to Software (collectively, “Feedback”) are the exclusive property of Sun and Licensee here by assigns all Feedback to Sun at no cost to Sun.”
“This Agreement will commence on the date on which Licensee receives Licensed Software (the “Effective Date”) and will expire ninety (90) days from the Effective Date, unless terminated earlier as provided herein.”
“Upon termination or expiration of this Agreement, Licensee will immediately cease use of and destroy Licensed Software, any copies thereof.”
“It is understood and agreed that, notwithstanding any other provision of this Agreement, Licensee’s breach of this Agreement will cause Sun irreparable damage for which recovery of money damages would be inadequate…”

IcedTea and OpenJDK backups

Mon, 12 Oct 2009 17:07:58 GMT

Since the various OpenJDK servers are seeing major ongoing outages the last couple of days it might be good to point out that at least the source code from the master jdk6 and jdk7 mercurial forests do have backups: http://classpath.wildebeest.org/hg/. These are read-only of course. You will also find backups of the various IcedTea mercurial repos there. The IcedTea builds can currently fail if you don’t have openjdk.tar.gz and/or hotspot.tar.gz around in some configurations. You can also get those from the IcedTea autobuilder/tester (and then use configure --with-openjdk-src-zip and/or --with-hotspot-src-zip). Besides creating binaries and test reports, the autobuilder also keeps copies of all the sources used to create the latest successful build run around: http://icedtea.classpath.org/builds/.