12 Jun 2006 titus   » (Journeyer)

Random citings

A Java coder admits to liking Python: Simon sez. But... if the problem is complex, shouldn't the tools you use to solve the problem also be complex?

;)

Terrorism

Here's a (recently rare) foray into politics for ya, just to keep you on your toes...

In response to a post to IP by Hiawatha Bray, I offered my reasons for disliking both the Bush Administration's approach to the War on Terror, and the naive "Golden Rule"-ism of some liberals. (Incidentally, I count myself as a liberal in these matters.)

Charlie Stross (a fantastic sci-fi writer) weighed in on the absurd "asymmetric warfare"</a> statement made about the four Gitmo suicides.

Authoring for O'Reilly

Well, Jason spoke, so I'll confirm: Grig, Jason and I have contracted to write a series of short PDFs for O'Reilly.

The umbrella topic is 'Testing Stuff', and a brief list of individual topics we hope to cover includes: Intro to Web Testing (covering twill & intro Selenium); Advanced Web Testing (some twill, lots more Selenium); Unit Testing in Python; Continuous Integration with buildbot; Python Testing with FitNesse; and perhaps more. Each book will cover a discrete chunk of material, and we hope the series will pull the individual books together into a whole, as well.

The first two, together with a (free) introductory book on testing, should be available within 3 months.

(And remember, Gentlemen prefer PDFs!)

figleaf

When last we met, I professed a rewrite of coverage.py. Since then, I realized that my tokenize-based method of extracting interesting lines of code ... didn't work. There were several situations where lines of code simply wouldn't be counted, because there wasn't enough context to determine whether or not it was an actual expression. Specifically, this kind of code broke the parser:

def f(
   a =
     (lambda x: x + 1)(1),
   b =
     (lambda
           y: y * 2)
):
   pass

After flailing a bit, I realized that you really needed the AST to properly determine what lines of code are worth counting. This realization was helped by the fact that the sys.settrace 'line' tracing function is only called on the 'lambda' and 'def' lines, above, and not on the 'a=' and 'b=' code. (Kudos to Ned Batchelder for including so many nasty evil tests with coverage.py -- I just stole his code. ;)

I delved into coverage.py and confirmed my suspicion that some nasty AST visitation was occurring, using code based on the 'compiler' package. Moreover, coverage.py used code way beyond me to determine what was actually an executable line of code... and then did even more clever things to count that code even when the sys.settrace function didn't hit it.

Now, my rule is, if it's too complicated for me to understand, it shouldn't be in software I write. So I set myself a new goal: make a really really simple coverage-measuring utility that (a) only counts lines that Python actually "executes" (as measured by sys.settrace); and (b) I can understand.

In the process of working on implementing this with the parser module, I discovered a few amusing details about Python. First: can you guess which lines of code are "executed" in the following?


def f(): "a" 5 "b" 6

f()

Well, in a bit of a surprise to me, it turns out that only the numbers are counted:

>- def f():
       "a"
>-     5
       "b"
>-     6

>- f()

(where '>-' represents executed lines). Yep, only the numbers, not the stringS! There are two reasons for this, I think: one is that each number is actually a numerical expression, to be evaluated and replaced by its value, while strings are just literals; and the other is that this doesn't count docstrings.

I'm also a bit surprised by some aspects of the AST that is generated, too. For example, here's what my AST pretty-printer outputs for the number "5", all alone in a file:

 file_input
  stmt
   simple_stmt
    small_stmt
     expr_stmt
      testlist
       test
        and_test
         not_test
          comparison
           expr
            xor_expr
             and_expr
              shift_expr
               arith_expr
                term
                 factor
                  power
                   atom
                    NUMBER ('5', 1)
    NEWLINE ('', 1)
  NEWLINE ('', 1)
  ENDMARKER ('', 1)

Is this really a necessary part of the AST?

I clearly need to read up on this more ;).

The last thing I did was build in an optimization: coverage.py uses a global trace function that is continually reassigned to the local trace function by calls into new code blocks. This means that all

Python code is traced. Figuring that this would be kind of a speed drain, I separated out the logic into a global trace function that only set the local trace function on a call into interesting code, where "interesting" could be specified by the user. In other words, rather than tracing coverage on everything, only code executing in user-specified modules would be traced.

The early results are pretty positive. With the usual caveats about naive benchmarking -- which this certainly is -- I found the following times for running the twill tests in nose:

  1. coverage.py -- 30 seconds total
  2. figleaf.py, tracing *all* code -- 23 seconds total

  3. figleaf.py, tracing *only* user code -- 15 seconds total
  4. no coverage analysis at all -- 7 seconds total.

Naively, it looks like I get a ~20% speedup from switching to my naive AST implementation, and I get another ~25% speedup from only looking at local code. Neat, huh? (Sadly, you still lose a factor of 2 because of the code coverage!)

Here's my (hideously ugly and un-re-factored, yes) implementation of a class to turn code into "interesting line numbers":

class _LineGrabber:
    def __init__(self, fp):
        self.lines = sets.Set()

ast = parser.suite(fp.read()) tree = parser.ast2tuple(ast, True)

self.find_terminal_nodes(tree)

def find_terminal_nodes(self, tup): """ Recursively eat an AST in tuple form, finding the first line number for "interesting" code. """ (sym, rest) = tup[0], tup[1:]

line_nos = [] if type(rest[0]) == types.TupleType: ### node

for x in rest: min_line_no = self.find_terminal_nodes(x) if min_line_no is not None: line_nos.append(min_line_no)

if symbol.sym_name[sym] in ('stmt', 'suite', 'lambdef', 'except_clause') and \ line_nos:

# store the line number that this statement started at self.lines.add(min(line_nos))

else: ### leaf if sym not in (token.NEWLINE, token.STRING, token.INDENT, token.DEDENT): return tup[2]

if line_nos: return min(line_nos)

## use like so: lines = _LineGrabber(open(filename)).lines

I will be eternally grateful to anyone who points out why this is a stupid way to do things, and/or can improve the logic. (I already know it's unmaintainable.)

I'll post the full figleaf module sometime soon; right now it's too dangerous to let loose on the Internet. If you're willing to handle such dangerous material, just drop me a line.

--titus

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!