Older blog entries for ingvar (starting at number 313)

Long while of not posting. Nothing (much) programming- related to mention, but...

New job. Turns out I am not emigrating right at the moment (bit of a last-minute surprise, there, as it happens). New employer seems to be a sensible multi-national corp, though.

Managed to score a (small) writing contract. That reminds me, need to poke the editor-in-chief that I haven't seen the paper copies of the contract yet. The task is to write an article about the BESK machine, a piece of hardware I have only seen the console of, but it is nonetheless a machine that warms the cockles of my heart.

Also noticed that there's been a bit of movement in the SHA1-cracking. Apparently Amazon EC2 GPU instances are real good for SHA1-based password hashes. I don't know, exactly, what this means for SHA1-based HMACs, but I am pretty sure it's not entirely pleasant. So, need to go back to the drawing board and switch out SHA1 and switch in something else for NOCtool's network protocol.

Not managed to poke code, as such, but I've checked (one of) the monitored variables (ifOutOctets on the ethernet port of a router at home). Next step is to write some more methods on the SHOW gf, to generate prettier graphs (background, grid, some text, that sort of thing).

Would probably need to add a selector-thing to allow easy graphing of each of the things monitored, but taht's but a keyword away.

Hm, there's some odd interaction between Swank and the existing NOCTool ping monitors. Nonetheless, there's now SNMP monitoring of a single network device running. Howpefully, I'll get a chance to poke around in the code tonight or tomorrow morning, to see if it's actually doing anything interesting.

So, back to poking at the NOCTool codebase again. This time, the intent is to get some SNMP support into it, to more easily be able to monitor routers and switches. There's also some opportunity to monitor server hosts via SNMP, but at the moment unix-based hosts can be adequately monitored via SSH.

First, I created a class for "things monitored by SNMP" (with curucial details like community strings and SNMP version in it, plus slots for anything we're monitoring via SNMP; this is currently just interfaces). Then, an SNMP configuration stanza for the config reader. Inside the SNMP stanza, configuration stanzas for anything monitored by SNMP (again, just interfaces at the moment).

The intent is to let an empty interface spec is to mean "auto-discover all interfaces, then monitor them" and for anyone wanting a subset, simply naming the interfaces.

The actual monitoring starts with an snmp-walk of the ifTable.ifEntry.ifDescr sub-tree, mapping names to current SNMP indexes, then passing the resulting hash table down to each "get stuff for this specific interface" function (taking the current SNMP session, mapping table and interface object as arguments).

So far, I am happy with what I have, but it's yet to be used in anger.

I was reading titus's recent post about data structures for k-mer filtering.

ONe thing struct me. Titus, you looked at binary Bloom filters, but seemingly ignored Counting Bloom filters (where instead of indexing a bit with your hashes, you index a counter of a suitable size, you do run into problems if you need both increments and decrements, since once you it max on the counter, you can no longer decrement).

Of course, it's not necessarily trivial going from a Bloom filter (counting or not) to the key, whereas the key is typically available when using a normal hash table.

I suspect, but have not verified, that a counting Bloom combined with the parallelization opportunity you found (and that one is the big win, I think) would be an equally interesting approach.

This morning, I finished off and packaged up a library (CL, linux; other OSes with CL implementations and OS-provided random (or random-backed pseudorandom) devices can probably be grafted in, but would require volunteers to test) to extract random bits from /dev/urandom and use those essentially unadorned to generate random numbers.

There's a single exported function from the package, RANDOMNESS:RND and takes an integer argument (I guess you could try feeding it a float, but it wouldn't do you that much good). That function then proceeds to extract log2 N bits and uses rejection smapling to find an integer in the 0..(N-1) span. As the bits sucked out of the internal pool run out (each bit is used only the once), they're replenished from the OS-provided randomness source.

All very nice, with some interesting corner cases for the unwary and probably still bug-ridden. But, such is life.

Download from here.

Neat(ish) hack...

Sometimes, I find myself writing anonymous functions, to fill out keyword arguments for functions or adapting the argument order. So, I thought, how hard would it be to write a macro to write the code for me? Turns out, not very complex, at all. The formatting is not QUITE what I started out with, as the Advogato edit box is a bit on the short end, but, hey...


(defmacro _ (form)
 (flet ((is-arg (sym)
	  (ignore-errors
	   (and (char= (char (symbol-name sym) 0) #\_)
		(cons
		 (parse-integer (symbol-name sym)
				:start 1)
		 sym)))))
   (let ((syms (loop for arg in form
		  for temp = (is-arg arg)
		  if temp collect temp)))
    `(lambda
	 ,(mapcar #'cdr (sort syms #'< :key #'car))
       ,form))))

With this in hand, you can, for example, easily make an adapter to parse C-style hex constants:


  (_ (parse-integer _1 :radix 16 :start 2))

Not that the lambda-wrapping of this would've been much more complex and I am not entirely sure this wins as far as readability is concerned, but that is as it may be. It's if nothing else a neat macro that would be more than a little tricky to pull off with a less capable macro facility.

So, one of the users of my Image library asked if there wasn't any way of using a font other than the rather ugly, built-in one. I said, roughly, "you can define a new one, in an analogous fashion to how the built-in oine is done?"

Apparently, that was not the wished-for answer and I started thinking. Bit-mapped fonts are (relatively) easy to deal with. There's a helluva lot of bitmapped fonts for X11. So, I set forth to code up a reader for PCF fonts, converting the bitmaps to internal format and some multi- font support in Image. All done, now.

Common Lisp is still very pleasant when dealing with binary file I/O, even if I wished that there was an easy way of changing the binaryness of a file on the fly.

Currently tooling away at asked-for functionality for my IMAGE library (specifically, a request was made to see if I can scare up another font or three, so I am currently noodling on a PCF font-file reader, as you do).

Common Lisp continues to be amazingly convenient for "binary I/O", although it does require a certain mind-set to consider it convenient, I guess.

I've just packaged up a new release of my IMAGE library, with some new functionality in place. It now supports copying (parts of) an image into another image (with either the same alpha across the whole copied section or with a provided alpha map, so one can do Clever Stuff that way).

I suppose I should write documentation for the image library at some point, but...

304 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!