Older blog entries for aleix (starting at number 70)

32-bit and 64-bit return values in OS X

I've spent a couple of days with a silly compiler problem. I have to admit that it was my fault but, at least, I learned something in the process. To make a long story short: do not forget to include header files and enable all warnings (-Wall).

In C, you can use a function without including its header file. Then, depending on how you set the compiler flags, it might not complain as long as it can find the function at linking time.

Imagine you have this function (defined in foo.h and implemented in foo.c) :

unsigned long foo_add (void);

and you use it in another module bar.c without including the foo.h header:

//#include "foo.h"

unsigned long bar_add (void)
{
  return foo_add () + foo_add ();
}

Compile with: gcc -c -O2 bar.c (we just missed -Wall)

If we disassemble the code of bar.o we get:

$ otool -tV bar.o
bar.o:
(__TEXT,__text) section
_bar_add:
...
000000000000000b callq _foo_add
0000000000000010 movl  %eax, %ebx
...
0000000000000024 ret

Now, let's try to include the header foo.h and disassemble again:

$ otool -tV bar.o
bar.o:
(__TEXT,__text) section
_bar_add:
...
0000000000000009 callq _foo_add
000000000000000e movq  %rax, %rbx
...
000000000000001f ret

Note how the compiler uses two different instructions: movl (32-bit) in the first case, when it really doesn't know anything about the foo_add function, and movq (64-bit) when it knows that foo_add returns a 64-bit value.

This can lead to unexpected behavior as I have found in the code I was working on. And actually, I have only found this in OS X.

So, please do not forget to include header files and enable -Wall. It will save you from some headaches.

Syndicated 2013-04-25 00:01:08 from aleix's blog

32-bit and 64-bit return values in OS X

I've spent a couple of days with a dummy compiler problem. I have to admit that it was my fault but, at least, I learned something in the process. To make a long story short: do not forget to include header files and enable all warnings (-Wall).

In C, you can use a function without including its header file. Then, depending on how you set the compilers flags, it might not complain as long as it can find the function at linking time.

Imagine you have this function (defined in foo.h and implemented in foo.c) :

uint64_t foo_add (void);

and you use it in a program bar.c without including the foo.h header:

#include <stdint.h>
#include <stdio.h>

int
main (int argc, char *argv[])
{
  printf ("%d\n", foo_add ());

  return 0;
}

(yes, printf will complain in this case because of type mismatches, just imagine printf is something else).

If we disassemble the code of our program we get:

$ otool -tV bar
bar.o:
(__TEXT,__text) section
_main:
...
0000000000000006 callq  _foo_add
...
0000000000000012 movl   %eax, %esi
...
000000000000001e ret

Now, let's try to include the header foo.h and disassemble again:

$ otool -tV bar
bar.o:
(__TEXT,__text) section
_main:
...
0000000000000004 callq  _foo_add
...
0000000000000010 movq  %rax, %rsi
...
000000000000001d ret

Note how the compiler uses two different instructions: movl (32-bit) in the first case, when it really doesn't know anything about the foo_add function, and movq (64-bit) when it knows that foo_add returns a 64-bit value.

This can lead to unexpected behavior as I have found in the code I was working on. And actually, I have only found this in OS X.

So, please do not forget to include header files and enable -Wall. It will save you from some headaches.

Syndicated 2013-04-24 22:56:42 from aleix's blog

add1, sub1 and + recursive process

In The Little Schemer you are asked to write the function + using the functions zero?, add1 and sub1, such as the result of (+ 46 12) is 58.

(define add1
  (lambda (n) (+ n 1)))

(define sub1
  (lambda (n) (- n 1)))

The solution given is:

(define o+
  (lambda (n m)
    (cond
      ((zero? m) n)
      (else (add1 (o+ n (sub1 m)))))))

which is correct but has a little problem.

If you are like me (hopefully not), you might have read twenty times the first chapters of SICP (and haven't gone any further). Remember about recursive and iterative processes and recursive procedures? In our o+ case we have a recursive procedure that generates a recursive process, as the process generates a chain of deferred add1 operations.

With a recursive process we can easily blow our stack:

scheme@(guile-user)> (o+ 10 100000)
While executing meta-command:
ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack overflow" ())'.

This is because this will generate:

(add1 (add1 (add1 (add1 ...... n))))

with as many add1 as m.

So, how to improve this? With a recursive procedure that generates an iterative process:

(define o+
  (lambda (n m)
    (cond
      ((zero? m) n)
      (else (o+ (add1 n) (sub1 m))))))

which will generate:

(o+ 11 99999)
(o+ 12 99998)
...
(o+ 100010 0)

and doesn't overflow the stack.

scheme@(guile-user)> (o+ 10 100000)
$1 = 100010

Yes, this is a tail-recursive function.

Syndicated 2013-04-13 14:56:15 from aleix's blog

The Law of Car

In The Little Schemer, The Law of Car is defined as:

The primitive car is defined only for non-empty lists.

In the implementation given for (firsts l), it seems to me that the law is broken:

(define firsts
  (lambda (l)
    (cond
      ((null? l) '())
      (else (cons (car (car l))
                  (firsts (cdr l)))))))

As (firsts '()) is '(). So, this would actually fail (in guile):

$ (firsts '((a b) () (e f)))
In procedure firsts:
In procedure car: Wrong type argument in position 1 (expecting pair): ()

I think a correct implementation would be:

(define (firsts l)
  (cond
   ((null? l) '())
   (else (cond
          ((null? (car l)) (firsts (cdr l)))
          (else
           (cons (car (car l))
                 (firsts (cdr l))))))))

in which we take care of the non-empty list before getting the first typical element (car (car l)). This would result in:

$ (firsts '((a b) () (e f)))
(a e)

Syndicated 2013-04-10 06:18:09 from aleix's blog

Install Emacs packages from command line

Lately, I have found myself playing with packages and my .emacs too much. Sometimes I had to comment the use of a package (e.g. smex) because it was not installed.

So, at the end, I wrote this basic elisp to install a package from the command line.

$ emacs --batch --expr "(define pkg-to-install 'smex)" -l emacs-pkg-install.el

The elisp script looks like this:

;;
;; Install package from command line. Example:
;;
;;   $ emacs --batch --expr "(define pkg-to-install 'smex)" -l emacs-pkg-install.el
;;

(require 'package)

(add-to-list 'package-archives
             '("melpa" . "http://melpa.milkbox.net/packages/") t)

(add-to-list 'package-archives
             '("marmalade" . "http://marmalade-repo.org/packages/") t)

;; Fix HTTP1/1.1 problems
(setq url-http-attempt-keepalives nil)

(package-refresh-contents)

(package-install pkg-to-install)

For convenience, you can wrap it in a shell script and simply type:

$ ./emacs-pkg-install.sh smex

The shell script:

#!/bin/sh

if [ $# -ne 1 ]
then
  echo "Usage: `basename $0` <package>"
  exit 1
fi

emacs --batch --eval "(defconst pkg-to-install '$1)" -l emacs-pkg-install.el

Syndicated 2013-01-08 15:33:36 from aleix's blog

Install Emacs packages from command line

Lately, I have found myself playing with packages and my .emacs too much. Sometimes I had to comment the use of a package (e.g. smex) because it was not installed.

So, at the end, I wrote this basic Emacs lisp to install a package from the command line. Just type:

$ emacs --batch --expr "(define pkg-to-install 'smex)" -l emacs-pkg-install.el

The script looks like this:

;;
;; Install package from command line. Example:
;;
;;   $ emacs --batch --expr "(define pkg-to-install 'smex)" -l emacs-pkg-install.el
;;

(require 'package)

(add-to-list 'package-archives
             '("melpa" . "http://melpa.milkbox.net/packages/") t)

(add-to-list 'package-archives
             '("marmalade" . "http://marmalade-repo.org/packages/") t)

;; Fix HTTP1/1.1 problems
(setq url-http-attempt-keepalives nil)

(package-refresh-contents)

(package-install pkg-to-install)

For convenience, you can wrap it in a shell script and simply type:

$ ./emacs-pkg-install.sh smex

The shell script:

#!/bin/sh

if [ $# -ne 1 ]
then
  echo "Usage: `basename $0` <package>"
  exit 1
fi

emacs --batch --eval "(defconst pkg-to-install '$1)" -l emacs-pkg-install.el

Syndicated 2013-01-08 10:20:12 from aleix's blog

git-cherry-base

If you use git, you might find yourself wanting to merge a huge list of commits into a branch, but you might not want all of them. git-rebase already helps with this task but it becomes less convenient when you have lots of commits as you may not remember all of them and also because it doesn't show you the contents of each commit

git-cherry-base is an interactive script (written in Guile) that lets you choose which commits you'd like to merge and even which parts of the commit you want. Given a list of revisions (obtained from git-rev-list) it will show the contents for each commit and will let you choose if you want to merge the commit, skip it or merge only parts of it.

Usage: git-cherry-base rev-file

I wrote it some months ago as I couldn't find any other command line tool to do this and, well, it was just a good excuse to hack something in Scheme.

Syndicated 2012-11-15 00:43:55 from aleix's blog

Packing and unpacking bit structures in Python

Last week, I released the first version of the BitPacket Python module which allows you to pack and unpack data like the struct and array modules, but in an object-oriented way. At work I needed an easy way to create network packets and at that time I did not know the existence of the struct and array modules, so I googled a bit and I found out the BitVector class for a memory-efficient packed representation of bit arrays, which I decided to use for my purpose.

I implemented three classes, BitField, BitStructure and BitVariableStructure (the lastest two are derived from BitField). A network packet would be represented by the BitStructure class, which at creation does not contain any field, and the idea is that any BitField subclass might be added to it.

I'll will show you the most basic example. Suppose, you need a simple network packet like the one below:

+---------------+-------------------+
|  id (1 byte)  |  address (4 byte) |
+---------------+-------------------+

You could easily create a network packet using BitStructure, like this:

>>> bs = BitStructure('mypacket')
>>> bs.append(BitField('id', BYTE_SIZE, 0x54))
>>> bs.append(BitField('address', INTEGER_SIZE, 0x10203040))

and print its contents:

>>> print bs
>>> (mypacket =
>>>   (id = 0x54)
>>>   (address = 0x10203040))

In order to unpack an incoming packet, we could use the variable created above or a new one without default values:

>>> bs = BitStructure('mypacket')
>>> bs.append(BitField('id', BYTE_SIZE))
>>> bs.append(BitField('address', INTEGER_SIZE))

In order to unpack an incoming array of bytes, we would do the following:

>>> data = array.array('B', [0x38, 0x87, 0x34, 0x21, 0x40])
>>> bs.set_stream(data)

We can then access to the packet fields by their name:

>>> print '0x%X' % bs['id']
0x38
>>> print '0x%X' % bs['address']
0x87342140

There are a lot more possibilities to pack and unpack bit field structures by using BitStructure and BitVariableStructure. You can see all of them in the module's online documentation.

Syndicated 2011-08-15 14:40:39 from aleix's blog

Emacs 23.3 for RHEL 6

Emacs version in RHEL 6.1 is very outdated, 23.1 (which was released on July 2009). I needed a newer version to run Geiser at work, so after searching for the package in the typical places (rpmfind.net and rpm.pbone.net) and Google I decided to build my own one. For it, I just followed these instructions on how to build Source RPM packages and did some minor updates on the RPM spec file.

So, simply download emacs-23.3-1.el6.src.rpm and follow these instructions:

This will only create an rpmbuild directory in your home:

$ rpm -i emacs-23.3-1.el6.src.rpm

Install rpm-build if you don't have it, and build the Emacs RPMs:

$ cd $HOME/rpmbuild/SPECS
$ rpmbuild -ba emacs.spec

Finally, upgrade your old installed Emacs and happy hacking!

$ cd $HOME/rpmbuild/RPMS
$ sudo rpm -U emacs-23.3-1.el6.x86_64.rpm emacs-common-23.3-1.el6.x86_64.rpm

Syndicated 2011-06-28 13:12:12 from aleix's blog

more tekuti hacks

I have had less time lately, but I still found some minutes to hack on tekuti. The most important new feature is performance. tekuti is now one or two order of magnitudes faster than before. No more delays when accessing posts or listing tags or archives posts. Basically, accesses to git have been reduced to the minimumn. So, these are the new features since last post:

  • Fixes: search works again.

  • Performance: reduction of git accesses (and adding Look-Up-Tables) has improved speed dramatically.

  • New widgets: blank and page-rank.

  • Page navigation: the new links at the bottom (go to the end of the page) allow navigation through all blog history.

  • Cosmetics: post lists (in tags or archives) show the post date besides the title...

You can check out all these new features from gitorious:

git clone git://gitorious.org/~aleix/tekuti/acf-tekuti.git

Happy hacking!

Syndicated 2011-03-01 15:58:43 from aleix's blog

61 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!