Older blog entries for aleix (starting at number 46)

Over the last two and a half years we (at work) have been using baz as our distributed version control system. Baz is based on tla, the original GNU arch implementation by Tom Lord. After all this time we have applied more than two thousand patches (distributed in various projects). We also have been using Xtla (a wonderful Emacs front-end to GNU Arch) and lately DVC (its successor, which handles GNU Arch, Bazaar, git, Mercurial and Monotone). After some months thinking about it, I decided that it was time for a change, baz was too slow working with this magnitude of patches and even slower in my PowerBook. I also wanted a fast web interface and ArchZoom was not an option as it took ages to show changes between two revisions and mostly in any operation. So, I watched the Mercurial Google TechTalk and I also read some blog entries about git and Bazaar. I decided to give Mercurial a try as it also had complete DVC support and a great web interface (like the darcs one).

One of the things I needed was a tool to convert our baz repositories to Mercurial. As far as I knew, there was no such a tool, so I thought I could use arch2darcs to convert the repositories to darcs and then use the Mercurial convert extension to convert them to Mercurial. I tried it but it didn’t work. The Mercurial convert extension worked really well with darcs for my personal projects, so, why not write a GNU Arch converter?

After some days trying it, I have posted an issue to Mercurial BTS with a GNU Arch converter. It currently does not support directory renaming, but all other things I have tried work well. The converter is slow as I didn’t know how to compute a complete changeset between patches easily (any help?). The log files might not provide all the information needed (added, removed, modified files), so I decided to use the delta command which computes all this information, and this takes a bit of time.

To use the converter (in 0.9.5) copy it to the convert extension directory (hgext/convert) and edit the __init__.py file following, for example, the darcs converter (you’ll easily see how to add it). Use the converter at your own risk, as I am sure it needs more work. I would appreciate any comments and bug reports until it is included in Mercurial (if ever).

Yes, Baz is deprecated, but we are still using it at work, so to make my life easier I have created the Fink package for the latest version (probably there won’t be more). Download this file and, as usual, type:

$ tar zxvf baz-1.4.2-fink.tar.gz
$ sudo cp baz-1.4.2-fink/*
          /sw/fink/dists/local/main/finkinfo/
$ fink index; fink rebuild baz; fink install baz

Baz depends on some cryptographic libraries (gpgme, libgcrypt and libgpg-error). The updated packages are also included (until updated in Fink) in the provided file.

Please, note that when the official Baz Fink package is updated you will have to remove your local copy (from /sw/fink/dists/local/main/finkinfo/).

ECL has a new release, and thus a new Fink package update. It is still under validation, so until it is committed you can use this file. To install it, just type:

$ tar zxvf ecl-0.9j-fink.tar.gz
$ sudo cp ecl-0.9j-fink/*
          /sw/fink/dists/local/main/finkinfo/
$ fink index; fink rebuild ecl; fink install ecl

Please, note that when the official ECL Fink package is updated you will have to remove your local copy (from /sw/fink/dists/local/main/finkinfo/).

Today, I discovered how to share my Mac OS X (Leopard) screen with another non-Mac OS X computer. Not a big deal, as it is almost automatic, but I’m posting it here as I did a search and I didn’t find how to do it.

What you need to do in your Mac OS X box is enabling the Screen Sharing service from the Sharing options in the System Preferences.

Before establishing the connection you need to setup a password. This is done in the “Computer settings…” button, just like this:

Do not avoid the password step, if you do it the connection will not be established.

Finally, you only need a VNC client in your GNU/Linux box. Not all the clients work, one that it does is the TightVNC viewer. In Debian, you can just type:

apt-get install xtightvncviewer

Once installed, you only need to run it and supply the IP address of your Mac OS X box and the password you configured.

Enjoy it!

Erlang has a new release, and thus a new Fink package update. It is still under validation, so until it is committed you can use this file. To install it, just type:

$ tar zxvf erlang-otp-R12B-0-fink.tar.gz
$ sudo cp erlang-otp-R12B-0-fink/*
          /sw/fink/dists/unstable/main/finkinfo/languages
$ fink index; fink rebuild erlang-otp; fink install erlang-otp

A while ago, I answered a message, in the RTEMS users mailing list, about how to build raw applications for the ERC32 architecture. With raw applications I mean applications that are not dependent of RTEMS, which is the usual RTOS for this architecture. I am now writing about it just to make it easier for people to find it and to add some information to the explanation.

Before starting with the process, you need to get and build the RTEMS toolchain in order to compile your future applications (I used RTEMS 4.6.1). This is because the specific toolchain for ERC32 non-RTEMS applications is no longer maintained. Note, that you don’t need to install RTEMS itself, but you will need the source code. This video shows, step by step, how to download and build these tools for the i386 BSP, which should be almost identical for ERC32.

I’m not sure if what I’m going to explain is the best way to do it, but it has worked for me, and I have ran programs generated this way in a real board for a long time. I also think this may be useful for other architectures as well.

1. You need to get the following source files from the ERC32 BSP and copy them to your application’s directory:

    cpukit/score/cpu/sparc/asm.h
    cpukit/score/cpu/sparc/rtems/score/sparc.h
    cpukit/score/cpu/sparc/rtems/score/cpu.h
    c/src/lib/libbsp/sparc/shared/start.S
    c/src/lib/libcpu/sparc/reg_win/window.S
    c/src/lib/libcpu/sparc/syscall/syscall.S

2. Modify start.S so it does not call any RTEMS code, that is, commenting these lines:

/*
    call    __bsp_board_init
    nop

    set     (SYM(rdb_start)), %g6   ! End of work-space area
    st      %sp, [%g6]

    set     (SYM(Configuration)), %l1
    ld      [%l1+4], %l3            ! work_space_size
    sub     %sp, %l3, %sp           ! set %sp to area below work_space
    andn    %sp, 0x0f, %sp          ! align stack on 16-byte boundary
*/

3. Compile start.S, window.S and syscall.S. You can remove many things from the header files, but that’s up to you.

sparc-rtems4.6.1-gcc -DASM -c -o start.o start.S

4. Compile your C files (e.g. test.c).

sparc-rtems4.6.1-gcc -O4 -Wall -mcpu=cypress -c -o test.o test.c

5. Link your program. It will not work right now, as you need to generate two final files, my_bsp_specs and linkcmds.

sparc-rtems4.6.1-gcc -mcpu=cypress -Betc -specs my_bsp_specs -o test
window.o syscall.o test.o

Note the arguments -B and -specs. -specs is used to specify a file with options to override the default switches passed to cc1, cc1plus, as, ld… By default, RTEMS uses the file bsp_specs (found in c/src/lib/libbsp/sparc/erc32). You can edit it and see that it automaticaly links with the RTEMS libraries and adds its own start.S object file. The -B flag specifies where to find the executables, libraries, include files, and data files of the compiler itself. I have used an etc directory where I have stored my_bsp_specs. If any of the needed files is not found by the compiler it will automatically search its own paths.

So, what we need to do is to create our own file, my_bsp_specs, which will not use any RTEMS library and which will point to our compiled start.S object file.

Basically,

*endfile:
crtend.o%s crtn.o%s

*lib:
-T linkcmds

*startfile:
start.o crti.o%s crtbegin.o%s

*link:
-dc -dp -N -e start

The %s indicates to search the file in the compiler specific directories. I have removed the %s from start.o so it will search for it in the current directory.

Another important thing to note in this file is the -T linkcmds option. This indicates the linker script, which describes where the code and data for the application will reside in memory, to be used by the GNU linker. Fortunately, we can use the one that comes with RTEMS (found in c/src/lib/libbsp/sparc/erc32/startup), so copy it to the etc directory.

Finally, note that when linking, we have not specified the start.o file as it is already done in the my_bsp_specs file.

You might also find more information in the RTEMS BSP and Device Driver Development Guide.

During the last two years I have had the opportunity to write embedded software for the SPARC ERC32 architecture. The first piece of software we have written is an application that needs to fit in a 64 Kbytes PROM with a minimal running system that can execute periodic tasks, send packets over a network, load another application stored in an EEPROM that can be remotely patched and many other things. Before starting to write it from scratch we tried to fit RTEMS (a Real-Time Operating System) in the 64 Kbytes. We achieved a running RTEMS in 55 Kbytes, but there was only 10 Kbytes left (note that modern versions of RTEMS are smaller).

The following tips are the ones I have found useful to decrease my application size (they might not work in all architectures). Note that all the tips are C oriented and that gcc has been used.

I would really appreciate if you can share your tricks and I will be glad to include them in the list.

Compress application code

As the application is stored in a PROM, you will normally have a minimal code to perform some initializations and checks (e.g. RAM tests), and the rest of the code for the application itself. The main application will run from RAM, so you could compress the application using a simple compression algorithm such as LZSS, uncompress the application to RAM and run it. The decompression algorithm will be obviously uncompressed.

If you are working with an ERC32 processor check the MKPROM-ERC32.

Use inline functions judiciously

As it is well-known, performance issues should not be treated while developing. So, writing a lot of inline functions might not have great performance effects, but could have big size penalties if you don’t use them judiciously.

Compiler optimization flags

If performance is not important in your application (i.e. you have no hard real-time requirement), you can use the size optimization options provided by your compiler. For example, in gcc, you can use the -Os option to decrease the executable size. We reduced 5 Kbytes.

Note, that this will avoid inline functions optimization, but you can still tell gcc to include them passing the -finline-functions option.

Split functions into separate files

If you are developing different applications that might share most of the code, you may think on writing an static library. The point is that having a reusable library might affect the executable size. This is because, probably, not all the functions compiled will be used in all the applications. When gcc compiles a source file, all the functions in it will be linked in the final binary even if your code does not use them. Of course, if non of the functions of the source file is being used, they will not be linked. Splitting functions into separate files might solve the problem.

There is one way to avoid this (I haven’t tried it) with gcc using the -ffunction-sections option. This creates a separate section for every function. The linker can then use the –gc-sections option to discard unused sections.

Separate debug and release code

While writing embedded software you may need to have debug and release code. Clear examples are the printing or logging functions that might not be necessary in production code. If this is the case, separate the debug and release code. In C, you could have a serial port printing function in debug and an empty macro in release. This can be achieved via a building system, separating the definitions in different files (which I recommend), or using macros.

Do not use packed structures extensibly

By definition, packed structures are not aligned, this means that, depending on the architecture, the compiler needs to generate extra code to get members of structures. So, using packed structures extensibly may increase your application size and may decrease the performance.

Another reason for not using packed structures is portability.

Enumerations vs. constant objects

As described in this article, constant objects, such as:

unsigned int const ITEM_ID = 0×01200020;

may increase your application size. So, if you really have a few bytes left, you might be interested in changing the definition by using an enumeration:

enum { ITEM_ID = 0×01200020 };

I started reading PCL some time ago (only time for 10 chapters). This is my first Common Lisp book, and by now I’m not ready to write a great article about Common Lisp details, but if you are looking for a Common Lisp tutorial, this could be one.

Bomb it!

This may be something basic, but I lost a bit of time last week trying to find a bug at work, so I thought it was worth mentioning it.

When comparing signed and unsigned expressions of the same size, the compiler produces what it might be unexpected results. Suppose you have this code:

#include <stdio.h>

int
main (void)
{
  unsigned short int a = -12;
  signed short int b = -12;

  printf ("%s\n", (a == b) ? "OK" : "failed");

  return 0;
}

Here, you would probably expect an “OK” output, but surprisingly you get “failed“. Why is this? If you know about integer promotion, you may skip to the end if you don’t want to follow all the process.

Fortunately, I received a copy of K&R last week, so I started digging into the issue to try to understand why this was happening.

About equality operators, K&R section A7.10 reads

The equality operators follow the same rules as the relational operators…

Section A7.9, about relational operator, reads

The usual arithmetic conversions are performed on arithmetic operands…

So, how these arithmetic conversions work?

This is also clearly explained in K&R section A6.5 (the same rules apply to the C99 standard, section 6.3.1.8, Usual arithmetic conversions). The part that interests us is after having evaluated all the real type conversions, when it says

Otherwise, the integer promotions are performed on both operands…

So, before performing the comparison of our operands, both undergo integer promotion. Integer promotion (K&R, section A6.2) says that

If an int can represent all the values of the original type, then the value is converted to int; otherwise the value is converted to unsigned int.

An int can represent all the values of our operands, so after the integer promotion is performed, both operands have int type. Having a look back to our operands, we know that the a variable holds the value 0xFFF4, and after applying the integer promotion, it maintains the value. The same happens with variable b, that holds 0xFFFFFFF4 to represent -12. Clearly, both values are different and the check fails.

At the end of K&R section A6.5 this is explicitly explained

The new rules are slightly more complicated, but reduce somewhat the suprises that may occur when an unsigned quantity meets signed. Unexpected results may still occur whan an unsigned expression is compared to a signed expression of the same size.

Note, that this issue does not occur with 32-bit values, as both operands would be “0xFFFFFFF4″.

So, be careful when comparing unsigned and signed types.

During my vacation I have had the opportunity to add unit testing to SCEW (Simple C Expat Wrapper). I looked at various C unit testing frameworks and I decided to use Check. Most of them follow the xUnit approach, but I chose Check because tests run in a separate address space other than the test runner.

I found that writing test cases was a bit hard using Check’s syntax, for example following the manual you can write this integer check:

fail_unless (money_amount (m) == 5,
             "Amount not set correctly on creation");

This is fine if you are reading the code, but if the check fails the output doesn’t show you what the actual or expected values are, so the manual suggests changing it for:

fail_unless(money_amount (m) == 5,
            "Amount was %d, instead of 5", money_amount (m));

which is quite better than the first one, but to painful if you have to write it for every check. So, why not write a helper macro that checks for integers, prints the actual and expected values and also shows you the check that is being done?

#define CHECK_INT(A, B, MSG, ...)                                       \
  do                                                                    \
    {                                                                   \
      enum { MAX_BUFFER = 250 };                                        \
      static char buffer[MAX_BUFFER];                                   \
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \
      fail_unless ((A) == (B),                                          \
                   "(%s) == (%s) \n  Actual: %d \n  Expected: %d \n  %s", \
                   #A, #B, A, B, buffer);                               \
    }                                                                   \
  while (0);

With this macro you can now write code like this:

CHECK_INT (scew_list_size (list), N_ELEMENTS,
           "Number of children found searching by name");

which is really easy to read and in the test’s output you can see the actual and expected values, the performed test and the user message clarifying the intention of the check.

element.c:205:F:Core:test_search:0: (scew_list_size (list)) == (N_ELEMENTS)
  Actual: 1
  Expected: 12
  Number of children found searching by name

The same happens with strings, so instead of writing this:

fail_if (strcmp (money_currency (m), "USD") != 0,
         "Currency not set correctly on creation");

or this:

if (strcmp (money_currency (m), "USD") != 0)
  {
    fail ("Currency not set correctly on creation");
  }

we can write a string checking macro that shows the actual and expected strings and the check being done:

#define CHECK_STR(A, B, MSG, ...)                                       \
  do                                                                    \
    {                                                                   \
      if (strcmp ((A), (B)) != 0)                                       \
        {                                                               \
          enum { CHECK_MAX_BUFFER = 250 };                              \
          static char buffer[CHECK_MAX_BUFFER];                         \
          sprintf (buffer, MSG, ##__VA_ARGS__);                         \
          fail ("(%s) == (%s) \n  Actual: %s \n  Expected: %s \n  %s",  \
                #A, #B, A, B, buffer);                                  \
        }                                                               \
    }                                                                   \
  while (0);

As before, this would be the new output:

element.c:97:F:Core:test_accessors:0: (scew_element_name (element)) == (NAME)
  Actual: new_root
  Expected: root
  Element name do not match

Well, this is not a big deal, but I have found it quite useful. Below, is the list of macros I am using right now:

#define CHECK_INT(A, B, MSG, ...)                                       \
  do                                                                    \
    {                                                                   \
      enum { MAX_BUFFER = 250 };                                        \
      static char buffer[MAX_BUFFER];                                   \
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \
      fail_unless ((A) == (B),                                          \
                   "(%s) == (%s) \n  Actual: %d \n  Expected: %d \n  %s", \
                   #A, #B, A, B, buffer);                               \
    }                                                                   \
  while (0);

#define CHECK_STR(A, B, MSG, ...)                                       \
  do                                                                    \
    {                                                                   \
      if (strcmp ((A), (B)) != 0)                                       \
        {                                                               \
          enum { CHECK_MAX_BUFFER = 250 };                              \
          static char buffer[CHECK_MAX_BUFFER];                         \
          sprintf (buffer, MSG, ##__VA_ARGS__);                         \
          fail ("(%s) == (%s) \n  Actual: %s \n  Expected: %s \n  %s",  \
                #A, #B, A, B, buffer);                                  \
        }                                                               \
    }                                                                   \
  while (0);

#define CHECK_PTR(A, MSG, ...)                                          \
  do                                                                    \
    {                                                                   \
      enum { MAX_BUFFER = 250 };                                        \
      static char buffer[MAX_BUFFER];                                   \
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \
      fail_unless ((A) != NULL, "(%s) != NULL \n  %s", #A, buffer);     \
    }                                                                   \
  while (0);

#define CHECK_NULL_PTR(A, MSG, ...)                                     \
  do                                                                    \
    {                                                                   \
      enum { MAX_BUFFER = 250 };                                        \
      static char buffer[MAX_BUFFER];                                   \
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \
      fail_unless ((A) == NULL, "(%s) == NULL \n  %s", #A, buffer);     \
    }                                                                   \
  while (0);

Update 2007/08/11: I have updated the macros so variable number of arguments are allowed (see variadic macros).