Well, after reading this message from notting about speeds and sizes of xz compression at various levels, I got curious about how gzip falls into the picture. So I wrote a little script to do some naive testing, found a 64MB text file (an sql database dump), and ran a naive benchmark. First, the script so you can all see what horrible assumptions I'm making:

#!/bin/sh                                              

 LZOP='lzop -U'
GZIP='gzip'   
BZIP='bzip2'  
XZ='xz'       

 TESTFILE='/var/tmp/test.dump'

 for program in "$LZOP" "$GZIP" "$BZIP" "$XZ" ; do
  case $program in
    gz*) ext='.gz' ;;
    bz*) ext='.bz2';;
    xz*) ext='.xz';;
    lz*) ext='.lzo';;
    *) echo 'error!  No configured compressor extension'
       exit
       ;;
  esac

   COMPRESSEDFILE="$TESTFILE$ext"

   for lvl in `seq 1 9` ; do
    c_time=`/usr/bin/time -f '%E' 2>&1 $program -$lvl $TESTFILE`
    c_size=`ls -l $COMPRESSEDFILE |awk '{print $5}'`
    d_time=`/usr/bin/time -f '%E' 2>&1 $program -d
$COMPRESSEDFILE`
    printf '%-10s %10s %10s %10s\n' "$program -$lvl" $c_time
$c_size $d_time
  done
done

As you can see, I'm not flushing caches between runs or anything fancy to make this a truly rigorous test. I'm also running this on my desktop (although I wasn't actively doing anything on that machine, it was logged into a normal X session with all the wakeups and polling and etc that that implies.) I also only used a single input file for data. Binary files or tarballs with a mixture of text and images and executables could certainly give different results. Grab the script and try this out on your own sample data. And if you get radically different results, post them!

Compressor   Compress     Size   Decompress
----------   --------   -------  ----------
none [*]_     0:00.43   67348587    0:00.00

 lzop -U -1    0:00.57   16293912    0:00.35
lzop -U -2    0:00.62   16292914    0:00.40
lzop -U -3    0:00.62   16292914    0:00.34
lzop -U -4    0:00.57   16292914    0:00.42
lzop -U -5    0:00.57   16292914    0:00.42
lzop -U -6    0:00.67   16292914    0:00.41
lzop -U -7    0:13.53   12824930    0:00.30
lzop -U -8    0:39.71   12671642    0:00.32
lzop -U -9    0:41.92   12669217    0:00.28

 gzip -1       0:01.96   11743900    0:01.02
gzip -2       0:02.04   11397943    0:00.92
gzip -3       0:02.77   11054616    0:00.89
gzip -4       0:02.59   10480013    0:00.82
gzip -5       0:03.42   10157139    0:00.78
gzip -6       0:05.44    9972864    0:00.77
gzip -7       0:06.71    9703170    0:00.76
gzip -8       0:13.64    9592825    0:00.91
gzip -9       0:15.89    9588291    0:00.76

 bzip2 -1      0:20.17    7695217    0:04.73
bzip2 -2      0:21.68    7687633    0:03.69
bzip2 -3      0:23.48    7709616    0:03.63
bzip2 -4      0:26.00    7710857    0:03.69
bzip2 -5      0:25.45    7715717    0:04.09
bzip2 -6      0:26.95    7716582    0:03.95
bzip2 -7      0:28.13    7733192    0:04.23
bzip2 -8      0:29.71    7756200    0:04.36
bzip2 -9      0:31.39    7809732    0:04.50
[@]_

 xz -1         0:08.21    7245616    0:01.86
xz -2         0:10.75    7195168    0:02.23
xz -3         0:59.45    5767852    0:01.90
xz -4         1:01.75    5739644    0:01.83
xz -5         1:09.70    5705752    0:02.60
xz -6         1:46.23    5443748    0:02.09
xz -7         1:50.37    5431004    0:02.19
xz -8         2:02.41    5417436    0:02.19
xz -9 [#]_    2:18.12    5421508    0:02.55

 .. _[*]: Time to copy the file.
.. _[@]: What's up with bzip2?  Why does the size increase
         with higher levels?
.. _[#]: Note, xz -9 is unfair on two counts:
         1) it pushed me into swap.          
         2) As for the size, xz had this output during that
            run::
              Adjusted LZMA2 dictionary size from 64 MiB
              to 35 MiB to not exceed the memory usage
              limit of 397 MiB 

 

My conclusions based upon entirely too little data :-)

• If you want transparent compression, use lzop at one of the lower compression settings. I got 25% of the size at 100 MB/s with lzop -2.
• Do not use lzop with -7 or higher. If you want more compression than -2/3/4/5/6 (the algorithm for these is currently all the same) use gzip. You'll get better compression with better speed.
• The only reason to use bzip2 is if you must have both a smaller size than gzip and you can't deploy xz there. If you don't need the smaller size or the remote side can get xz then bzip2 is a waste. This applies to distributing source code tarballs as two formats, for instance. If you're going to release in two formats, use tar.gz and tar.xz instead of tar.gz and tar.bz2.
• xz gets the smallest size but it's versatile in other ways too: xz -2 is faster than gzip -9 with better compression ratios.
• gzip beats xz at decompression but not nearly as badly it beat bzip2.