Name: dangermouse zenovka
Member since: 2003-08-14 09:12:50
Last Login: 2010-01-15 21:07:17
Notes: I have an ohloh profile. My ham radio call is HB9TVM.
I got a master in CS with focus on computational sciences, astronomy and cryptography from ETH Zuerich. I do lot of things in parallel, most of them fail (99.9%), but I do not worry :-).
My first toaster was a C64, I learned BASIC by copying programs from magazines around 1990. Then, I wrote some Turbo Pascal programs on DOS and Windows 3.1 and 95. In 2002, I had a small 80486 running with Mandrake Linux in my bathroom, but then I took a shower, the keyboard broke and killed the server with a flood of nonsense :-(... My laptop is called <andromeda> (the only deep sky object one can see with a binocular in a dark clear sky) and runs both Gentoo Linux and Windows XP. My server was mainly built by Ninja and runs Gentoo as well. I am happy owner of a LAKS clock with damn small linux distro on its 64 MB memory.
My favourite programming languages are Delphi, C, C++, PHP and Java.
The three Open Source projects I am involved with:
This is my collection of Google gadgets: Meteo Valposchiavo Il Bernina GPU deltasql HB9TVM Boy scout group Webserver Gate Coffee Break Planner
Install Ajaxterm on Gentoo
These are the steps I followed, they should complement the explanation of the wiki of Ajaxterm, created by Antony Lesuisse.
1. setup a Dyndns account and ddclient
Publish the IP address of your router first with these steps:
Visit Dyndns, register an
account and your host. Choose a freely available DNS name
like In Launch and register ddclient with
2. install Ajaxterm
If launching ajaxterm fails with [!!], do the following:
At the bottom of the file If launching ajaxterm still fails with [!!], in
Once ajaxterm starts with a green [OK], point a Javascript
enabled browser (the text based links or lynx do not work!)
to It's time to register ajaxterm to the default runlevel with
3. Emerge and configure Apache
In the wiki of
Ajaxterm, it is suggested to use Apache as SSL tunnel to
Ajaxterm, this for security reasons.
Let's build an apache with some modules ;-)
USE="ssl" emerge -av apache
Make sure that Configure the SSL tunnel with
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /ajaxterm/ http://localhost:8022/
ProxyPassReverse /ajaxterm/ http://localhost:8022/
</VirtualHost>
[...]
Start and register Apache2 with
4. create a certificate for Apache
Here the steps to create a self-signed certificate are
described:
Remove the passphrase with
Convert the request into the signed certificate:
Indicate to apache where to find the signed certificate and
the key:
Source for these steps: Apache SSL
docs
5. configure NAT on your router
Login to your router, go to the NAT section and forward the
port 443 to the server where Ajaxterm is running...
That's all, from a caged environemnt, simply point your
browser to a Have fun!delta.dyndns.org.
On your Gentoo box, install ddclient
USE="ssl" emerge ddclient
su
vi /etc/ddclient/ddclient.conf
ddclient.conf enter your account details for
Dyndns.
/etc/init.d/ddclient start
rc-update add ddclient default
ACCEPT_KEYWORDS="~x86" emerge ajaxterm
/etc/init.d/ajaxterm start
vi /etc/services
services add
ajaxterm 8022/tcp
ajaxterm 8022/udp
/etc/init.d/ajaxterm, change
the DAEMON variable to
/usr/bin/python2.5.
http://localhost:8022/. If your Gentoo box
does not have Gnome or KDE on it, you can also point a
browser on the same network segment, e.g. to
http://192.168.1.33:8022/.
Ajaxterm will welcome you with the login window :-).
rc-update add ajaxterm default
so that it survives reboot...
su
echo 'APACHE2_MODULES="authz_host actions alias auth_basic
auth_digest authn_anon authn_dbd authn_dbm authn_default
authn_file authz_dbm authz_default authz_groupfile
authz_host authz_owner authz_user autoindex cache dav dav_fs
dav_lock dbd deflate dir disk_cache env expires ext_filter
file_cache filter headers ident imagemap include info
log_config logio mem_cache mime mime_magic negotiation proxy
proxy_ajp proxy_balancer proxy_connect proxy_http rewrite
setenvif speling status unique_id userdir usertrack
vhost_alias"' >> /etc/make.conf
D_PROXY and
D_PROXY_HTTP are defined in the
apache2 file:
vi /etc/conf.d/apache2
[...]
APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D
SSL_DEFAULT_VHOST -D PROXY -D P
ROXY_HTTP"
[...]
vi /etc/apache2/vhosts.d/00_default_ssl_vhost.conf
[...]
VirtualHost *:443>
ServerName localhost
SSLEngine On
/etc/init.d/apache2 start
rc-update add apache2 default
openssl req -new > new.cert.csr
When asked for the CommonName, give the DNS
name you chose (e.g. delta.dyndns.org). This is important to
make the self-signed certificate valid.
openssl rsa -in privkey.pem -out new.cert.key
openssl x509 -in new.cert.csr -out new.cert.cert -req
-signkey new.cert.key -days 3650
vi /etc/apache2/vhosts.d/00_default_ssl_vhost.conf
[...]
SSLCertificateFile /path/to/certs/new.cert.cert
SSLCertificateKeyFile /path/to/certs/new.cert.key
[...]
https://delta.dyndns.org/ajaxterm/
and add a certificate exception, as the certificate was self
signed and not issued by a CA authority.
Cescozen
Cescozen is a game invented by Cesco. If you point your browser to the initial page of the game, you'll encounter very difficult enigmas that wait for a solution (note: the game is in Italian). To answer each quiz, you put the solution on the URL, so that you move one directory down the tree. To me and Virgi, it took about a week investing 2-3 hours each day to get through all levels. Sometimes, the quiz requires that you search something in Google, or you have a satellite image and you need to figure out where it is. Very impressive, amazing game!
People who like quiz like also to follow howto instructions to get
The full Map Stack up and running
I am fascinated by the Open Source technology behind OpenStreetMap, which I believe will soon be succesful as Wikipedia is.
After importing the dump of Switzerland in the old Gentoo server, I showed the result to my father and he gave me another piece of iron to reach the next level of this particular game: it is an IBM with 3.2 GHZ and 1 GB of RAM. I installed Ubuntu Jaunty (9.04) on the server and repeated the PostgreSQL and mapnik install following these howto instructions made by Johnny Dobbins. This tutorial additionally explains how to install the WMS server called ogcserver and how to put a cache in front of it, so that already computed tiles do not need to be recomputed but can just be served directly from disk (or from memcache or even from Amazon S3).
Just some remarks on how the job was done:
Choosing a bounding box
the server has 74 GB of disk, so based on this specs I decided to import Europe and part of Asia till longitude of Moskau and not the whole planet. For the whole planet you probably need at least 200 GB of disk.
To be precise I chose this rectangle with coordinates taken from an old fashioned Atlas: Top left corner: 75 N, 35 W Bottom right corner: 20N 60 E
This command extracted from the whole planet file the bounding box and created a new dump called eurasia.osm.bz2. The decompression and compression is done with bzcat and bzip2 piped together.
bzcat ../../planet-latest.osm.bz2 | ./bin/osmosis --read-xml enableDateParsing=no file=/dev/stdin --bounding-box left=-35 bottom=20 right=60 top=75 --write-xml file=- | bzip2 > ../../eurasia.osm.bz2
The execution lasted for about 14 hours. Out of the 7.4 GB
of planet-latest.osm.bz2, about 2.2 GB of data where
extracted for eurasia.osm.bz2. After the import with
osm2pgsql in Postgres database, the bounding
box takes about 40 GB of disk space.
Minor remarks on the howto of Johnny
I did not find the lines in cgiserver.py, so I
left this step out.
For the file wms.py: you need to completely
replace the existing file with the lines shown in the howto.
Setup of ogcserver
When browsing to http://localhost:8080, I got an error.
Looking at /var/log/apache2/error.log,
I noticed this error:
IOError: sys.stdin access restricted by mod_wsgi
I then added at the bottom of
/etc/apache2/sites-available/default outside
the VirtualHost section the following two lines:
WSGIRestrictStdout Off WSGIRestrictStdin Off
Tilecache: Seeding the cache
After setting everything up, pieces of the map started appearing. However, often the server got overloaded, if it had to compute tiles which where full of features (streets and rivers). Especially the tile with Germany never appeared as it was too feature rich. Eventually, the server hang up when even the swap cache got full.
The solution to this performance problem was to seed the cache. I chose a point in the center of Europe (46 N, 10 E) and a radius of 30 degrees.
I then issued in the directory
ogcserver/tilecache-2.10/ the following command:
./tilecache_seed.py basic 46,10,30 [press Control D on the keyboard]
and I let compute map tiles for about 48 hours.
The result
Here it is :-). Happy browsing, press twice the plus button first, to reach the correct zoom level. The speed in serving tiles might vary, as I have an ADSL and use dynamic DNS...
11 Dec 2009 (updated 11 Dec 2009 at 13:29 UTC) »
Your own Openstreet map tile server on Gentoo
I wrote down some steps that differ from the excellent tutorials here and here, because my installation is performed on Gentoo instead of Ubuntu: of course, this is working now as for December 2009, but an emerge --sync in the future might solve some of these problematic steps (maybe someone more knowledgeable than me will read this tutorial and fix some ebuilds :-).
Step: Subversion and Java
These one were easy and you probably figured out faster than me:
emerge -av subversion
emerge -av sun-jdk
Step: Installing Postgresql with Postigs spatial extensions
osm2pgsql does not work with Postgresql 8.1 which is marked stable in Gentoo. The reason is that Postgresql 8.1.11 does not know about the command 'DROP TABLE IF EXISTS' which is used in oms2pgsql. Defining a stored procedure that does the same and modifying the source code of osm2pgsql did not work for me.
Therefore I chose another way, which goes as follows: you
should add at the end of /root/.bashrc the following alias:
alias aemerge='ACCEPT_KEYWORDS="~x86" emerge '
source /root/.bashrc
will load this setting for the first time. The
setting is
kept after, as .bashrc is read at every login.
From now on, you can use "aemerge" to emerge unstable
packages instead of "emerge".
Then, you can install an unstable Postgresql with
aemerge -av postgresql
This will install a Postgresql 8.2.14 which is fine for our purposes.
Install also Postgis:
aemerge -av postgis
Postgis-1.4.0 is then installed.
Configure the database postgis extensions not by using emerge --config as suggested at the end of emerge, but use the instructions provided in this tutorial. Remember to follow the instructions as normal user and not as superuser (a part of the inital step of setting up the database).
As noted in the previous comment, lwpostgresql.sql does not
exist in postgis anymore, it was renamed in postgis.sql, you
should therefore use
psql -d gis -f /usr/share/postgresql/contrib/postgis.sql
Step: Installing mapnik
For mapnik I had problems, it did not compile from source as
described in the tutorial, it was failing when compiling
the postgis plugin for linking reasons against the library
libpq.la, though the library was present on the proposed path.
I did the following trick:
USE="postgresql" aemerge -av mapnik
emerged and installed a mapnik with all dependencies, which however was not able to recognize the postgis extensions on the 'gis' database.
In fact, when running ./z0_generate_tiles.py, mapnik was
failing with:
Could not create datasource. No plugin found for type
'postgis' (encountered during parsing of layer 'leisure')
when connecting to the imported database.
Then I did the following:
I unmerged the unstable libpq library (libpq is a library
used by postgresql used as connection layer.
emerge --unmerge libpq
I unmerged mapnik without its dependencies with
emerge --unmerge mapnik
I installed the stable libpq library with:
emerge -av libpq
Then I installed mapnik from source with the following steps
mkdir src
cd src
svn co http://svn.mapnik.org/trunk/ mapnik
cd mapnik
scons/scons.py -c install
scons/scons.py PGSQL_LIBS=/usr/lib/postgresql/
PGSQL_INCLUDES=/usr/include/postgresql/
sudo scons/scons.py install PGSQL_LIBS=/usr/lib/postgresql/
PGSQL_INCLUDES=/usr/include/postgresql/
ldconfig
Therefore: postgresql is compiled against an unstable
version of libpq, while mapnik needs a stable one to compile.
Of course, postgresql will survive the restart with
/etc/init.d/postgresql restart
:-)
Have fun with your maps! Import the planet on your favourite Gentoo box!
Coffee Break Planner
To increase the number of visitors to our webserver, I finally found a killer application. I developed it as a Google Gadget and is called the Coffee Break Planner. This gadget helps you to meet with your friends at the right time for a tasteful coffee... It also increases the overall performance of your company, as in coffee breaks most of the time people speak about work anyway :-)
Again, the gadget is only an xml envelope, while PHP and mySQL do the hard work.
I created this gadget to upload files to my webserver as ftp is firewalled in the company I am working for. I prepared it so that other people can install on their webserver as well, if they read the Readme on the gadget :-)
To work on the webserver I use this service, which costed me 15 euro for one year. It is a Unix shell via HTTPS that can be opened with a normal browser. Tunneling with httptunnel or corkscrew or any other trick did not work for me... Too stupid to break out without commercial support, ehm!
Joomla
Since beginning of the year I manage the local boy scout homepage with Joomla and I am very happy with this solution, last but not least released as Open Source.
Android emulator v2.0
I am experimenting with Eclipse, Android v2.0 and a tutorial a friend did... But I still do not have meaningful results. Some Java exception crashes the small application and the emulator does not work behind the proxy, even if the -http-proxy option is set in the launch parameters. Notice that the mysql trick works only with Android 1.6. Probably the proxy filters also on the browser identification string... and the emulator does not mimic a Mozilla or IE as identification... And the proxy is difficult to overcome...
dangermaus certified others as follows:
Others have certified dangermaus as follows:
[ Certification disabled because you're not logged in. ]
FOAF updates: Trust rankings are now exported, making the data available to other users and websites. An external FOAF URI has been added, allowing users to link to an additional FOAF file.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!