jas is currently certified at Master level.

Name: Simon Josefsson
Member since: 2001-08-12 00:08:06
Last Login: 2012-09-17 06:53:14

FOAF RDF Share This

Homepage: http://josefsson.org/


Recent blog entries by jas

Syndication: RSS 2.0

Replicant 4.2 on Samsung S3

Since November 2013 I have been using Replicant on my Samsung S3 as an alternative OS. The experience has been good for everyday use. The limits (due to non-free software components) compared to a “normal” S3 (running vendor ROM or CyanogenMod) is lack of GPS/wifi/bluetooth/NFC/frontcamera functionality — although it is easy to get some of that working again, including GPS, which is nice for my geocaching hobby. The Replicant software is stable for being an Android platform; better than my Nexus 7 (2nd generation) tablet which I got around the same time that runs an unmodified version of Android. The S3 has crashed around ten times in these four months. I’ve lost track of the number of N7 crashes, especially after the upgrade to Android 4.4. I use the N7 significantly less than the S3, reinforcing my impression that Replicant is a stable Android. I have not had any other problem that I couldn’t explain, and have rarely had to reboot the device.

The Replicant project recently released version 4.2 and while I don’t expect the release to resolve any problem for me, I decided it was time to upgrade and learn something new. This time I decided to use the pre-built ROM images rather to build my own because I had issues building replicant 4.2 on my Debian wheezy machine (C++ compilation errors which apparently does not happen if you use a newer compiler).

Before the installation, I wanted to have a full backup of the phone to avoid losing data. I use SMS Backup+ to keep a backup of my call log, SMS and MMS on my own IMAP server. I use oandbackup to take a backup of all software and settings on the phone. I use DAVDroid for my contacts and calendar (using a Radicale server), and reluctantly still use aCal in order to access my Google Calendar (because Google does not implement RFC 5397 properly so it doesn’t work with DAVDroid). Alas all that software is not sufficient for backup purposes, for example photos are still not copied elsewhere. In order to have a complete backup of the phone, I connect the phone using a USB cable and enable USB tethering on the phone. The network is automatically set up on my Debian machine, I did not have to do anything. Then I invoke the following commands to take a backup using rsync:

adb shell dropbear -F -E &
sudo rsync -av --delete --exclude /dev --exclude /acct --exclude /sys --exclude /proc -e ssh root@ /root/s3-bup/

Now feeling safe that I would not lose any data, I removed the SIM card from my phone (to avoid having calls, SMS or cell data interrupt during the installation) and followed the Replicant Samsung S3 installation documentation. Installation was straightforward. I booted up the newly installed ROM and familiarized myself with it. My first reaction was that the graphics felt a bit slower compared to Replicant 4.0, but it is hard to tell for certain.

I connected to the network using USB tethering and took a quick rsync-backup of the freshly installed phone, to have a starting point for future backups. Since my IMAP and CardDav/CalDav servers use certificates signed by CACert I first had to install the CACert trust anchors, to get SMS Backup+ and DAVDroid to connect. For some reason it was not sufficient to add only the root CACert certificate, so I had to add the Class 3 sub-ca cert as well. To load the certs, I invoke the following commands, selecting ‘Install from SD Card’ when the menu is invoked (twice).

adb push root.crt /sdcard/
adb shell am start -n "com.android.settings/.Settings\"\$\"SecuritySettingsActivity"
adb push class3.crt /sdcard/
adb shell am start -n "com.android.settings/.Settings\"\$\"SecuritySettingsActivity"

I restore apps with oandbackup, and I select a set of important apps that I want restored with settings preserved, including aCal, K9, Xabber, c:geo, OsmAnd~, NewsBlur, Google Authenticator. I install SMS Backup+ from FDroid separately and configure it, SMS Backup+ doesn’t seem to want to restore anything if the app was restored with settings using oandbackup. I install and configure the DAVdroid account with the server URL, and watch it populate my address book and calendar with information.

After organizing the icons on the launcher screen, and changing the wallpaper, I’m up and running with Replicant 4.2. This upgrade effort took me around two days to complete, with around half of the time consumed by exploring different ways to do the rsync backup before I settled on dropbear and USB tethering. Compared to the last time, when I spent almost two weeks researching various options and preparing for the install, this felt like a swift process.

I spent some time researching how to get the various non-free components running. This is of course sub-optimal, and the Replicant project does not endorse non-free software. Alas there aren’t any devices out there that meets my requirements and use only free software. Personally, I feel using a free core OS like Replicant and then adding some non-free components back is a better approach than using CyanogenMod directly, or (horror) the stock ROM. Even better is of course to not add these components back, but you have to decide for yourselves which trade-offs you want to make. The Replicant wiki has a somewhat outdated page on Samsung S3 firmware. Below are my notes for each component, which applies to Replicant 4.2 0001. You need to first prepare your device a bit using these commands, and it is a good idea to reboot the device after installing the files.

adb root
adb shell mount -o rw,remount /system
adb shell mkdir /system/vendor/firmware
adb shell chmod 755 /system/vendor/firmware

GPS: The required files are the same as for Replicant 4.0, and using the files from CyanogenMod 10.1.3 works fine. The following commands load them onto the device. Note that this will load code that will execute on your main CPU which is particularly bothersome. There seems to exist a number of different versions of these files, CyanogenMod have the same gpsd and gps.exynos4.so in version 10.1.3 and 10.2 but the libsecril-client.so differs between 10.1.3 and 10.2. All files differ from the files I got with my stock Samsung ROM on this device (MD5 checksums in my previous blog). I have not investigated how these versions differs or which of them should be recommended. I use the files from CyanogenMod 10.1.3 because it matches the Android version and because the files are easily available.

adb push cm-10.1.3-i9300/system/bin/gpsd /system/bin/gpsd
adb shell chmod 755 /system/bin/gpsd
adb push cm-10.1.3-i9300/system/lib/hw/gps.exynos4.so /system/lib/hw/gps.exynos4.so
adb push cm-10.1.3-i9300/system/lib/libsecril-client.so /system/lib/libsecril-client.so
adb shell chmod 644 /system/lib/hw/gps.exynos4.so /system/lib/libsecril-client.so

Bluetooth: Only one file has to be installed, apparently firmware loaded onto the Bluetooth chip. Cyanogenmod 10.1.3 and 10.2 contains identical files, which has a string in it “BCM4334B0 37.4MHz Class1.5 Samsung D2″. The file I got with my stock ROM has a string in it “BCM4334B0 37.4MHz Class1.5 Samsung M0″. I don’t know the difference, although I have seen that D2 sometimes refers to the US version of a Samsung device. My device is the international version, but it seems to work anyway.

adb push cm-10.1.3-i9300/system/bin/bcm4334.hcd /system/vendor/firmware/bcm4334.hcd
adb shell chmod 644 /system/vendor/firmware/bcm4334.hcd

Front Camera: Two files has to be installed, apparently firmware loaded onto the Camera chip. CyanogenMod 10.1.3 and 10.2 contains identical files, which has a string in it “[E4412 520-2012/08/30 17:35:56]OABH30″. The file I got with my stock ROM has a string in it “[E4412 533-2012/10/06 14:38:46]OABJ06″. I don’t know the difference.

adb push cm-10.1.3-i9300/system/vendor/firmware/fimc_is_fw.bin /system/vendor/firmware/fimc_is_fw.bin
adb push cm-10.1.3-i9300/system/vendor/firmware/setfile.bin /system/vendor/firmware/setfile.bin
adb shell chmod 644 /system/vendor/firmware/fimc_is_fw.bin /system/vendor/firmware/setfile.bin

NFC: I’m happy that I got NFC to work, this was one of my main issues with Replicant 4.0 (see my earlier blog post). Only one file is needed, however CyanogenMod does not seem to distribute it so you have to get it from your stock ROM or elsewhere. The md5 of the file I have is b9364ba59de1947d4588f588229bae20 (and no I will not send it to you). I have tested it with the YubiKey NEO and the Yubico Authenticator app.

adb push clockworkmod/blobs/ee6/7188ca465cf01dd355a92685a42361e113f886ef44e96d371fdaebf57acae /system/vendor/firmware/libpn544_fw.so
adb shell chmod 644 /system/vendor/firmware/libpn544_fw.so

Wifi: I haven’t gotten wifi to work, although I have not tried very hard. Loading the CyanogenMod firmwares makes my device find wireless networks, but when I try to authenticate (WPA-PSK2), I get failures. Possibly some other files has to be loaded as well.

flattr this!

Syndicated 2014-02-27 11:07:45 from Simon Josefsson's blog

Necrotizing Fasciitis

Dear World,

On the morning of the 24th December I felt a unusual pain in my left hand between the thumb and forefinger. The pain increased and I got high fever (at some point above 40 degrees Celsius or 104 degree Fahrenheit), and was hospitalized during the night between the 24th and 25th of December. On the afternoon of 26th of December I underwent surgery and was after that diagnosed with Necrotizing Fasciitis (the wikipedia article on NF gives a good summary), caused by the common streptococcus bacteria (again see wikipedia article on Streptococcus). A popular name for the disease is flesh-eating bacteria. Necrotizing Fasciitis is a rare and aggresive infection, deadly if left untreated, that can move through the body at speeds of a couple of centimeters per hour. Fortunately I was healthy at the time when this started, and with bi-weekly training sessions for the last 1.5 year I was physically at my strongest peak in my 38 year old life (weighting 78kg or 170lb, height 182cm or 6 inches). I started working out to improve back issues, improve strength, and prepare for getting older; exercise has never been my thing although I think it is fun to run medium distances (up to 10km).

I have gone through 6 surgeries, and have experienced extreme angst and pain — even with help from Opiat-like pain-killers (more specifically OxyContin and OxyNorm) and the pain-dissociative drug Ketamine (more specifically Ketalar). I am very grateful to be alive. I find joy in even the simplest of things, like being able to drink water or seeing trees outside the window. I have learned many things about medicine and my body, and I am curious by nature so I look forward to learn more. I hope to be able to draw strength from this incident, to help me prioritize better in my life. My loving wife Åsa has gone through a nightmare as a consequence of my diagnosis. At day she had to cope with daily life taking care of our wonderful little 1-year old daughter Ingrid and our 3-year old boy Alfred. All three of them had various degrees of strep throat with fever, caused by the same bacteria — and anyone with young kids know how intense that alone can be. She gave me strength over the phone. She kept friends and relatives up to date about what happened, with the phone ringing all the time. She worked to get information out from the hospital about my status, sometimes being rudely treated and just being hanged up on. Since I had only became worse and worse throughout the third operation, and the infection was still spreading, she at one time after a call with the doctor started to plan for a life without me.

My last operation were on Thursday January 2nd and I left hospital the same day. I’m writing this on the Saturday of January 4rd. I have regained access to my arm and hand and doing rehab to regain muscle control, while my body is healing. I’m doing relaxation exercises to control pain and relax muscles, so that I’m now rid of the strong drugs. I take antibiotics (more precisely Dalacin) and the common Paracetamol pain-killer together with on-demand use of an also common NSAID containing Ibuprofen. My wife and I were even out at a restaurant tonight.

Naturally, I now want to focus on getting well and spend time with my family. I’m not out of dangerous waters yet. Don’t expect anything from me in the communities and organization that I’m active in (i.e., GNU, Debian, IETF, Yubico), I will come back as energy, time and priorities permits.

flattr this!

Syndicated 2014-01-04 22:44:49 from Simon Josefsson's blog

Using Replicant on Samsung Galaxy S III

For the last half-year I have used CyanogenMod on an Nexus 4 as my main phone. Recently the touch functionality stopped working on parts of the display, and the glass on the back has started to crack. It seems modern phones are not built to last. For comparison, before the N4 I used a Nokia N900 for around 3 years without any hardware damages (in my drawer now, still working). A few weeks ago I started looking for a replacement. My experience with CyanogenMod had been good, but the number of proprietary blobs on the N4 concerned me. Finding something better wasn’t easy though, so I’m documenting my experience here.

My requirements were, briefly, that I wanted a phone that I could buy locally that had a free software community around it that produced a stable environment. I have modest requirements for things I wouldn’t give up on: telephony, data connection (3G), email (IMAP+SMTP), chat (XMPP), and a web browser. I like the philosophy and openness around the Firefox OS but the more I have read about it, it seems unlikely that it would deliver what I need today. In particular none of the devices capable of running Firefox OS appealed to me, and the state of email reading seemed unclear. I’m sure I’ll revisit Firefox OS as an alternative for me in the future.

As I had been happy with CyanogenMod, but concerned about its freeness, it felt natural to move on and test the more free software friendly project Replicant. Replicant only supports a small number of devices. After talking with people in the #replicant IRC channel, it seemed the Samsung S3 would be a decent choice for me. The Samsung S2 would have worked as well, but it cost almost as much as the S3 where I looked. Despite the large number of Samsung S3 devices out there, it seems the prices even for used devices are high (around 2500 SEK in Sweden, ~380 USD). I ended up buying a brand new one for 3200 SEK (~500 USD) which felt expensive, especially after recalling the recent $199 sale for Nexus 4. Noticing that brand new Nexus 4 devices are still over 3000 SEK in Sweden comforted me a bit. I would have preferred a more robust phone, like the CAT B15, but the state of free software OSes on them seem unclear and I wanted something stable. So, enough about the background, let’s get started.

Building and installing Replicant on the device was straight forward. I followed the Replicant Samsung S3 Build instructions to build my own images. The only issue I had was that I had not set JAVA_HOME and the defaults were bad; make sure to set JAVA_HOME before building. I built everything on my Lenovo X201 running Debian Wheezy, with OpenJDK 6 as the Java implementation. Installing the newly built firmware was easy, I just followed the installation process documentation. I made sure to take a clockworkmod backup to an external SD card before wiping the old system. To get a really clean new device, I also re-formated /sdcard inside clockworkmod; I noticed there were some traces left of the old system there.

I spent about one week testing various configurations before settling on something I could use daily. A fair amount of time was spent looking into backup and restore options for Android devices. My idea was that I would take a backup of the apps I ran on the N4 and transfer them to the S3. The Android Debug Bridge (adb) has a backup/restore command, however it (intentionally) ignores apps marked as allowBackup=false which a number of apps has. It doesn’t seem possible to override that settings — so much for the freedom to backup your own device. I then discovered oandbackup. It can backup your entire system, saving each app (together with associated data) into a separate directory, for simple review and inspection before restore. You can do batch backups and batch restore. I couldn’t get it to automatically restore things, though, which would be neat for really automated re-installations (there is an open issue about this feature). After noticing that some apps did not like being moved from the N4 (running Android 4.2) to the S3 (running 4.0), I ended up installing most apps from scratch on a freshly installed Replicant. I use oandbackup to the external SD card so that I can quickly restore my phone. For backup/restore of SMS/MMS and Call Log, I use SMS Backup+ against my own IMAP server. Camera pictures are synced manually using adb when I am connected to my laptop.

There is a number of apps that deserve to be mentioned because they are what I use on a daily basis. All of them come via the free software market F-Droid. For email (IMAP/SMTP), I use K-9 Mail which is feature rich but still easy to use. For chat, I use Xabber. I use NewsBlur‘s free software app to read RSS flows. For two-factor authentication, I use Google Authenticator. I haven’t evaluated different PDF viewers, but the first one I tried (APV PDF Viewer) has worked fine so far. Handling a a synchronized address book and calendar deserve its own blog post because it is a challenging topic, but briefly, I’m currently using a combination of aCal and DAVdroid.

Finally, since Replicant is still work in progress, some words about stability and notes on what doesn’t work. This is probably the most interesting part if you are considering running Replicant on an S3 yourself. Overall system stability is flawless, I hadn’t had any crash or problem with the fundamental functionality (telephony, 3G, Camera). People have said graphics feels a bit laggy, but I cannot compare with the stock ROM and it doesn’t get in the way of daily use. First some notes about non-free aspects:

  • Bluetooth doesn’t work by default. After installing /system/vendor/firmware/bcm4334.hcd (MD5 b6207104da0ca4a0b1da66448af7ed4c) pairing and testing with a Bluetooth headset worked fine.
  • Front camera doesn’t work by default. After installing /system/vendor/firmware/fimc_is_fw.bin (MD5 52eeaf0889bc9206860075cd9b7f80b9and) and /system/vendor/firmware/setfile.bin (MD5 0e6fdeb378fb154c39fd508ae28eaf2a) it works. The extensions are *.bin but I don’t believe this code is executed on the main CPU.
  • GPS doesn’t work by default. After installing /system/bin/gpsd (MD5 6757ed2e2a283259e67c62e6b2b9cfef), /system/lib/libsecril-client.so (MD5 a836df0f947d2aa2ef20dcb213317517), /system/lib/hw/gps.exynos4.so (MD5 1ea1d67f297dd52d59d40dbad9b02a0a) it works. This is code that runs on the main CPU, and even more alarming, it embeds a copy of OpenSSL and talks to various online servers for A-GPS, and is thus a likely (and probably not too challenging) attack vector for anyone wanting to remotely exploit any phone.
  • Wifi doesn’t work, and I haven’t gotten this to work. There is a list of non-free S3 firmware on the Replicant wiki however my stock ROM did not ship with those files. I don’t believe any of the blobs run on the main CPU.
  • NFC doesn’t work, and I haven’t gotten it to work. It seems the infrastructure for NFC is missing in Replicant 4.0, it doesn’t even expose the NFC hardware permission capability. This is quite unfortunate for me, since I daily work with YubiKey NEO and would have preferred to replace Google Authenticator with the YubiOATH that uses the NEO for OATH secret storage.

Some other observations:

  • Panorama mode in the Camera crashes; see issue about it.
  • There is a number of smaller graphical issues. I believe these are related to the EGL but haven’t understood the details. What I’ve noticed are the following issues. The task switcher doesn’t show mini screenshots of all running apps (the screenshots are just black). ZXing is not able to QR decode images (I’m told this is because Replicant uses a RGB color plane instead of the required YUV color plane). Video playback in the gallery is laggy to the point of being unusable. Video playback on Youtube in the default web browser works in full screen (not laggy), but not embedded in the webpage.
  • MTP has been a bit unreliable, my main laptop is able to import photos, but another laptop (also running Debian Wheezy) just stalls when talking to it. This may be a host issue, I have experience similar issues with a Nexus 4 2nd generation device.

I am quite happy with the setup so far, and I will continue to use it as my primary phone.

flattr this!

Syndicated 2013-11-11 21:10:29 from Simon Josefsson's blog

BLURB: Software repository metadata convention

As a maintainer of several software packages I often find myself copying text snippets from the README file into different places (savannah, github, freecode, emails, etc). Recently I had a need to generate a list of software packages that included every project’s name, brief summary, license and URL. I could have generated that list manually by copying the text from every project’s README and COPYING files. However then I would have to maintain my list manually to keep it in sync with all the projects. This easily leads to stale information, so I thought a better approach would be to put the information I needed into each projects’ source code version control system. The advantage is that the manual work to extract the information may be automated by a script, since the data is in a usable format. I’ll explain here how my solution works.

To be able to find the information using only the URL to the repository, I needed a filename convention. The filename I chose was BLURB; for the etymology, see Wikipedia’s page about Blurb. The data format in the file is similar to normal email headers.

An example illustrate the principles well. Below is the BLURB file for one of my projects.

Author: Yubico
Basename: libyubikey
Homepage: http://opensource.yubico.com/yubico-c/
License: BSD-2-Clause
Name: Yubico C low-level Library
Project: yubico-c
Summary: C library for manipulating Yubico YubiKey One-Time Passwords (OTPs)

The format is simple: UTF-8 text with each line starting with a header followed by a colon (“:”), some whitespace, and some content. If a line starts with whitespace, it is a continuation of the previous line’s content (trim leading whitespace). The following table describes the fields that I use. I may update this blog post in the future with new fields or improved explanations (for reference, current date is 2013-09-24).

Header Meaning
Project Short identifier for the project (e.g., ‘gcc’, ‘emacs’)
Name Official name of the project in English
Summary Brief one-line summary of the project’s purpose in English
Author Origin of the project
Homepage URL to the project’s website
License License keyword, preferrably using one of the SPDX license identifiers
Basename The tarball basename, if different from the project name

Finally some reflection of the solution. After quick design, I thought that I couldn’t be the first one with this problem, and I tried to find other similar efforts. I haven’t been able to find any standardization effort that have the following properties:

  • Stores the information inside each upstream project’s own source code repository
  • Provides a filename convention so that it is possible to find the data with only the source code repository link
  • Encode data in a format that is easy to extract using simple command line tools
  • Not encode information about releases (i.e., what happened in a particular version)

The related efforts that I found were SPDX which at first look seemed to offer what I wanted. However on closer examination it failed to deliver on all the requirements above, and appeared to have somewhat different goals. However I found the SPDX license list useful and refer to it. Another effort is Eric S. Raymond’s freecode-submit and shipper but its primary focus is to encode information about each release. The design of the BLURB file is clearly influenced by these tools. Another influence has been Debian’s specification for machine-readable copyright information. The Free Software Foundation’s list of software projects seemed like another candidate, but it doesn’t suggest any way to store the information in the upstream project itself.

flattr this!

Syndicated 2013-09-24 21:03:45 from Simon Josefsson's blog

Portable Symmetric Key Container (PSKC) Library

For the past weeks I have been working on implementing RFC 6030, also known as Portable Symmetric Key Container (PSKC). So what is PSKC? The Portable Symmetric Key Container (PSKC) format is used to transport and provision symmetric keys to cryptographic devices or software.

My PSKC Library allows you to parse, validate and generate PSKC data. The PSKC Library is written in C, uses LibXML, and is licensed under LGPLv2+. In practice, PSKC is most commonly used to transport secret keys for OATH HOTP/TOTP devices (and other OTP devices) between the personalization machine and the OTP validation server. Yesterday I released version 2.0.0 of OATH Toolkit with the new PSKC Library. See my earlier introduction to OATH Toolkit for background. OATH Toolkit is packaged for Debian/Ubuntu and I hope to refresh the package to include libpskc/pskctool soon.

To get a feeling for the PSKC data format, consider the most minimal valid PSKC data:

<?xml version="1.0"?>
<KeyContainer xmlns="urn:ietf:params:xml:ns:keyprov:pskc" Version="1.0">

The library can easily be used to export PSKC data into a comma-separated value (CSV) format, in fact the PSKC library tutorial concludes with that as an example. There is complete API documentation for the library. The command line tool is more useful for end-users and allows you to parse and inspect PSKC data. Below is an illustration of how you would use it to parse some PSKC data, first we show the content of a file “pskc-figure2.xml”:

<?xml version="1.0" encoding="UTF-8"?>
<KeyContainer Version="1.0"
    <Key Id="12345678"

Here is how you would parse and pretty print that PSKC data:

jas@latte:~$ pskctool -c pskc-figure2.xml 
Portable Symmetric Key Container (PSKC):
	Version: 1.0
	Id: exampleID1
	KeyPackage 0:
			Id: 12345678
			Issuer: Issuer-A
			Algorithm: urn:ietf:params:xml:ns:keyprov:pskc:hotp
			Key Secret (base64): MTIzNA==


For more information, see the OATH Toolkit website and the PSKC Library Manual.

flattr this!

Syndicated 2012-10-11 20:32:10 from Simon Josefsson's blog

32 older entries...


jas certified others as follows:

  • jas certified kaig as Journeyer
  • jas certified jwz as Master
  • jas certified rms as Master
  • jas certified walters as Journeyer
  • jas certified abraham as Journeyer

Others have certified jas as follows:

  • mathieu certified jas as Apprentice
  • walters certified jas as Journeyer
  • sdodji certified jas as Journeyer
  • abraham certified jas as Journeyer
  • vab certified jas as Journeyer
  • bagder certified jas as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page