h-online article covering OpenBTS and OpenBSC
You can find a 3-page article about OpenBTS, OpenBSC and related projects available from the h-online web site.
h-online article covering OpenBTS and OpenBSC
You can find a 3-page article about OpenBTS, OpenBSC and related projects available from the h-online web site.
Using a cheap (USD 20) DVB-T USB stick as SDR receiver for (not only) gnuradio
Fellow Osmocom hacker Steve Markgraf has been working on what now seems to be the cheapest way to receive real-world radio signals for PC-based SDR like gnuradio: rtl-sdr. RTL refers to the RTL2832U chipset frequently used in such devices. It can be used to obtain 2.8 Ms/s of 8-bit I+Q samples.
Below is a picture (courtesy of Steve) how the hardware looks like:
Osmocom GPS timing source with u-blox LEA6-T
Recently we have been looking for an inexpensive way to generate a high-accuracy clock source for E1 lines, as it is required by a number of classic BTSs that don't have a sufficiently accurate OCXO built-in.
Luckily, the Digium E1 cards like TE-410P have a timing connector, to which an 8.192 MHz signal can be injected. Unfortunately, there don't seem to be any OCXOs around for that frequency. That's where the u-blox LEA-6T comes into play: It has a configurable TIMEPULSE2 output that can generate any frequency up to 10 MHz. We use this in our board to generate 8.192 Mhz and want to feed that into the Digium card.
So all we had to do is build a small board that contains the module and connector for antenna input, clock output and the obligatory 2.5mm stereo jack for the OsmocomBB-style UART:
Thanks to Sylvain for doing the schematics/PCB design, and thanks to Pablo for writing the code to configurea and activate the 8.192MHz output.
Once the design is verified, the schematics + gerber will be available, as well as board from the sysmocom webshop.
Alcatel MTK phone UART pinout
The Alcatel OT-890D is a MT6573 based smartphone. It seems one of the UARTs is available on test pads as seen in this picture:
The voltage level is still 3.3V, so no fancy 1.8V gear is required.
During boot, the UART is first used at 19200 bps, where it prints the strings "MW01" and "MW02". I then switches to 115200 bps where it prints "READY", and finally switches to 921600 bps, where it seems to output some mixed binary/text messages containing AT commands and responses between AP and BP, as well as some debug information:
�Ue� � � T+CREG=2 �Ue�!�!�!T+CSQSQ=1 �Ue�!�!�!AT+CREG=2 �Uew�"w�"w�"SQSQ=1 'Ue""" AT+EFUN=1 SML: Load!_Ue"""""" SML: Load!hU("("("
I haven't yet investigated if the binary between the text is some standard HDLC framing or a TS 07.10 multiplex.
If anyone knows more about the boot process (MW01/MW02/READY) or the binary protocol, please let me know.
OsmoSDR status update
It has been two months since I first was able to play with the OsmoSDR hardware prototypes. Back at that time, there was no FPGA code yet, and some hardware bugs still had to be resolved. Nonetheless, the e4k tuner driver could already be implemented and tuning was confirmed by looking at the analog i/q spectrum.
Meanwhile, the hardware has been re-worked by SR-Systems and FPGA VHDL code written by maintech.de. Ever since that, they dropped the ball again with me as I had been careless enough to volunteer for writing the firmware.
And that's what I did or at least tried to do for quite some time during the last two weeks. The main problem was that I didn't have much time. The second problem was that I never was able to get the SSC (synchronous serial controller) receive DMA working.
This was a really odd experience, as I've worked a lot with that very same SSC peripheral before, while writing firmware for the OpenPICC some 6 years ago. However, this was in an at91sam7s, where the SSC is interfaced with the PDC (Peripheral DMA Controller). In the at91sam3u of OsmoSDR, it interfaces with a more modern DMAC/HDMA controller, capable of scatter-gather DMA and other fancy stuff.
Atmel has provided reference code that uses the SSC DMA in transmit mode (for a USB audio device playing back music via the Wolfson codec on the SAM3U-EK board). After thoroughly studying the DMAC/HDMA documentation I set out to write code for DMA-based SSC receiver. And it never worked.
I actually wrote two independent implementations, one from scratch and the other based on Atmel reference code. Neither of them worked. It seemed to be a problem with the hardware hand-shaking between SSC and DMAC. The SSC was successfully receiving data, and that data could be read out from the CPU using a polling or IRQ based driver. But if you're running at something like 32 Mbps and don't have a FIFO, you desperately want to use DMA. When the DMA handshaking was turned off, the DMA code worked, but of course it read the same received word several thousand times before the next data arrived on the SSC.
In the end, I was actually convinced it must be a silicon bug. Until I thought well, maybe they just connected the flow controller to a different ID in Rx and Tx direction. Since there are only 16 such identifiers, it was relatively easy to brute-force all of them and see if it worked. And voila - using the identifier 4, it worked!
So what had happened? The Atmel-provided reference code contained a
and that was wrong. 3 is valid for SSC TX but not for SSC RX. Unfortunately I never found any of those magic numbers in the SAM3U manual either. They are not documented in the chapter of the SSC, and they are not documented in the chapter about HDMA/DMAC either. And they are not identical with the Peripheral Identifiers that are used all over the chip for the built-in peripherals.
In case anyone else is interested, a patch can be found at my at91lib git repository.
I filed a ticket with Atmel support, and they pointed out in fact there was a table with those identifiers somewhere in the early introductory chapters where you can see a brief summary of the features of each integrated peripheral. Unfortunately they use slightly different naming in that chapter and in the DMAC, so a full-text search also didn't find them. Neither is that table visible in the PDF index.
So about four man-days later it was finally working. Another day was spent on integrating it with the USB DMA for sending high-speed isochronous transfers over the bus into the PC. And ever since I'm happily receiving something like 500,000 or 1,000,000 samples / second from an alsa device, using snd-usb-audio. Luckily, unlike MacOS or Windows, the Linux audio drivers don't make arbitrary restrictions in the sample rate. According to the USB Audio spec, the sample rate can be any 24bit number. So audio devices with 16.7 Ms/s are very much within the spec. I hope some of the other OS driver writers would take that to their heart.
One of the first captures can be found at this link, containing a bzip2ed wave file in S16LE format Stereo (I/Q). It contains a FM audio signal transmitted using a small pocket-sized FM transmitter.
There is no I/Q DC offset calibration yet, but once that is done we're probably able to finally put the design into production.
The next project on the horizon: A Free Software CardOS
Now that we have a 100% free software GSM protocol stack and baseband firmware for the network and mobile phone side, the only remaining proprietary part is the SIM card. And what is a SIM card? It's a small embedded computer / SoC with integrated flash + RAM.
Once again, like in many other areas of the telecommunications industry, development of Free Software has been hampered by lack of available register-level hardware documentation. Without such information, how should you be able to program? Hardware without such documentation is an insult to every software developer.
The next problem is that typically, the Card Operating System (COS) is written into mask ROM of the smartcard SoC. Making such a mask is quite expensive, and it means that for every software version, different silicon will have to be produced. So unless you are going to have millions of units in quantity, it is unlikely that it would make economic sense.
However, in recent years, purely flash based smartcard chips have been available and getting less and less expensive. However, none of them (like the Atmel AT90SC7272 or similar devices) have freely available documentation. Furthermore, availability on the open market is somewhat of a problem, mainly because they have been used extensively by people cracking encrypted satellite TV channels. In recent years, the smartcard industry is trying hard to cut any kind of supply to that group of users.
However, luckily, we now see small/independent chip design houses in China picking up and producing their own smartcard chips. They are not only cheaper, but they simply hand out the documentation to anyone who asks them. No questions asked, no NDA required. Welcome to the promised land! That's what Free Software developers like:
As you can see, I am quite enthusiastic about this. I like this no-bullshit approach. No stupid marketing and sales droids who charge ridiculous fees for proprietary development tools that are inflexible and force developers to use one particular OS/IDE/toolchain.
I'm not sure how much time there will be, given the multitude of other projects that are all asking for attention. However, I think this is a chance that the Free Software community doesn't get every day. Let's hope some other people like bare iron programming in small embedded systems can get excited and we can create a FOSS COS. It doesn't have to be something serious. Something quite simple would be sufficient for the beginning. I'm not thinking of EAL4+ certification, multiple channels and public key crypto. SIM/USIM cards are simple, they just require a bit of filesystem read/write operations plus authentication. And luckily, SIM toolkit development doesn't have to be done in Java this way, either ;)
More research into the Motorola Horizon macro and Mo-bis
Once upon a time there was an Americans company called Motorola, and they decided to implement GSM. Unfortunately they decided to deviate significantly from the specification and implement their own proprietary back-haul protocol between BTS and BSC, called Mo-bis. It replaces the standardized A-bis interface.
Today, There are plenty of phased-out Motorola Horizon / Horizon II macro BTSs that have been phased out. Basically you can get them for scrap value, which makes them an ideal target for GSM enthusiasts willing to run a single-cell network with little investment. So while there are actually people who are interested in operating a power-consuming device roughly the size of a washing machine in their home/office - they are normally not interested in running a 19" rack sized Motorola BSC with it. Also, the BSCs are much less frequently to be found compared to the BTS.
So it would be great to support Mo-bis from within OpenBSC. A couple of brave young men have set out to try the seemingly impossible. There's absolutely zero documentation available on that protocol, and no wireshark support either. However, the University of Brno (Czech) has a functional Motorola BTS + BSC setup, and I was able to obtain protocol traces from them and actually experiment with the equipment in their lab.
The entire Motorola GSM architecture seems to be over-engineered without end. Basically you are looking at a distributed computer from the early 1990ies. Lots of processor cards (m68k, ppc) interconnected by HDLC links on top of synchronous 2Mbps links with 64k timeslots. Those links are available e.g. on the backplane of the BTS as a TDM highway. So basically even inside the BTS, the individual processors talk over E1 to each other. In the BSC, there is a token ring based LAN between some of the cards instead. And the MCUF in the BTS even supports to transport those proprietary inter-cpu links via fiber optic (!).
Each processor has a 16bit identifier by which it can be addressed in form of physical addresses. Individual processes on the processors have fixed process identifiers, and they allocate a variety of mailboxes in which they can receive messages from remote processors. There are routing functions at intermediate notes.
So any process on any processor card can send messages to any mailbox of any other process on any other processor, independent of its physical location (locally at the BTS, or at the remote BSC, or even at remote BTSs).
Besides physical addresses, there are also functional addresses. Thos addresses are used particularly to support fail-over. Every board in a BTS and BSC can be fully redundant, and if you use physical addresses, you would address one of the two redundant boards. Using functional addresses, you address the function they both can perform, and some routing magic will make sure it ends up at the current active node in the pair.
There are multiple processors in every TRX, and a couple of processors for each BTS, processors in the E1 line cards, etc. Now speaking of the actual Mo-bis interface: It seems to be a weird mixture between 08.58 (RSL) and 08.08 (BSSAP/BSSMAP). However, after staring at the messages sufficiently long, I have been able to write a more or less complete wireshark dissector for them. Radio Channel Activation (RACH/IMM.ASS) are for example handled directly inside the BTS, they don't exist as transactions on the Mo-bis like they do in A-bis.
So implementing the actual location update / MO+MT voice call and SMS related transactions is actually not all that hard. What makes things really difficult is the way the BTS is initialized at startup. Basically what resembles the OML part of standardized A-bis.
There is a lot of low-level management and bring-up of the individual processes and boards, and the download of a large 500 kByte-sized BLOB simply called database. This binary database contains literally hundreds of configuration parameters for the BTS and its neighbors. It also contains sophisticated configuration of the message routers, the switching/multiplexing of 64k timeslots on the various links, information on redundant paths within the back-haul network, etc.
Interestingly, using the password combination 3beatles and 4stooges on any of the serial consoles of the BTS or BSC, you can enter into a "god-mode" which permits you to enter the executive monitor (EMON). The executive is the operating system they run on both m68k and ppc processors. It provides access to something like a syslog of messages from the various processes, and you can manually generate messages that are to be sent to mailboxes of processes. You can inspect the object table (application programs an databases), read/write to PCMCIA flash cards, read and write to logical and physical memory, inspect CPU and I/O usage and much more. In fact, the integrated Code Object Manager (COM) even allows the processors to synchronize their code versions and remotely boot other CPUS via HDLC channels.
For a communications system geek like myself, it's extremely fascinating to see such a sophisticated and versatile system. I only wonder why on earth somebody would come up with something as complex, only to connect a couple of BTSs to a BSC. Thus, the only logical explanation is that Motorola has developed this distributed proprietary computing system way before they went into GSM, and they probably just recycled it as it already existed.
If anyone knows more about the history of this, I would be excited to hear about it. It literally feels like being an archaeologist. Analyzing ancient technology from our forefathers. But then, it only is 20 odd years old. The only time I had a similar feeling was when I briefly came in touch with IBM mainframes in 2001 and looking at IBMs SNA protocol stack.
Some comments on the heated debate on SFC / Busybox / Linux GPL enforcement
It's been relatively difficult for me to figure out what is really going on here. It is well-known that the Free Software Conservancy has been actively enforcing the GPL on Busybox. But then, at the same time gpl-violations.org has been (and still is!) similarly active in enforcing the GPL on the Linux kernel. Still, I haven't yet seen calls to write a non-GPL Linux kernel replacement. Of course, the complexity is on an entirely different scale, so this point is moot.
However, for quite some time there have been rumors about the intensity (some would say aggressiveness) of the enforcement. I don't want to accuse anybody of anything, so I'm going to write speculatively about it.
This post is to summarize my thoughts on all of this:
It is well within the right of each author / copyright holder to decide on the enforcement strategy and license interpretation. As such, I respect the decision of the authors. It is their work, they should decide what to do.
In any kind of GPL enforcement, you of course not only want the complete corresponding source code to one program, but to all of the GPL/LGPL/AGPL or otherwise copyleft licensed programs contained in the product. We at gpl-violations.org have always been requesting the complete corresponding source code to all GPL licensed software during our communication with the infringing companies. This request was typically honored by everyone, without the need to apply any pressure onto it. After all, releasing only one bit of code causes the risk to get sued by somebody else who owns the other not-yet-compliant part of the code.
Now there have been rumors that SFC was not only requesting non-Busybox source code, but also making it a condition for the explicit re-instatement of the license on Busybox. Whether or not there was such a hard condition is subject to debate and there are different opinions on it. For those in the field of FOSS licensing, it has always known that there are different lines of thought with regard to the requirement to explicit reinstatement. We in Germany generally think that it is not required at all, and the existing preliminary injunctions at least implicitly acknowledge that as they enjoin companies from distributing a product as long as it is not in compliance with the license. In other (particularly the U.S.), it is generally assumed that explicit reinstatement is required. In such a case, it may very well be legally possible to use it as a lever to obtain source code for other programs like the Linux kernel. However, I am personally not sure if that really is the right strategy. Not everything that is possible legally is ethically the right thing to do. But then, ethics and legal customs differ widely in the FOSS communities, as they do in society in general. Some countries and communities believe in the death penalty, others don't. Some countries allow abortion, others don't. Some allow prostitution, others don't. So when judging about whether that "reinstatement lever" is acceptable or not, we have to accept that there may be different lines of thought. I for my part definitely think that the far superior method is, beyond doubt, to have a rights holder on those other program in order to make any demand for source code (as opposed to a mere request without implicit or explicit legal threat).
There also have been rumors about a requirement on submitting future source code releases to a compliance audit by the Conservancy. According to SFC sources, there never was any such demand, and the rumors are likely spawned by some incorrect claims of a defendant in a court case, which ended up in the public record. If there was such a requirement, I wouldn't think it is just - at least not for a first-time non-intentional infringement case. If there was repeated infringement and a clear sign that it would happen again and again, such a requirement for future audits may be justified, depending on the case.
People who claim that GPL enforcement is scaring away companies from using Linux and/or other Free Software also have to be careful in what they say. If a commercial entity enters a new market (let's say Android Tablets), then there is a certain due diligence required before entering that market. So if you don't understand Free Software and particularly GPL licensing, then you shouldn't place a Linux-based device on the market. Just think about an analogy: If you have a recycling company and enter a new market (disposal of hazardous chemicals), then you cannot simply treat those chemicals as regular waste, wait until you run into legal trouble and expect to get away with it.
I think there are still far too many GPL violations out there, and we need to see more enforcement in order to get all the major players in their respective lines of business into compliance. But come on, dealing with embedded devices in 2012 and still getting compliance outright wrong really means that there has not been the least bit of attention on this subject. And without enforcement, it is never going to change. People who want no enforcement should simply use MIT-style licenses.
Last, but not least, I also think GPL compliance is a matter of fair competition. There are some companies who really do a good job in ensuring compliance with the various Free Software licenses. If their competition doesn't invest the funds into the respective skills, procedures and business processes, they are getting an unfair competitive advantage against those who are doing it right. If there was no enforcement, the motivation would be to reduce efforts in compliance, not increase it.
Let me conclude with a clear statement to anyone who thinks that by replacing Busybox with a non-GPL licensed project they can evade GPL enforcement: It will not work. There are others out there enforcing the GPL. Last but not least gpl-violations.org. Despite the notoriously outdated webpage, we are still alive and kicking, churning down on the violation reports that we receive. Armijn Hemel, Joachim Steiger, Tim Engelhardt, Julia Gebert and Till Jaeger deserve much of the credit for all that work, while I'm mostly spending each awake minute hacking Free Software for mobile communications. Yes, we should publish more about our activities, and I hope to find the time to do so. There should at least be an annual report with the number of cases...
New OsmocomBB RSSI monitor firmware
Jolly has been hacking up a nice new RSSI monitoring firmware application for OsmocomBB.
I let the pictures speak for themselves:
I really hope this trend continues and we'll get some actual user interface in OsmocomBB at some point this year..
First osmo-nvs-gps evaluation boards soldered
At the osmocom project, we recently discovered the most interesting NVS NV08C-CSM module. It not only is a superb GPS receiver, but it includes GALILEO and GLONASS receivers, too. However, it's only available as an industry module, or as an expensive (700 EUR or so) evaluation kit.
Given the cheap PCB prototyping service at seeedstudio, I thought I'd spend an afternoon creating the schematics and PCB layout for an evaluation board. It exports the two 3.3V UARTs on OsmocomBB-style 2.5mm jacks, so they can be used with the T191 cables. I have the feeling this 2.5mm jack is becoming a new standard for low-voltage RS232 links ;)
Furthermore, it exports the SPI, I/O and I2C on a 20pin 2.54mm pitch header, connects to an external antenna via a MCX socket and has an optional footprint for a CR2032 battery on the bottom side.
So far, the board seems to be working fine. If there is interest in the bare PCB itself (without components!), please send me an e-mail. Depending on the amount of interest we might add it to the sysmocom webshop.
Schematics and Gerber files will be available at http://openbsc.osmocom.org/trac/wiki/osmo-nvs-gps soon.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!