Older blog entries for brouhaha (starting at number 75)

Nonpareil 0.66

I started work on a RAM display window for Nonpareil's GUI debugger. The first attempt resulted in a window taller than the screen; I had to learn how to use a GTK viewport to get scrolling capability. Now the window starts out way to small, but at least is resizable. I'll have to figure out how to set the initial size to something reasonable.

Like the register display window, it isn't yet fully functional. Although they both use editable entry boxes for the display, if you edit them the change is not actually written back to the register or memory. And they don't update automatically, so you have to close the window and reopen it to see simulation changes. Still, it's been somewhat useful for debugging.

I'm not happy with the use of the standard GTK+ entry widget. I really want a widget that displays a number using a fixed base and digit count, in a monospaced font (or at least with the characters in fixed column positions), so that the digits in all the registers or RAM locations line up vertically, and there isn't extra empty space at the right side of the widget. Colored highlights for the P and Q pointer positions would also be useful. I'll have to write such a widget myself, and I probably won't get around to it for a while.

Google revealed that Adam Sampson has built a GARstow package for Nonpareil, and was frustrated with SCons. I discussed it with him by email, and all of his problems were really due to inadequacies in the way I used SCons, rather than with SCons per se. For instance, my SConstruct/SConscript files supported a "prefix" configuration option, but not "bindir" and "libdir" for more flexible filesystem layout, nor "DESTDIR" for a virtual filesystem root for installation packaging. Also, doing "scons install" was putting ".sconsign" files into the installation directories, which was resulting in file conflicts between various packages that use SCons.

I expect the same problems would affect almost any packaging system, including my own anticpated effort to offer RPMs for Fedora Core 3 and 4, so I've fixed them today for release 0.66. I added "bindir" and "libdir", as well as "destdir" (note lower case for consistency with other Nonpareil configuration options).

Also, by putting an "SConsignFile()" directive in the SConscript, it will put all of the signature information it uses for dependency analysis and tracking into a single ".sconsign.dblite" database file in the top directory, so it no longer puts .sconsign files into the install directories.

I added caching of the configuration options in "nonpareil.conf" to replace my previous "local.py" configuration file. And a "Help" directive so that "scons -h" will show all of the configuration options with their default and current values. To revert all options to the defaults, you can simply delete the "nonpareil.conf" file from the top directory.

I updated the INSTALL file as well as the credits in the README, added the people from the credits to the About dialog box, and moved the code for the About dialog box into separate about.c and about.h source files.

Thomas Olesen has sent me his patches to get the HP-41CX running, based on earlier modifications to nsim by Chris Roccati. I should have that integrated for the next release.

I sent Maciej Bartosiak email about the possibility of using POSIX threads in place of the glib thread support on MacOS X, and it turns out he'd already done that. He sent me his modified version of proc.c.

Nonpareil

Finally squashed a nasty bug. I thought I'd previously managed to fix the calculator state restoration bug, but it turns out that the bug only manifested on 32-bit machines and I was trying to debug it on a 64-bit machine. There was a call to strtoul() that should have been strtoull().

It's unfortunate that the C standard doesn't specify any replacement for those that is based on the actual size of the type (e.g., uint32_t, uint64_t) rather than the weakly defined ``unsigned long'' and ``unsigned long long''. I suppose the correct thing to do would either be to have a configuration test at build time to pick the right one, or to use sscanf() instead, since the C standard does provide size-appropriate format specifier strings in <inttypes.h>. Perhaps I'll add a strtouint64() function in my utility package, based on sscanf().

lcdtest

lcdtest release 1.01 has a new ``dots'' test pattern, and an on-screen help message.

Nonpareil

Release 0.63 of Nonpareil now simulates the HP-11C, HP-12C, and HP-15C calculators.

13 May 2005 (updated 13 May 2005 at 05:40 UTC) »
First impression of Eclipse

When I heard that Fedora Core 2 was going to include natively-compiled Eclipse, I was excited since I'd wanted to try Eclipse for quite a while but had never been able to make it work. Unfortunately the native Eclipse didn't actually make it into FC2 or even FC3, but it is part of the forthcoming FC4.

I'm running Fedora Core 4 Test 3 on an Athlon 64 3500+ with 1 GB of RAM, so I thought I'd give the native Eclipse a try, along with CDT for C development. I haven't done much with it yet, but the feature set seems quite nice. I use SCons instead of make, so I wanted to try the SConsBuilder plugin, but there isn't yet a version for Eclipse 3. I installed the Subclipse plugin for use with the Subversion configuration management system, but I haven't yet figured out how to use it.

The big problem is that Eclipse is mind-numbingly slow. Even just right-clicking on a source file takes a long time to bring up a context menu. If it's this slow when natively compiled on an Athlon 64, I can't even imagine what running a non-native version must be like. How can anyone find this useful?

Finally a new release of Nonpareil

I released version 0.61 of Nonpareil last night. Now it has support for the HP-16C "Computer Scientist" calculator, and due to bug fixes the HP-34C, HP-37E, HP-38E, and HP-38C finally work correctly.

The big news is that there is finally support for saving and restoring the calculator state, so there is finally "Continuous Memory", even for calculators that didn't actually have that as a hardware feature.

The state save uses compressed XML files thanks to libxml2.

This is also the first release that builds with SCons instead of make.

I had to solve a tricky memory corruption bug to get this release ready. One of my data structures was being corrupted. I used GDB to set a hardware watchpoint on a field of the structure, but the watchpoint never tripped, even though the field did get changed! Possibly a problem with 64-bit GDB in Fedora Core 4 test 2.

Nonpareil ad nauseum

I've been fixing bugs in Nonpareil, and improving the SCons build files I use instead of Makefiles.

I've finally got the 14/15-digit display mode of the Woodstock processor (ACT) working correctly, as far as I can tell. That's used in the HP-19C, HP-67, and the HP-9x printing desktop calculators. I'm trying to get the HP-67 running.

The HP-67 normally uses the "KEYS -> A" instruction to read the keyboard, then does fairly complicated stuff to the value in A to determine where to branch. Earlier calculators just used "KEYS -> ROM", which directly replace the low eight bits of the program counter with the hardware keycode (not the same as the user-visible keycode). I was not able to figure out what the hardware keycodes should be from Tony Duell's reverse-engineered schematics, which is not his fault, but rather because I don't have enough documentation on the ACT. I tried to figure out what they should be by studying a microcode disassembly, and by trial and error, but it was very slow progress and it wasn't easy to figure out the correct codes because many incorrect codes map to normal operations. I finally decided that the best thing to do was to write a test program to run on the actual calculator. I don't currently have a means to do that, but I wrote the program and tested it on the simulator. As each key is pressed, it displays the hardware keycode as a three-digit octal number.

Bernhard Engl build a EPROM-based ROM emulator and has it interfaced to an HP-67 in place of the normal ROM chips, so I sent him my code. He burned it into EPROMs and ran it, and we were both surprised that it worked correctly the first time. He was able to determine all of the hardware keycodes and email them to me, and when I plugged them into the KML file and fixed a few minor bugs, the HP-67 started partially working in simulation. Bernhard says that he'll send me some photos of his hardware setup.

I've studied a lot of disassembled Woodstock code, but that's the first Woodstock code I've written. Just reading and studying the code others have written before wasn't enough to give me a full appreciation for how challenging it can be, due to the non-orthagonality of the registers. I'm thinking about writing a game in microcode to get more practice. Too bad most people will never be able to play it except on the simulator.

Currently the HP-67 simulation works until you do anything that requires the calculator to search for a label in program memory. The search routine enters an infinite loop. The loop is entered with P=1, and each iteration through the loop P gets incremented by 2. The maximum value of P is 13, beyond which it wraps to zero, so the next loop value after 13 is 1. However, the test for the end of the loop tests for P=0. An engineering spec for the ACT states that when P wraps on either an increment or decrement, the position "disappears" for one cycle. I suspect that this vague description is wrong, and that in this particular case P happens to have a transient value of 0 at the time the comparison is made. It's possible that this behavior is different than that of the earliest ACT chips; it is known that some revisions of the HP-67 and HP-97 microcode are not compatible with all ACT chips, and perhaps this could be why.

Typical SCons "Builders" take one source file and produce one target file. To define a rule that produces multiple target files, it is necessary to write an "Emitter". As part of the Nonpareil build process, the uasm microassembler gets built, then uasm is used to assemble microcode for several calculator models. Although it's not strictly necessary, I want the assembly to result in listing files as well as object files. I tried to write an Emitter for that months ago, based on the examples in the SCons documentation, and didn't get very far. Now that I know a tiny bit more Python, I hacked on that a bit more, and now it's basically working. The remaining problem is that I want the listing files to go into the obj subdirectory where the object files go, but it's actually creating another obj directory under that for them, so that they are in paths like obj/obj/hp45.lst.

It turns out that I'm not the only person that has been surprised that SCons doesn't come with good support for building source code distribution tarballs. Only recently this has come up on the scons-users mailing list, and Paul Davis posted a nice source tarball Builder. I tried to get it working with the Nonpareil SConstruct and SConscript files, and get an error I don't understand. So I created a simpler test case, and it works for that. Seems to be something about the way my hierarchy works. I hope to have it figured out soon, since I want to package a new source release.

8 May 2005 (updated 8 May 2005 at 23:06 UTC) »
Nonpareil

This morning I was trying to figure out why Nonpareil isn't coming anywhere close to simulating the HP-67 correctly. It comes up with a single decimal point lit in the display, and the only visible responses to the keyboard are for the decimal point to move to another position or disappear.

So I captured an execution trace from the simulator, and started studying it. Almost immediately, I found that they are doing a ``DELAYED ROM SELECT'' instruction preceding a ``A -> ROM'' instruction.

On the real hardware, I don't know whether DELAYED ROM SELECT is only delayed by one instruction, or whether it is delayed until the next branch. When I wrote the Woodstock CPU code, I implemented it as being delayed until the next branch. But I forgot to handle it with the ``GOTO KEYS'' and ``A -> ROM'' instructions, which are computed branches based on the keycode and A register, respectively.

I also determined that opcode 0120 octal is almost certainly the ``KEYS -> A'' instruction.

Adding KEYS->A and fixing the DELAYED ROM SELECT did not fix the HP-67 problem. But it did fix the problems with the HP-34C, HP-37E, HP-38E, and HP-38C, all of which now seem to work correctly!

Nonpareil

Thanks to some help from HrastProgrammer, the HP-11C and HP-16C are now fully functional in the Nonpareil simulator. The display annunciators don't work yet, but everything else seems fine.

Recently I've mostly been working on improvements to the simulation infrastructure. I've now got working code to save the state of any simulated calculator to a disk file as a gzip-compressed XML file, and to restore from such a file, so Nonpareil will finally have continuous memory. :-)

There's still some remaining work on making the save/restore work nicely with the File menu, and making it automatically save on exit and restore on startup. And after a restore, I need to force a display update.

Dealing with compressed XML using libxml2 is very easy. It's a great library.

I decoded the "hex table" of the HP-16C. There are only 243 normal user programmable operations, so there are 13 illegal codes. I haven't yet tried executing the illegal codes, but I did wedge them into a save state file to see how they would be decoded while stepping through memory in program mode. Of these, two are "WINDOW 8" and "WINDOW 9", four duplicate "FLOAT 6" through "FLOAT 9", five are "FLOAT B" through "FLOAT F", and two duplicate "STO . 0" and "0".

I also spent some time matching up math routines in the HP-11C code with the VASM listings of the HP-41C math routines (file cn6b). They are almost identical, which isn't too surprising.

1 May 2005 (updated 1 May 2005 at 20:22 UTC) »
Nonpareil

I put together a crude register display window using the pseudo-reflection API I described previously. It uses a GtkTable with two columns, with GtkLabels in the left colum for the register names, and GtkEntry fields in the right column. I set the text width of the GtkEntry fields to the maximum number of characters the register can display as (i.e., 14 for full-word 56-bit registers in hexadecimal). By putting a GtkEntry directly into a table column, it forces all of them to be the same width, which I don't want.

For the stack, which is an array of four 16-bit values, I put a GtkHbox into the table, and put four GtkEntry fields into that. These entry fields still are wider than I would like, but they don't fill up the whole width of the table column. Hmmm... maybe I should put all of the entry fields into Hboxes, even when there's only a single one on the line.

Anyhow, the GtkEntry is OK for a prototype, but it's not what I want in the longer term.

I think I need to add some flags to the register description returned by pseudo-reflection, so that certain registers can be displayed in a special manner. For instance, it would be nice for the P and Q registers which point to specific digit positions in the word registers to be shown visually as pointers at the appropriate horizonal position.

Also, some registers are not significant unless execution is stopped in the middle of a multiple-cycle instruction, so they should not normally be shown (or saved).

groom: I've been thinking about the clunky SysVinit orgnaization for a while, and I think the entire current scheme should be done away with. There should be one text file per service. That text file should list what runlevels the service is supposed to run in, and what other services it depends on (requires to be running first). Then init can just do a dependency analysis (ala make) to determine what order to start things in, what things can be started concurrently (make -j), and what order to shut things down in.

Only the top-level services desired in a run level would need to be explicitly enabled, and the services they depend on would be inferred. Though there could be a setting to explicitly prohibit a service from running in a particular runlevel, which would also prevent things depending on it from running in that runlevel.

I've been wanting to prototype this for quite a while, but I haven't gotten a Round Toit yet.

Nonpareil

I've implemented what I call ``pseudo-reflection'' in Nonpareil. Each processor architecture has a unique C structure defined which represents the architecturally visible register state of a simulated processor, and in order for general-purpose state save/restore and debugging access to be possible, the GUI thread needs to be able to ask the simulator thread what fields (registers) exist in the structure, their characteristics (name, element size, array bounds), and function calls to read and write those registers.

If I was writing this in Java, Smalltalk, or Lisp, I could use a native reflection API provided by the language to get that information. In C and C++ there is no equivalent, although there do exist some incredibly kludgy hacks to do it in C++.

Anyhow, fundamentally all that is needed is to declare an array of ``meta structures'': structures that define the characteristics of the fields of the structures of interest. I've now done almost completely for the Nut (HP-41C) architecture, and for a few registers of the Classic and Woodstock architectures. I've implemented the API that the simulator thread will use: sim_get_register_info(), sim_read_register(), and sim_write_register().

sim_get_register_info() takes a numeric index (from 0 up) and returns a pointer to a substructure containing public information about a register. Since the data is static, it can execute in any thread, and will generally execute in the GUI thread.

sim_read_register() and sim_write_register() are contending for access to the registers with the simulation thread, so to get coherenet reads and writes, there needs to be some mutual exclcusion. This could be a lock per register or a big lock for the whole structure, but either way, it would require a *very* large number of code sequences in the simulator thread to acquire and release locks, which is not a good idea. So instead, these functions send a request through the async message queue to tell the simulator thread that the GUI (or debugger) thread wants to read or write a register, and the work is done in the simulator thread. This is how a lot of other simulator interface functions already work.

The next steps will be to attach the XML state save/restore code to this API, and to build a register display window that uses it as well.

Nonpareil

I've written a DTD for an XML Nonpareil state save file format, and code to read and write such files using libxml2. It was very easy. It's not yet integrated into the main Nonpareil program, because it will need the new memory API and pseudo-reflection. In the mean time, I wrote a utility that can convert the XML state save file format to and from the old NSIM simulator's state save file format.

XML is more verbose than the NSIM format, but because the XML files are automatically compressed on the fly, the new state files end up being 25 to 50% of the size they use in NSIM format.

66 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!