Older blog entries for brouhaha (starting at number 70)

8 May 2005 (updated 8 May 2005 at 23:06 UTC) »
Nonpareil

This morning I was trying to figure out why Nonpareil isn't coming anywhere close to simulating the HP-67 correctly. It comes up with a single decimal point lit in the display, and the only visible responses to the keyboard are for the decimal point to move to another position or disappear.

So I captured an execution trace from the simulator, and started studying it. Almost immediately, I found that they are doing a ``DELAYED ROM SELECT'' instruction preceding a ``A -> ROM'' instruction.

On the real hardware, I don't know whether DELAYED ROM SELECT is only delayed by one instruction, or whether it is delayed until the next branch. When I wrote the Woodstock CPU code, I implemented it as being delayed until the next branch. But I forgot to handle it with the ``GOTO KEYS'' and ``A -> ROM'' instructions, which are computed branches based on the keycode and A register, respectively.

I also determined that opcode 0120 octal is almost certainly the ``KEYS -> A'' instruction.

Adding KEYS->A and fixing the DELAYED ROM SELECT did not fix the HP-67 problem. But it did fix the problems with the HP-34C, HP-37E, HP-38E, and HP-38C, all of which now seem to work correctly!

Nonpareil

Thanks to some help from HrastProgrammer, the HP-11C and HP-16C are now fully functional in the Nonpareil simulator. The display annunciators don't work yet, but everything else seems fine.

Recently I've mostly been working on improvements to the simulation infrastructure. I've now got working code to save the state of any simulated calculator to a disk file as a gzip-compressed XML file, and to restore from such a file, so Nonpareil will finally have continuous memory. :-)

There's still some remaining work on making the save/restore work nicely with the File menu, and making it automatically save on exit and restore on startup. And after a restore, I need to force a display update.

Dealing with compressed XML using libxml2 is very easy. It's a great library.

I decoded the "hex table" of the HP-16C. There are only 243 normal user programmable operations, so there are 13 illegal codes. I haven't yet tried executing the illegal codes, but I did wedge them into a save state file to see how they would be decoded while stepping through memory in program mode. Of these, two are "WINDOW 8" and "WINDOW 9", four duplicate "FLOAT 6" through "FLOAT 9", five are "FLOAT B" through "FLOAT F", and two duplicate "STO . 0" and "0".

I also spent some time matching up math routines in the HP-11C code with the VASM listings of the HP-41C math routines (file cn6b). They are almost identical, which isn't too surprising.

1 May 2005 (updated 1 May 2005 at 20:22 UTC) »
Nonpareil

I put together a crude register display window using the pseudo-reflection API I described previously. It uses a GtkTable with two columns, with GtkLabels in the left colum for the register names, and GtkEntry fields in the right column. I set the text width of the GtkEntry fields to the maximum number of characters the register can display as (i.e., 14 for full-word 56-bit registers in hexadecimal). By putting a GtkEntry directly into a table column, it forces all of them to be the same width, which I don't want.

For the stack, which is an array of four 16-bit values, I put a GtkHbox into the table, and put four GtkEntry fields into that. These entry fields still are wider than I would like, but they don't fill up the whole width of the table column. Hmmm... maybe I should put all of the entry fields into Hboxes, even when there's only a single one on the line.

Anyhow, the GtkEntry is OK for a prototype, but it's not what I want in the longer term.

I think I need to add some flags to the register description returned by pseudo-reflection, so that certain registers can be displayed in a special manner. For instance, it would be nice for the P and Q registers which point to specific digit positions in the word registers to be shown visually as pointers at the appropriate horizonal position.

Also, some registers are not significant unless execution is stopped in the middle of a multiple-cycle instruction, so they should not normally be shown (or saved).

groom: I've been thinking about the clunky SysVinit orgnaization for a while, and I think the entire current scheme should be done away with. There should be one text file per service. That text file should list what runlevels the service is supposed to run in, and what other services it depends on (requires to be running first). Then init can just do a dependency analysis (ala make) to determine what order to start things in, what things can be started concurrently (make -j), and what order to shut things down in.

Only the top-level services desired in a run level would need to be explicitly enabled, and the services they depend on would be inferred. Though there could be a setting to explicitly prohibit a service from running in a particular runlevel, which would also prevent things depending on it from running in that runlevel.

I've been wanting to prototype this for quite a while, but I haven't gotten a Round Toit yet.

Nonpareil

I've implemented what I call ``pseudo-reflection'' in Nonpareil. Each processor architecture has a unique C structure defined which represents the architecturally visible register state of a simulated processor, and in order for general-purpose state save/restore and debugging access to be possible, the GUI thread needs to be able to ask the simulator thread what fields (registers) exist in the structure, their characteristics (name, element size, array bounds), and function calls to read and write those registers.

If I was writing this in Java, Smalltalk, or Lisp, I could use a native reflection API provided by the language to get that information. In C and C++ there is no equivalent, although there do exist some incredibly kludgy hacks to do it in C++.

Anyhow, fundamentally all that is needed is to declare an array of ``meta structures'': structures that define the characteristics of the fields of the structures of interest. I've now done almost completely for the Nut (HP-41C) architecture, and for a few registers of the Classic and Woodstock architectures. I've implemented the API that the simulator thread will use: sim_get_register_info(), sim_read_register(), and sim_write_register().

sim_get_register_info() takes a numeric index (from 0 up) and returns a pointer to a substructure containing public information about a register. Since the data is static, it can execute in any thread, and will generally execute in the GUI thread.

sim_read_register() and sim_write_register() are contending for access to the registers with the simulation thread, so to get coherenet reads and writes, there needs to be some mutual exclcusion. This could be a lock per register or a big lock for the whole structure, but either way, it would require a *very* large number of code sequences in the simulator thread to acquire and release locks, which is not a good idea. So instead, these functions send a request through the async message queue to tell the simulator thread that the GUI (or debugger) thread wants to read or write a register, and the work is done in the simulator thread. This is how a lot of other simulator interface functions already work.

The next steps will be to attach the XML state save/restore code to this API, and to build a register display window that uses it as well.

Nonpareil

I've written a DTD for an XML Nonpareil state save file format, and code to read and write such files using libxml2. It was very easy. It's not yet integrated into the main Nonpareil program, because it will need the new memory API and pseudo-reflection. In the mean time, I wrote a utility that can convert the XML state save file format to and from the old NSIM simulator's state save file format.

XML is more verbose than the NSIM format, but because the XML files are automatically compressed on the fly, the new state files end up being 25 to 50% of the size they use in NSIM format.

26 Apr 2005 (updated 26 Apr 2005 at 08:59 UTC) »
Nonpareil

Rather than developing another ugly little language to save and restore calculator state in Nonpareil, I decided that I'd try using xml, via the libxml2 library, which I've never used before. The last time I wrote any software to do anything with XML was six years ago, when I hand-coded routines to emit and parse XML. That was a pain. I was pleased to discover that libxml2 provides fairly clean, easy-to-use APIs. For parsing, you can either let it build an entire parse tree in memory (DOM style), or give it callback functions to invoke as it reads the various XML elements, attributes, entities, and user data (SAX style). For Nonpareil, SAX is clearly the right thing.

For writing XML files, the simplest API provided by libxml2 is xmlTextWriter. This particular API appears to only be suited to single-threaded use, but that's fine for Nonpareil. (Nonpareil has multiple threads, but only one will ever be reading or writing a particular XML file at any given time.)

It's also nice that there is support for doing gzip compression and decompression on the fly.

It only took me three hours to come up with a suitable DTD, study the libxml2 documentation and write and debug a mockup consisting of one program that writes a Nonpareil save state file and one that parses it.

In fact, this worked so well that I'm considering replacing the use of (modified) KML files for configuring Nonpareil with another XML file. In this case, I'd probably want the XML file to come out of a ZIP file, so I'd have to write a little bit of glue to get the libxml2 parser to read from a gsfInput stream.

lcdtest

Judging by the Freshmeat stats, lcdtest is way more popular than I ever would have guessed. There must have been some unsatisfied demand for a test pattern generator. Perhaps I'll build Windows and MacOS X binaries for it as well. There already are some similar programs for those platforms, but perhaps there's some utility in having a common cross-platform program, and anyhow, since I used SDL it should be easy.

lcdtest

On a mailing list, someone pointed out a Windows lcd test program that was useful for adjusting the pixel clock frequency and phase of LCD monitors with analog VGA inputs (vs. DVI). It puts up alternating white and black vertical lines one pixel wide. This is useful either for auto-setup or for fine tuning.

lcdtest was able to do that, but you had to type the five key sequence ``wv0-'' to get it. So I've added the command ``a'' to do it with a single key.

I also added command line options to list the available video resolutions, select a resolution, and get help. There's a new option to display in a window, intended for debugging and example screenshots.

I now consider the program to be feature complete, so the new release is version 1.00.

lcdtest

I just replaced my aging Nokia 445Xi 21" CRT monitor with a new Samsung SyncMaster 213T 21" LCD monitor. Much easier on my aging eyes that can no longer focus close in, as the LCD face can sit much further back on my desk than the CRT face did.

I thought I'd try to count bad pixels. A Google search turned up various LCD test pattern programs for Windows and MacOS, but none for Linux. So naturally I thought I'd whip up a quick hack to fill this void.

lcdtest uses SDL for full-screen display, and in principle should work on other operating systems as well, but I've only tested it on an Athlon 64 running Fedora Core 3 Linux.

7 Apr 2005 (updated 7 Apr 2005 at 00:03 UTC) »
Nonpareil , la2vcd (new)

As mentioned in my last entry, I've been capturing microcode execution traces from an HP-97 using an HP 16500B logic analyzer, in order to reverse-engineer the additional instructions used in the Topcat series and similar HP calculators, to add those to Nonpareil. The logic analyzer provides the captured data as a set of text files, one per signal, with one data sample as binary (represented as ASCII text) per line.

That format isn't directly useful for much, so I've had to write a few translation programs. One of them that might be useful to other people is la2vcd, which will take the HP (or Agilent) logic analyzer traces, and convert them into a VCD (Value Change Dump) which is supported by most waveform-viewing software, including GTKWave.

Unfortunately extracting the data from the analyzer is a very slow process, even over Ethernet. At most I've seen a little over 10 KBytes per second. Apparently it is doing on-the-fly format conversion from its internal binary representation into the test form. The poor little 68000 in the analyzer is just too wimpy. A 2 megasample capture of five signals using a 16555A module took nearly two hours to FTP out of the analyzer. In the future I'll just tell the analyzer that all five (or more) inputs are part of a single vector, which will at least reduce the overhead somewhat.

Even with this long delay to get data out of the analyzer, it is still much better using GTKWave to inspect the results than trying to do it with the analyzer's built in display and knob. The poor thing takes over five seconds for each display update when you turn the knob one detent (which is like clicking a scroll arrow; it only scrolls a small amount of new material onto the display).

Maybe someday I'll be able to afford a used 16500C, which is 68020 based and might be less sluggish. But at least the actual acquisition capabilities of the 16500B/16555A are quite good.

I expect that the latest Agilent logic analyzers probably can directly export in VCD format, but I haven't researched it since I can't afford to buy one. If they don't, they certainly should!

Nonpareil

Hardware mystery solved:

Ever since I've been working on reverse-engineering the Woodstock/Spice era hardware, and figured out how the bank-switching works on Spice (and probably on Woodstock, though I haven't yet confirmed that), I've wondered how the multiple ROM chips all make sure they agree on the bank, and how they're forced to start in bank 0 at power-up. The mystery is that there is no reset signal going to the ROM chips (or ROM/RAM chips).

In theory it would be possible for the chips to each have their own internal power-up-reset logic, but in practice this would increase chip area and might not reliably get them all reset properly. Using 1970s technology, this would have required external RC components. It would have made even less sense for HP to put those in for each ROM than just to bus a reset signal from the ACT to the ROMs. But they didn't do either.

Studying a logic analyzer trace, I think I now know the answer. Each word cycle consists of 56 bit cycles, which are defined by the two-phase clock pulses. The various chips have to know when a word cycle begins, and they do that by watching the SYNC line, which is generated by the ACT. The SYNC line is low most of the time, but goes high for ten bit cycles of each word. Those are the ten cycles in which a ROM instruction is read out. Each chip on the bus other than the ACT has an internal counter that tracks the current bit time, and it uses the SYNC line to initialize that.

There are some two-word instructions, and the ACT does not raise the SYNC line while fetching the second word, so the counters in the chips are designed to "ride through" the missing SYNC period. They also know that they shouldn't try to interpret the ROM words bits that are read out when SYNC is not present as an instruction, because it's really a branch target.

I've captured a lot of logic analyzer traces in the past, but tonight I captured one from an HP-97 during initial power-up, and I think I now understand how the system-wide reset is implemented. At powerup, the ACT starts generating the clocks, and it holds the SYNC line low for about four milliseconds to allow various things in the system to stabilize. Then it starts generating narrow SYNC pulses once per clock. It generates about 30 of those, with somewhat variable width, then starts holding SYNC high. It keeps SYNC high for about 22 ms. Finally, it lowers SYNC, then begins pulsing it at the usual rate (10 bit times high out of every 56, except for the second word of two-word instructions).

So it appears that the ROMs and other "peripheral chips" are designed to reset themselves if they seen SYNC high for more than ten bits consecutively. Quite a clever solution to having a hardware reset without any extra pins.

I think the narrow SYNC pulses before the 22 ms high time are probably just an artifact of the ACT implementation; the ACT likely is not yet fully initialized yet. I doubt that any of the other chips notice these pulses at all, because they happen after the phase 2 pulse, rather than staying high throughout the phase 2 pulse.

I've also spent a little time recently on infrastructure improvements to Nonpareil, like making the word size architecture-specific (56 bits for most HPs, but 48 bits for the HP-01 watch, 64 bits for more recent Saturn-based calculators, and 8 bits for the HP-75 and Series-80 computers). I've come up with a new API to deal with memory reads and writes from the GUI task, a new API to manage multiple sparse address spaces with bank switching, and what I call a "pseudo-reflection" API for the GUI task to ask the simulator task what processor state can be accessed.

I still haven't decided whether to change the internals from using an array of 14 bytes (uint8_t) to represent a 56 bit calculator word (4 bits per byte) to using a packed value (uint64_t). The latter makes the memory API more elegant, but adds code to the simulation task.

Altogether

I've got Altogether using libgsf to read the microcode files. It works great. I wish the ISO C committee had added some way to hook into the stdio interface from the underside, i.e., to supply an alternate implementation of the functions needed to implement the FILE abstraction. Some platforms do provide this but not in a portable way. If it could be done, libgsf could allow you to use stdio calls on the components of ZIP files. As it is, libgsf provides its own read and write calls, rather than working through the GLib I/O channel API. Sigh.

61 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!