Recent blog entries for Pizza

Mitsubishi CP-D70 family, now with more Raspberry Pi Goodness!

Back in March, my reverse-engineered processing library for the Mitsubishi CP-D70DW family of printers reached quality pairity with the official Windows/OSX drivers and their SDK. This was a huge milestone; it's now possible to drive these printers using entirely Free Software, with no quality compromises, on non-x86 CPU architectures. To do this, you will need Gutenprint 5.2.13-pre1 or newer, along with an up-to-date lib70x snapshot.

Here are the happy models:

  • Mitsubishi CP-D70DW
  • Mitsubishi CP-D707DW
  • Mitsubishi CP-D80DW
  • Mitsubishi CP-K60DW-S
  • Kodak 305
  • Fujifilm ASK-300 [completely untested]

I held off on announcing this achivement due to one rather annoying problem. It seems most folks interested in using this family of printers wanted to do so using the popular Raspberry Pi systems.
Unfortunately, for (at the time) unknown reasons, the CP-D70DW spectacularly failed to print with RPi systems, with the image data failing to transfer to the printer.

To further complicate things, I was unable to recreate the problem on the Kodak 305 (aka a rebadged CP-K60DW-S) and an RPi2. Software-based sniffs were useless, and things stayed at this impasse for quite some time.

What finally broke the logjam was one particularly motivated user running into a similar problem with an RPi3 and a CP-D80DW model -- but only when there was nothing plugged into the ethernet controller. This was a bizarre development, and a working theory that there was something funky going on with the RPi's onboard SMSC USB hub/ethernet chip began to develop.

He convinced Mitsubishi/Europe that it was in their interests to help figure this problem out, and I soon found myself with a CP-D80DW and several boxes of media on my doorstep and a backchannel to their technical team.

Try as I might, I was unable to recreate the problem on a RPi1 or RPi2, but on an RPi3, I was finally able to trigger the failure. Armed with a true USB sniffer and a copy of the USB spec, I was able to rapidly identify what was acutally going on.

The USB2 spec (section 8.5.1) dictates that, when in high-speed mode, a flow control mechanism (NYET/PING/[N]ACK) be used to maximize transfer throughput. NYET is sent by the device after a successful transfer to signal that it can't accept more. The controller then starts polling the device with PING requests, with the printer responding with a NACK until it's ready, at which point it responds with an ACK.

The problem, in a nutshell, is that the RPi's USB controller ends up sending the PING requests so rapidly -- about 400 nanoseconds apart) that it effectively triggers a denial-of-service (DoS) situation in the printer. The funny thing is that the RPi's controller starts with a PING interval of about 1.2 microseconds, and everything is fine until it pulls in the timing, triggering the DoS.

(A regular PC with an xHCI controller demonstrates variable PING timing; it starts at about 400ns, but backs off to about a 4us interval)

If other USB traffic is present, then the bus is too busy to send the PINGs so rapidly, and the printer is fine. Notably, simply establishing a physical lin with the onboard (USB-attached) ethernet controller (even if the interface is "down") generates sufficient traffic to mitigate the problem. Plugging in a thumbdrive and dumping data back or forth is also more than adequate.

I've reported this problem to both Mitsubishi and the RPi kernel folks.
I don't think one can expect much progress on the root cause of this problem, but it the above workarounds are at least adequate to using these printers with RPi systems.

...Enjoy, folks, and happy printing.

Syndicated 2017-06-15 23:04:06 from I Dream of Rain (free_software)

Mitsubishi P95D

The Mitsubishi P95D is the latest model in a line of Medical/Scientific monochromatic thermal printers that can often be found attached to the likes of Ultrasound stations.

As of December 20th, it now has first-class Linux support as part of Gutenprint, complete with status/error reporting, mutiple copy support, custom page sizes, and every other feature the printer exports.

I may try to extend support to older models in the family (P93 and P91) or other MedSci thermal printers if there's any interest.

Oh, here's a shot of the P95 in action:

228432:338206

Syndicated 2017-01-07 02:59:11 from I Dream of Rain (free_software)

Improvements for newer Canon SELPHY models

About a year of so ago I added support for the newer Canon SELPHY printers (CP820, CP910, CP1000, and CP1200) into Gutenprint. Despite using the same media kits as their older siblings, under their plasic bodies they sported a new print engine that worked fairly differently.

Slightly different print sizes, a Y'CbCr image format, and, surprisingly, they appeared to be sane USB Printer class models and not require a special backend to handle communications.

Fast forward to last week, and it turns out that was a premature assessment. While the printers didn't require any special handholding to print a single image, they would lock up if one would send over two jobs back-to-back. Canon still can't implement proper flow control.

Time to reak out the sniffer and capture some multi-page jobs! A quick flurry of hacking later, and the 'canonselphyneo' backend was born. It brings along sane flow control, status reporting, and error detection on par with the selphyneo's older siblings.

I also discovered the 'L' print size was incorrect. All of this will will be in Gutenprint 5.2.12-pre5 or newer, but the current backend can always be grabbed from my selphy_print repository.

Oh, as I write this, I don't have the USB IDs for the CP820 or CP1000 models. I need those so they'll be recognized by the backend. Holler if you have one!

Syndicated 2016-12-10 02:30:20 from I Dream of Rain (free_software)

Call for testing on Mitsubishi printers

In the past I've written about the particularly poor level of support for Mitsubishi printers under Linux. In the past couple of months, that has changed substantially, although not due to any action or assistance on Mitsubishi's part.

Gutenprint 5.2.11 had usable support for the CP-9550DW and CP-9550DW-S models, including an intelligent backend that handled the printer communications. However, the rest of the CP-9xxx family wasn't supported.

The 5.2.12 release of Gutenprint will support most of the CP-9xxx family. This includes a considerable amount of work on the backend, genericizing it so that the other models can be cleanly handled. This was a rather disruptive change, so it's possible the formerly-working CP-9550 family has regressed. Beyond that, the newly-supported models need testing to confirm everything functions as expected.

Here is the list of all models affected by this development:

  • CP-9000DW
  • CP-9500DW
  • CP-9550DW (previously working, needs retesting)
  • CP-9550DW-S (previously working, needs retesting)
  • CP-9600DW
  • CP-9600DW-S
  • P-9800DW
  • CP-9800DW-S (confirmed working!)
  • CP-9810DW
  • CP-9820DW-S

Also, I still need testers for the following models:

  • DNP DS80DX
  • Sony UP-CR10L (aka DNP DS-SL10)
  • Mitsubishi CP-D70DW, CP-D707DW, CP-D80DW, and CP-D90DW (plus their -S variants!)
  • Fujifilm ASK-300
  • Sinfonia CHC-S1245/E1 and CHC-S6245/CE1
  • Kodak 7000, 7010, 7015, and 8810

If anyone reading this has access to one or more of these printers, please drop me a line!

Syndicated 2016-10-18 15:23:19 from I Dream of Rain (free_software)

Mitsubishi CP-D70 family, working!

Over the past few years, I've written a lot about the various members of the Mitsubishi CP-D70 family of printers, and how they were utterly worthless under Linux.

These printers are unusual in that they required the host computer to perform perform gamma correction and thermal compensation to the image data in order to generate sane output. (If this sounds familiar, it's because the Sinfonia CS2 was similarly afflicted). This relied on unknown, proprietary algorithms only implemented within Mitsubishi's drivers.

To make along story short, I've been attempting to reverse engineer those algorithms for nearly three years, and I'm pleased to announce that I've finally succeeded. While I can't promise the results are identical to Mitsubishi's own code, It is now possible to generate high-quality prints with these printers using entirely Free Software.

The library is called 'libMitsuD70ImageReProcess', released to the public under a GPLv3+ license.

Just to be absolutely clear, Mitsibushi is not responsible for this library in any way, and will not support you if you complain that the output is somehow deficient or your printer catches fire when you try to print pictures of Margaret Thatcher posing in her skivvies.

Here's the list of the now-functional printers:

  • Mitsubishi CP-D70DW, CP-D707DW, CP-K60DW-S, CP-D80DW
  • Kodak 305
  • Fujifilm ASK-300

While all of these models are expected to work, only the Kodak 305 has actually been tested. Please drop me an email or comment here if you have one of these printers and would like to help test things out.

In particular, if there's someone out there with a dual-decker CP-D707DW, there are opportunities for further enhancements that I'd like to try.

All code except for the library is now committed into Gutenprint, but is not yet part of any [pre-]release. So if you want to test this stuff out, you'll need to grab the latest Gutenprint code and libMitsu70ImageReProces out of git and compile/install them manually. Once this is a little more stable I'll package the library code in a tarball for simpler distribution.

In other news, while the Kodak 305's official Windows drivers only expose 4x6 and 8x6 prints, the printer firmware also supports the 6x6, 4x62, and 2x62 sizes that the Mitsubishi CP-K60DW-S supports. You'll need to ensure you're using the 1.04 firmware! I have also received reports that the '305 will accept the K60's print media, so in theory 5x7 and 6x9 support is possible.

Happy printing!

Syndicated 2016-10-06 19:08:53 from I Dream of Rain (free_software)

Progress with the Mitsubishi CP-K60DW-S and Kodak 305

As mentioned earlier, the Kodak 305 is a rebadged variant of the popular Mitsubishi CP-K60DW-S. Both models, along with their siblings (ie the CP-D70/D707/D80 and the Fuji ASK300), utilize a print engine that requires the host system to perform thermal compensation and other processing typically handled within the printer itself.

Not knowing how those algorithms work, these printers haven't been terribly useful under Linux, with the only prospect being a painful reverse-engineering of the Windows drivers.

A couple of months ago, Mitsubishi released a x86 binary-only CUPS driver for the D70 and D707 models. Other models weren't supported, and naturally this wouldn't work on other CPU architechures. Still, it provided a much more attractive reverse-engineering target, and as my interest came and went, I made slow progress.

Another wrinkle is that the CP-K60DW-S and Kodak 305 are slightly different from the others, to the point where my backend code was unable to print at all -- the printer communicates fine until the image data is sent over, and then nothing. All attempts to diagnose this further hit a dead end. That said, due to the unknown image processing algorithms, it was largely an academic exercise anyway.

Fast forward to a week ago. I lowballed a bid for a used Kodak 305 along with some unused media, in an "as-is, no warranty or returns" state. The printer arrived two days ago, and I got lucky; not only did the printer work, but it had about 140 prints worth of remaining media, far more than enough to get things going. An added bonus is that the printer had only logged 587 prints in total, a pittance for this kind of printer.

I quickly found and fixed a small pile of bugs, but successful printing still eluded me. I could find no differences in the data being sent between Windows vs Linux. Lacking a true hardware USB sniffer, I turned my attention to the enumeration to see if anything different was going on there.. and in the process I discovered something unexpected.

It turns out that these two printers enumerated differently under Linux than under Windows. Under Windows, only two endpoints were reported, one each for bulk input and output. However, under Linux, there were three endpoints; two for output and one for input. It seems that the backend was attaching to the second output endpoint, which accepted all commands, but not the actual image data needed to submit prints.

I'm at a loss to explain why the device would enumerate differently under Windows vs Linux; perhaps there's somthing funky going on behind the scenes and usbpcap only saw "cooked" data based on the driver's needs, whereas Linux's view more accurately represented the actual negotiation?

In any case, the backend now binds to the first pair of endpoints it finds, and can now print on all models of the printer family -- but the output is still unusably awful.

That success enables me to resume my reverse-engineering efforts, with an actual printer to test my experiments upon. Fortunately, the printers all use the same algorithms, differing only in a couple of tabular (CSV-based) data files. Inferring the internal data structures based on how the data flows into the system will be my next step.

Things are looking up, and perhaps sooner rather than later Mitsubishi's current printer family will be fully usable with Free Software. It's a shame this effort is even necessary, but I do love a challenge!

Syndicated 2016-09-02 13:22:23 from I Dream of Rain (free_software)

Netflix and Hurricane Electric's IPv6 service

For a few years now, I've used Hurricane Electric to get a native IPv6 tunnel to the internet. I've also been using Netflix streaming since it was first introduced. Life was good.

Netflix, on behest of its content suppliers, has started to crack down on folks using VPNs or proxys, because they're often used to work around artificial geographical restrictions.

A day or two, that blocked proxy list grew to include Hurricane Electric's IPv6 service, which I make heavy use of. Despite a US billing address, being physically located in the US, and using a US tunnel endpoint, Netflix treats me as an eeeevil bad person.

Their only advice is "disable your proxy", which is not an option as I have IPv6-attached servers that need to remain online.

Netflix's applications don't provide a way to utilize IPv4 only, which basically means I had to figure out a way to force netflix traffic to travel over IPv4. Ideally, I'd block the IPv6 AAAA dns lookups, but there's no simple way to do that.

However, one can just null-route the entire Netflix IPv6 address range:

    ip -6 route add blackhole 2406:DA00:FF00::/48

This will, after a little delay, cause Netflix to fall back to using IPv4, and all is well.

Ironically, being able to avoid this sort of BS is one of the reasons why Netflix was such a compelling service, but the balance is tilting back towards piracy providing a better overall user experience. Part of me hopes that the stats show a nice correlation between making legal services less useful and piracy rates going back up.

Addendum: About a year ago, my ISP (Comcast Business) rolled out native IPv6 service which by all acconts works quite well. Unfortunately, they don't offer a static IPv6 allocation, which renders the whole thing useless for my needs.

Syndicated 2016-06-04 12:41:06 from I Dream of Rain (free_software)

This is why ECC RAM is a good thing

    [2954432.978093] [Hardware Error]: Corrected error, no action required.
    [2954432.982285] [Hardware Error]: CPU:6 (10:8:0) MC4_STATUS[-|CE|MiscV|-|AddrV|CECC]: 0x9c5cc820001c017b
    [2954432.986453] [Hardware Error]: MC4 Error Address: 0x000000054a089400
    [2954432.990528] [Hardware Error]: MC4 Error (node 1): L3 data cache ECC error.
    [2954432.994525] [Hardware Error]: cache level: L3/GEN, tx: GEN, mem-tx: EV
    [2954432.998450] mce: [Hardware Error]: Machine check events logged

Who knows what was going at the time...

Syndicated 2016-02-24 02:08:18 from I Dream of Rain (free_software)

Sinfonia CHC-S6145 (CS2) and Ciaat Brava 21, working!

Over the past year, I've written a bit about the situation involving the Sinfonia CHC-S6145 printer and its rebadged sibling, the Ciaat Brava 21. To summarize, the printers worked but required use of a proprietary, binary-only library ('libS6145ImageProcess') to perform thermal compensation and other transformations to the image data in order to generate sane output.

To make a long story short, I set out to reverse-engineer how that library worked... and a couple of weeks ago, I succeeded, with my reimplemented library generating completely identical results.

After some back and forth with Sinfonia, I'm quite pleased to announce that my re-implmented library, called 'libS6145ImageReProcess', is now released to the public under a GPLv3+ license. Except for the differing name, it is a drop-in replacement for the Sinfonia library.

Just to be absolutely clear, Sinfonia is not responsible for this library in any way, and will not support you if you complain that the output is somehow deficient or your printer catches fire when you print images of Donald Trump biting the heads off of adorable kittens.

Now in order to actually utilize these printers, you'll need to compile and install three components:

  • Gutenprint 5.2.11 (just released!)
  • libS6145ImageReProcess library
  • Latest selphy_print backend code

I should have the necessary backend code in the Gutenprint development repo soon, but due to licensing complications the library will probably remain separately distributed.

Particular thnaks go to Sinfonia and Ciaat for providing documentation on the printer communication protocols, and Matt Koglin for his SinfoniaCam(tm) and many, many rounds of testing.

This has been a long time coming, and is the culmination of quite a bit of work. I hope it proves useful, and if you do purchase one of these printers intending to use it with Linux (or a more obscure OS), please let your Sinfonia distributor know. :)

Syndicated 2016-01-30 20:24:40 from I Dream of Rain (free_software)

Lifting the skirts of Kodak photo printers

You have to hand it to Kodak. They have been selling their workhorse 6850 dyesub photo printer for more than ten years, and are still actively supporting it with updated drivers and firmware. It's even outlasted one of its successors (the 605), which is no longer sold, yet is still actively supported.

One of those firmware updates led to a discovery that resulted in a flurry of hacking on the 6800/6850 and 605 backends, resulting in considerably improved reliability, robustness, and performance. As well as many bug fixes, both backends now support full job pipelining and vastly improved status and error handling.

So what had I learned? The Kodak 68x0 family are variations of the Shinko S1145 and the Kodak 605 is actually a Shinko S1545. Digging deeper into other Kodak models, I discovered that the 7000/7010/7015 are variations of the Shinko S1645, and that the 8810 is a Shinko S1845.

I'd done my earlier reverse-engineering work on these Kodak models before some kind folks at Shinko/Sinfonia sent me documentation on several of their printers -- So when I re-examined what I had previously figured out with the other docs as a reference, I discovered that from a protocol perspective the 68x0/S1145 models were 6" variations of the 8" S1245, the 605/S1545 and 70xx/S6145 models were very close to the S2145, and the 8810/1845 are apparently identical to the S6245.

This means that I should be able to support the 70xx and 8810 printers with only minor modifications to the existing backend code. Granted, until I can get my hands on any of these printers all of this is conjecture.

So, I'll re-iterate my call for testers for these printers:

  • Shinko CHC-S1245 (aka Sinfonia E1)
  • Shinko CHC-S6245 (aka Sinfonia CE1)
  • Kodak 8810
  • Kodak 7000/7010/7015
  • Kodak 605 (Need to ensure no regressions were introduced)

As my personal printing needs are very well met at this point and these are all fairly expensive models (especially the 8810 and 70xx series), I can't justify buying more printers just to try and make them work with Linux. Someone else is going to have to step up to help make this possible.

On that note, I should mention the S6145/CS2 (and the Ciaat Brava 21), where the situation is a bit more complex. The backend is already written and partially tested, but it currently relies on a proprietary library that is only available in binary form - and which I lack permission to redistribute.

I'm pursuing a multi-prong approach to rectify that situation. In order of desireability:

  • Obtain source code to the library
  • Obtain algorithmic documentation so I can independently re-implement the library
  • Obtain permissions to redistribute the (binary) library, and also get it compiled for a variety of ARM targets
  • Reverse-engineer the library so I can re-implement it

Let me just say that curiousity, in of itself, is poor motivation for enduring the the combination of tedium and frustration that comes from trying to reverse-engineer an opaque blob of x86 code.

Ugh. I need to get out more.

Syndicated 2015-09-01 21:40:31 from Solomon Peachy

164 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!