Recent blog entries for AlanHorkan

OpenRaster with JPEG and SVG

OpenRaster is a file format for layered images, essentially each layer is a PNG file, there is some XML glue and it is all contained in a Zip file.

In addition to PNG some programs allow layers in other formats. MyPaint is able to import JPG and SVG layers. Drawpile has also added SVG import.

After a small change to the OpenRaster plugin for The GNU Image Manipulation Program, it will also allow non-PNG layers. The code had to be changed in any case, it needed to at least give a warning that non-PNG layers were not being loaded, instead of quietly dropping them. Allowing other layer types was more useful and easier too.
(This change only means that other file types with be imported, they will not be passed through and will be stored as PNG when the file is exported.)

Syndicated 2015-04-12 18:08:16 from Alan Horkan

OpenRaster Paths (or Vectors)

Summary: plugin updated to allow round-trip of paths.

The MyPaint team are doing great work, making progress towards MyPaint 1.2, I encourage you to give it a try, build it from source or check out the nightly builds. (Recent windows build Note: the filename mypaint-1.1.1a.7z may stay the same but the date of build does change.)
The Vector Layers feature in MyPaint is particularly interesting. One downside though is that the resulting OpenRaster files with vector layers are incompatible with most existing programs. MyPaint 1.0 was one of the few programs that managed to open the file at all, presenting an error message only for the layer it was not able to import. The other programs I tested, failed to import the file at all. It would be great if OpenRaster could be extended to include vector layers and more features but it will take some careful thought and planning.

It can be challenging enough to create a new and useful feature, planning ahead or trying to keep backwards compatibility makes matters even more complicated. With that in mind I wanted to add some support for vectors to the OpenRaster plugin. Similar to my previous work to round-trip metadata in OpenRaster I found a way to round-trip Paths/Vectors that is "good enough" and that I hope will benefit users. The GNU Image Manipulation Program already allows paths to be exported in Scalable Vector Graphics (SVG) format. All paths are exported to a single file, paths.svg and are imported back from that same file. It is not ideal, but it is simple and it works.

Users can get the updated plugin immediately from the OpenRaster plugin gitorious project page. There is lots more that could be done behind the scenes, but for ordinary users I do expect any changes as noticeable as these for a while.


Back to the code. I considered (and implemented) a more complicated approach that included changes to stack.xml, where raster layers were stored as one group, and
paths (vectors layers) as another group. This approach was better for exporting information that was compatible with MyPaint but as previously mentioned, the files were not compatible with any other existing programs.

To ensure OpenRaster files that are back compatibility it might be better to always include a PNG file as the source for every layer, and to find another way to link to other types of content, such as text or vectors, or at some distant point in the future even video. A more complicated fallback system might be useful in the long run. For example the EPUB format reuses the Open Packaging Framework (OPF) standard, any pages can be stored in multiple formats, so long as it includes a fallback to another format, ending with a fallback to a few standard baseline formats (i.e. XHTML). The OpenRaster standard has an elegant simplicity, but there is so much more it could do.

Syndicated 2015-04-06 22:00:30 from Alan Horkan

OpenRaster Metadata

Summary: plugin updated to allow round-trip of metadata.

OpenRaster does not yet make any suggestions on how to store metadata. My preference is for OpenRaster to continue to borrow from OpenDocument and use the same format meta.xml file, but that can be complicated. Rather than taking the time to write a whole lot of code and waiting do metadata the best way, I found another way that is good enough, and expedient. I think ordinary users will find it useful -- which is the most important thing -- to be able to round-trip metadata in the OpenRaster format, so despite my reservations about creating code that might discourage developers (myself included) from doing things a better way in future I am choosing the easy option. (In my previous post I mentioned my concern about maintainability, this is what I was alluding to.)

A lot of work has been done over the years to make the The GNU Image Manupilation Program (GIMP) work with existing standards. One of those standards is XMP, the eXtensible Metadata Platform originally created by Adobe Systems, which used the existing Dublin Core metadata standard to create XML packets that can be inserted inside (or alongside) an image file. The existing code creates an XMP packet, let's call it packet.xmp and include it in the OpenRaster file. There's a little more code to the load the information back in and users should be able to go to menu File, Properties and in Properties dialog go to the tab labelled Advanced to view (or set) metadata.

This approach may not be particularly useful to users who want to get their information out into other applications such as MyPaint or Krita (or Drawpile or Lazpaint) but it at least allows them not to lose metadata information when they use OpenRaster. (In the long run other programs will probably want to implement code to read XMP anyway, so I think this is a reasonable compromise, even though I want OpenRaster to stay close to OpenDocument and benefit from being part of that very large community.)

You can get the updated plugin immediately from the OpenRaster plugin gitorious project page.

If you are a developer and want to modify or reuse the code, it is published under the ISC License.

Syndicated 2015-04-06 20:36:09 from Alan Horkan

OpenRaster Python Plugin

Thanks to developers Martin Renold and Jon Nordby who generously agreed to relicense the OpenRaster plugin under the Internet Software Consortium (ISC) license (it is a permissive license, it is the license preferred by the OpenBSD project, and also the license used by brushlib from MyPaint). Hopefully other applications will be encouraged to take another look at implementing OpenRaster.

The code has been tidied to conform to the PEP8 style guide, with only 4 warnings remaining, and they are all concerning long lines of more than 80 characters (E501).

The OpenRaster files are also far tidier. For some bizarre reason the Python developers choose to make things ugly by default, and neglected to include any line breaks in the XML. Thanks to Fredrik Lundh and Effbot.org for the very helpful pretty-printing code. The code has also been changed so that many optional tags are included if and only if they are needed, so if you ever do need to read the raw XML it should be a lot easier.

There isn't much for normal users unfortunately. The currently selected layer is marked to the OpenRaster file, and also if a layer is edit locked. If you are sending files to MyPaint it will correctly select the active layer, and recognize which layers were locked. (No import back yet though.) Unfortunately edit locking (or "Lock pixels") does require version 2.8 so if there is anyone out there stuck on version 2.6 or earlier I'd be interested to learn more and I will try to adjust the code if I get any feedback.
I've a few other changes that are almost ready but I'm concerned about compatibility and maintainability so I'm going to take a bit more time before releasing those changes.

The latest code is available from the OpenRaster plugin gitorious project page.

Syndicated 2015-03-23 18:35:16 from Alan Horkan

Krita 2.9

Congratulations to Krita on releasing version 2.9 and a very positive write-up for Krita by Bruce Byfield writing for Linux Pro Magazine.

I'm amused by his comment comparing Krita to "the cockpit of a fighter jet" and although there are some things I'd like to see done differently* I think Krita is remarkably clear for a program as complex as it is and does a good job of balancing depth and breadth. (* As just one example: I'm never going to use "File, Mail..." so it's just there waiting for me to hit it accidentally, but far as I know I cannot disable or hide it.)

Unfortunately Byfield writes about Krita "versus" other software. I do not accept that premise. Different software does different things, users can mix and match (and if they can't that is a different and bigger problem). Krita is another weapon in the arsenal. Enjoy Krita 2.9.

Syndicated 2015-02-25 23:42:50 from Alan Horkan

OpenRaster Python Plugin

OpenRaster Python Plugin

Early in 2014, version 0.0.2 of the OpenRaster specification added a requirement that each file should include a full size pre-rendered image (mergedimage.png) so that other programs could more easily view OpenRaster files. [Developers: if your program can open a zip file and show a PNG you could add support for viewing OpenRaster files.*]

The GNU Image Manipulation Program includes a python plugin for OpenRaster support, but it did not yet include mergedimage.png so I made the changes myself. You do not need to wait for the next release, or for your distribution to eventually package that release you can benefit from this change immediately. If you are using the GNU Image Manipulation Program version 2.6 you will need to make sure you have support for python plugins included in your version (if you are using Windows you wont), and if you are using version 2.8 it should already be included. (If the link no longer works, see instead https://gitorious.org/openraster/gimp-plugin-file-ora/ as I hope the change will be merged there soon.)

It was only a small change but working with Python and not having to wait for code to compile make it so much easier.

* Although it would probably be best if viewer support was added at the toolkit level, so that many applications could benefit.

Syndicated 2015-02-18 19:14:34 from Alan Horkan

OpenRaster and OpenDocument: Metadata

OpenRaster is a file format for the exchange of layered images, and is loosely based on the OpenDocument standard. I previously wrote about how a little extra XML can make a file that is both OpenRaster and OpenDocument compatible. The OpenRaster specification is small and relatively simple, but it does not do everything, so what happens if a developer wants to do something not covered by the standard? What if you want to include metadata?

How about doing it the same way as OpenDocument, it does not have to be complicated. OpenDocument already cleverly reused the existing Dublin Core (dc) standard for metadata, and includes a file called meta.xml in the zip container. A good idea worth borrowing, a simplified example file follows:

Sample OpenDocument Metadata[Pastebin]

(if you can't see the XML here directly, see the link to Pastebin instead.)

I extended the OpenRaster code in Pinta to support metadata in this way. This is the easy part, it gets more complicated if you want to do more than import and export within the same program. As before the resulting file can renamed from .ora to .odg and be opened using OpenOffice* allowing you to view the image and the metadata too. The code is Pinta OraFormat.cs is freely available on GitHub under the same license (MIT X11) as Pinta. The relevant sections of are "ReadMeta" and "GetMeta". A Properties dialog and other code was also added, and I've edited a screenshot of Pinta to show both the menu and the dialog at the same time:

[* OpenOffice 3 is quite generous, and opens the file without complaint. LibreOffice 4 is far less forgiving and gives an error unless I specifically choose "ODF Drawing (.odg)" as the file type in the Open dialog]

Syndicated 2015-02-13 00:38:44 from Alan Horkan

OpenRaster and OpenDocument

OpenRaster is a file format for layered images. The OpenRaster specification is small and relatively easy to understand, essentially each layer is represented by a PNG image, and other information is contained written in XML and it is all contained in a Zip Archive. OpenRaster is inspired by OpenDocument.
OpenDocument is a group of different file formats, including word processing, spreadsheets, and vector drawings. The specification is huge and continues to grow. It cleverly reuses many existing standards, avoiding repeating old mistakes, and building on existing knowledge.

OpenRaster can and should reuse more from OpenDocument.



It is easy to say but putting it into practice is harder. OpenDocument is a huge standard so where to begin? I am not even talking about the OpenDocument Graphics (.odg) specifically but more generally than that. It is best that show it with an example. So I created an example OpenRaster image with some fractal designs. You can unzip this file and see that like a standard OpenRaster file it contains:


fractal.ora  
 ├ mimetype
 ├ stack.xml
 ├ data/
 │  ├ layer0.png
 │  ├ layer1.png
 │  ├ layer2.png
 │  ├ layer3.png
 │  ├ layer4.png
 │  └ layer5.png
 ├ Thumbnails/
 │  └ thumbnail.png
 └ mergedimage.png

It also unusually contains two other files manifest.xml content.xml. Despite the fact that OpenDocument is a huge standard the minimum requirements for a valid OpenDocument file comes down to just a few files. The manifest is a list of all the files contained in the archive, and content.xml is the main body of the file, and does some of the things that stack.xml does in OpenRaster (for the purposes of this example, it does many other things too). The result of these two extra files, a few kilobytes of extra XML, is that the image is both OpenRaster AND OpenDocument "compatible" too. Admittedly it is an extremely small tiny subset of OpenDocument but it allows a small intersection between the two formats. You can test it for yourself, rename the file from .ora .odg and LibreOffice can open the image.

To better demonstrate the point, I wanted to "show it with code!" I decided to modify Pinta (a Paint program written in GTK and C#) and my changes are on GitHub. The relevant file is Pinta/Pinta.Core/ImageFormats/OraFormat.cs which is the OpenRaster importer and exporter.

This is a proof of concept, it is limited and not useful to ordinary users. The point is only to show that OpenRaster could borrow more from OpenDocument. It is a small bit of compatibility that is not important by itself but being part of the larger group could be useful.

Syndicated 2014-12-12 04:03:51 from Alan Horkan

Developing MComix. Part 1: A rose by any other name...

Comix is a comic and image viewer program written using Python and GTK. I was interested in making a few changes to the project. I am getting back into programming, learning a more about PyGTK and hopefully making some improvements the program will fit better on a small screen netbook. Unfortunately Comix is unmaintained, the developer of the project seems to be unavailable and has not made any updates since 2009. (You can discover this for yourself by checking the Comix website particularly by checking the Sourceforge page for Comix and taking a look at the last date on the Changelog, and various reports in the bug tracker. It may still be possible to get in contact with him but the project is not active.)

I was pleased to discover MComix a fork of Comix, continuing on from Comix version 4.0.4. It isn't clear what the M in MComix stands for but from the picture in the about dialog I think "Monkey Comix" is a fairly safe guess.
(Later I also discovered another fork of Comix by HellHover which incorporates many of the patches submitted to Comix. It is more like a spork than a fork, as it keeps close to the original.)

The attitude and prompt reponse from the MComix developers was encouraging so I got to work putting together a few changes and sending in a patch (after a discussion clarifying that MComix is licensed under the GNU GPL version 2).
Freedom to change the source code and even fork the project is a great power to have but it comes with responsibility. Getting others to help is not easy, you want people to submit code, help with translations, or get a project packaged nicely for different operating systems, and generally help with the work of maintaining a project, so even though forking is possible it is not something you want to do unless absolutely necessary. Knowing it would take some time and effort to make changes it is a great relief to know there is a good chance others will accept the change help maintain it. Putting a lot of work into a patch and having it go to waste, is a big disappointment and severly discourages anyone from contributing to a project.

The patches took less time to write and test than the email explaining the rationale for the changes. Developers often like to do things their own way, and a without a proper explanation, patches might not be accepted. As a fork of Comix I was optimistic the developers would be more accepting of change but I wasn't going to take any chances.

The first - and to my mind most important - change was relatively simple but something I would strongly recommend to any program, especially a fork: separate out the name of the program. I set a constant and called it "APPNAME" and replaced the word MComix, so that if the name ever needs to be changed again or even changed back to Comix it can be done with only a few small changes.
The long and winding history of Mosiac, Netscape, Mozilla, m/b, Phoenix, Firebird, Firefox, Iceweasel, is a particularly extreme example of the many name changes a project can go through. If someone doesn't like the name the can easily change it without needing to fork the project. Making it a little bit easier for others to customize and fork the code gives them one less reason why they would need to, and they can continue to pool their efforts on the things they do agree on.

The patch also contained changes to help with internationalisation (i18n) and localisation (l10n).
There was also a small change to the command line arguments.

When the patch was accepted I was very satisfied and confident the developers were the kind of people I would enjoy working with further. Answers to a few other questions in my email confirmed our interests were different but mutually beneficial. Straight away I was thinking about the next changes I might make.

Syndicated 2011-04-23 23:49:11 from Alan Horkan

Peazip: Convert CBR to ZIP

I had a tshirt with the slogan "More people have read this shirt than have read your blog" and I am always a little surprised when someone does read my writings on the web. I was pleased to see that someone thought my batch file to repack from CBR to CBZ might be useful for people who want to read comics on mobile devices.

I was thinking about rewriting the batch file to work with programs other than 7-zip and I rediscovered Peazip a program I had not tried in years. The user interface is still a bit rough in places but it includes a very useful tool for Convert archives from different formats. It is a convenient way to convert files from CBR to ZIP which I think some users will prefer to using a DOS batch file. There does not seem any way to automatically rename the ZIP files to CBZ, so for I prefer to keep using the batch file. (The Convert tool in Peazip will unpack all the files and then repack all files, which can take up a lot of disk space with temporary files, so be careful if you try to convert many files at once.)

The biggest advantage of the batch file is that it does more than just repack from CBR to CBZ. I continue to modify it in other ways to better clean up the Comic files as I repack them, such as removing signature or banner images added by scanners. It is tricky to come up with a pattern that works all the time so I only included a simple example before, but the following patterns catch a lot of the tags and banner images:

MOVE z*.* ..\  
MOVE xx*.* ..\  
MOVE *tag*.* ..\  
MOVE *team*.* ..\  

The banner images can take up quite a bit of space, more than 1MB in rare cases, so removing them often saves more space than anything else such as special compression tricks. Instead of adding a large image file the batch file could be changed to add a small text file containing comments by adding the following command before recompressing:
echo "Repacked by 7-zip" > Comment.txt 

Tidying up file names is another thing that could be added to the batch file.

As I said earlier I am glad at someone found the batch script useful, hopefully Peazip will be useful too.

Syndicated 2011-04-06 00:52:03 from Alan Horkan

364 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!