The Wayback Machine - https://web.archive.org/web/20170630143304/http://www.advogato.org/person/Chicago/diary.html?start=81

Older blog entries for Chicago (starting at number 81)

.Net & Browsercaps
I've been working really hard on our new product (spam alert Momo-it.com) but we're having problems with mobile devices and getting the best possible rendering.

Now to explain the situation first - we don't use the .Net controls to generate the page, instead we use the browser capabilities of .Net to work out if html, wml or xhtml should be rendered (where I am specifying xhtml as designed for mobile devices).

The problem appears where we have a mobile device which is slightly more advanced (PDA's like the XDA or PSP etc). Here we get conflicting information. We are told that either they can handle full websites (And therefore we are sending them a html website) or we are told that they are only capable of rendering WML.

Now we want to give the consumer the best user experience possible, so we have to overload the browsercaps section of our configuration but the problem is there dosn't seem to be a suitable existing one that we can grab to use, which is a real pain - we will have to write a tool to enable customer support to enter individual phones as being xhtml compatible and then use that to populate the servers with an updated browsercaps file.

Thats not to say that there are several people out there who have very good browsercaps files - they just all seem to be based on 'normal' browsers not mobile devices.

What the hell - so I get my head down to do some serious work and come back to this stuff!!! Advogato disbanding. PAh!. Well I'm going to continue on as normal. (PS, I support the StevenRainwater takeover bids as an alternative to hosting, and offer some limited financial / hosting support).

Right. So I've been working with FFMPEG and its conversion. The first thing thats weird is that it seems to lack certain elements (in Windows) that it has in Linux, causing a couple of weird bugs (that and the misuse of the MJPEG causing some... big images).

To recap, we're taking 3GPP videos from mobiles and re-coding them for the web. Now, very cool, but we're having a lot of problems with the audio - whilst some of the clips (taken at gigs right infront of 10 mega watt sound systems) actually have good sound on them, however others, taken in front of normal talking people, have terrible sound levels.

The only solution (As far as we can tell) will be to strip the audio from the 3GPP file, to perform some audio jiggery pokery (techincal term) and then return that audio to the original movie and hope everything is still in synch.

That's after the show next tuesday of course.

21 Aug 2006 (updated 22 Aug 2006 at 17:21 UTC) »

I don't really ever get into hyper angst about code that works - I mean I get annoyed and rewrite stuff, but usually after a few months of having to work with it and work around issues that have come due to shifting goal posts. But this thing I wrote for Kaz, well, its really got up my nose, and here is why:

Classes should have a single job

The main class was supposed to be a line on the graph (I will have to rant about issues of drawing graphs at a later point in time), but that's what it is, so it is a collection of coordinates and associated line data (such as title, colour and origin information). On top of this there are associated class functions such as loading, saving, normalization and peak detection (plus any number of extra analysis tools). Around this there is a wrapper class to hold a collection of these objects, which also contains functional elements to be able to call all of the sub elements in turn, such as finding peaks on all the lines. On top of this there may also be the interface to the application.

My mistake was to make all of the above into a single class. What this means is that firstly, from the outside there isn't a single public function (there should be). There isn't because the object works in either one of two modes - as the collection of lines object or as the single line object, not as both at once (so its not recursive). Because of this, the single line object has to produce public methods to the collection object so that it can do its job. But because the collections object is actually the same one that means its public all the way to the outside. That's not right.

Data Structures should not be misused

There are some sub classes, including a coordinate object. The coordinate object is most basically two double values - something like

class coordinate {
	public double X;
	public double Y;
}

So, there's nothing wrong with that (there are also associated methods with this object which is why it's an object not a struct). However, in the bit of the code which has to return a pair of double's (to be precise, the part which looks for maximum and minimum values of X) it is not appropriate to return a coordinate where c.x is the minimum and c.y is the maximum. Either a new structure should have been made, or a hash table, or another associative array

Well Named Variables decrease risk of Bug

My last point is one that I'm still working on but doesn't actually affect the Chemistry App for Kaz (see point about me being good below) - its one where good choice of variable name's can decrease the probability of error.

It happened to our junior programmer at work this week - or rather he wrote it and I fell for the trap. He was doing some image manipulation and had some sub images that he was positioning. To do so he was dealing with the height of the sub image. However, the local variable height was not the height of the sub image - it was some random number that he had assigned further up the script and was still in scope. What got me was that a) the variable height was in scope and when I came to write the next line, it was the most obvious match from the intellisense, and b) the actual height variable was much more complicated in name. What it resulted in was a bug in code which (if we weren't looking that carefully at each line of code in turn, but writing many lines before test) could have ended in a bug which may have been near impossible to locate.

So I've been making a special effort this past week to use proper variables in code. Isn't that good of me?

Weirdness to investigate this week

C#: Threads continue execution after main() exits, and those threads can spawn new threads, but any events associated from main will not execute after main() has died (or so it seems...)

Good news I just passed the RSGB Foundation test. Woo.

Apenwarr writes about making things that will last forever. Forever is a *long* time.

Recently (ok not so recently) I attended a conference by eeegr on decommissioning oil platforms and nuclear power stations. One of the key things that I came away from this was the failure of modern (I mean seventies onwards) companies to keep information. The point being that these companies don't know basic information about the rigs - how many concrete weights there are at the bottom, what chemicals have been dumped. The reason they don't have this are mainly due to the cost of keeping the paperwork.

But they're all electronic now aren't they? A copy electronically and it takes up a lot less space - well, that would be nice now wouldn't it, so why then in practice isn't that happening? Do we all remember the Edinburgh AI Library fire. Sure, the current research (on backup) was saved but why wasn't this information on disk - especially as it's about electronic stuff, why wasn't it stored electronically?

The idea of software rot isn't just caused by technical improvements of the hardware its running on, but also on changing goal posts and adaptation of tools. Suppose a threaded application that used third party library works fine and does not create a race condition, on the changing of this library (for security or otherwise) where the running code changes but the API does not, the call might take longer - perhaps even long enough to suddenly cause race conditions.

And as for pure mathematics - well, how about something as fundamental (and supposedly never changing) as the ratio of the circumference to its radius, and yet over the past few thousand years it did changed. Now, I don't think (and I don't think anyone else does) that the laws of physics have changed, they just didn't have the ability to work it out any better. Now, does that mean that some of the maths that we are doing to day (some of the ground breaking new maths) is also... shaky?

So I'm writing in C# and I have a security problem - uploading of virii to my system, so I need to scan. I was hoping to use ClamWin and its libclamav.dll to scan the buffer, but I've met the following interesting quirks:

Yes, I know I could save the file to the disk and call clam as a process and look at its return value, but that's not very elegant is it? I also know that on the FAQ it says

Q: Does ClamWin scan BLOBs (Binary Large OBjects) in databases?

A: No, the only way to do this is export the object to a file and scan the exported file. Ideally, the objects should be scanned before they are entered into the database.

Erm ok - but in linux clamscan has the ability to scan from stdin - something like:

Echo ``something to be scanned'' | clamscan -someoptiontoscanforstdin

Is that not possible with clamwin? (Point 1)

The second thing is I can view the libclamav.dll file and see what functions are available in there, but I have no idea as to what order they need to be called in - at a guess (and looking at *some* documentation) I think I can get away with something similar to this:

using System;
using System.Runtime.InteropServices;
namespace ClamScanTest
{
    class Program
    {
        [DllImport("libclamav.dll")]
        static extern string cl_scanfile(string filename);
        
        public static void Main()
        {
            Console.WriteLine(cl_scanfile("test"));
            Console.ReadKey();
        }
    }
}

Apart from the obvious (if you where to run that) fact that it doesn't work - I get the error string:

The runtime has encountered a fatal error. The address of the error was at 0x79ef281a, on thread 0x8fc. The error code is 0xc0000005.

Now this may be related to point 1 - the nux version seems to have the function scanbuf(), although my dll inspector doesn't show that in windows. Presumably this is a problem with something to do with the (fundamental) differences between Windows and Posix file handling routines.

What should I try next? Is there an easier way of doing this (maybe not using clamscan)? Maybe I should learn more about running unmanaged dll's in .Net before trying something this complicated? Perhaps, but any aid would be appreciated.

Debugging & Logging I just fell out of the zone, which means that I'm going to update my journal. Firstly, I'ld like to point people at the Apache Logging group. They have some logging tools which are actually really well put together - I'm now using their Log4Net in my code and its really saved a lot of effort on my part to provide a suitable logging service.

The only strange thing is the lack of error logging on its constructor - I had a massive problem in getting it to find my XML file (partially because I forgot that I wasn't compiling into and running from the same directory that I was programming in). However providing it with a dodgy path to a non-existent file didn't throw any errors - this would mean that the logging would never be able to interrupt the execution of the program. I will have to look into how to detect if the logging has managed to succeed or not (at least on start-up).

Its ability to have a watch on the configuration file is also quite nice, and it doesn't seem to have that much impact on performance when you start using (and ignoring) its debug() functionality. Its main power however is the ability to use the logger in one environment with one configuration, but then (for instance when using the code in a program as opposed to the test harness) having the logging go to different places. The ability to control the end-location of the logs to different streams, emails or files is just awesome. I mean I am really, really impressed with this software.

And its not too annoying in the code either, which means I'm even happier - at the top of each of my classes I just put

private ILog myLog = LogManager.GetLogger(typeof(currentclass));

then I just put

mylog.debug("Some Debug Stuff");
or whatever I want inline, replacing all my Console.WriteLines(). Its great. Go use it!

Human Interfaces

Prozac: I use a tiny mouse - I mean a really really tiny mouse. This gives some idea as to how small my mouse is to my hand (also shown). I hold it in my fingers, and I actually have no problem with the straight down menu styles, because my fingers just move when my hand doesn't.

Plus I generally click as little as possible. I'm annoyed that I haven't found a suitable way in windows to have the sloppy focus of fluxbox that I love and enjoy - just to cast my mouse to a location.

But then again I also have a good memory - or rather not necessarily memory but my hands know the shortcuts. I mean they know the keys off by heart, (I have a das keyboard - did I mention that in here yet? I aught to have - its very cool).

7 Jul 2006 (updated 7 Jul 2006 at 12:26 UTC) »
Doppleganger: I can't work out if you are a bot or a real person. Where are your links to your `posting'. Are you actually some kind of megahal bot that's been fed a data structures and algorithm textbook and just preparing an account that can be later used for link spamming?

Edit Thanks bi. That looks very much like an implementation of a Trie. As such, it would be very efficient, (in searching speed) but I fear the amount of memory used up would be *fearsome* and creating it in the first place may also be very slow.

Stuck with the unending task of applying resources to fit deadlines of a project, what can you do other then beg for more resources (which will invariably be left with a negative response) or drop features from the project? Whilst some more resources have been given (two guys are starting within the next couple of weeks) one of the deadlines has been hastily moved forwards by the combined efforts of demands from customers, the result, a hastily re-drawn map of features and requriements diagrams alowing us to cancel features whilst still leaving us with enough product to move to the beta demoing stage.

The only thing to do now is move from the features based diagrams to the implementation diagrams. These cause me headaches - even though I know that no one diagram can succesfully describe the methods of displaying the code that will be written - class diagrams get horrendiously complicated when you start using more and more generic systems, and sequence diagrams begin to get horrendious when you have one per 'use' of the use diagrams. With seventy user features on the project, that leading to seventy diagrams is overkill, and it would be easier just to write the thing and write how it was written afterwards - but that would be bad now.

15 Jun 2006 (updated 15 Jun 2006 at 12:45 UTC) »

I am unsure what to think about Unified Modeling Language. The sort of size projects we are about to embark on would be difficult to maintain due to the nature of the program. Our new product, which the sales and upper management are in danger of followign the Dilber t 'Name' syndrome. My main questions about UML are will we (the technical team) be given enough freedom to be able to have good software development process whilst still being able to be flexible enough to meet demands of prototypes and last minute deal-clinching feature requests - this is more a case of will the tools that are available to us be easy to use and actually aid, or will they hinder application development? I don't want to have to go down any route which will create barriers to developers enthusiasm for developing cool new products.

Then, there is of course the inevitable battle between the technical and design sides when it comes to good template based design. Perhaps I am being too harsh in expecting the CSS based HTML templates to look the same in both IE and other alternative browsers. I also know that in this area, this is paramount to preaching to the converted, and that I know that you shouldn't back down by saying at the bottom of the website best viewed in - I know of many users who do not have the ability to use IE - not just linux computer geeks, but Apple Mac users (for whom, the IE has not been available for a few years now (see the Microsoft for Apple downloads (although using virtual PC for Mac might enable that use) which include several non-geek friends of mine, of varying ages (including a retired lady who is very good with her Mac) who have no desire to perform complicated manipulations of their operating systems to view a single web page - they would just go elsewhere to a site which does view correctly, as well as certain companies which use Firefox as their main web browsing application by default (whether or not the employees want or like to, or even know.)

Edit: Issues with RSS feed of this post. The rss feed of this post (here) has some issues with it. Primarily, only the Dilbert link shows correctly - the other links all seem to link to the rss feed. The only difference I can tell is that the Dilbert link uses double quote marks, where the others use single. I will change them to double to see if there is any resolution in this. Looking at the RSS feed, it seems to produce:

<a href="" 'http://www.mozilla.com/firefox/'>
from

<a href='http://www.mozilla.com/firefox/'>

Is there any reason for that?

Interviews

At work start tommorrow. My techincal test has been written, and this time dosn't include the difficult database question that caused the thirty minute stumping of so many previous people. I am considering writing a simpler database question, but what would be the point? They have to be challanging questions...

My current questions now consists of:

  • Set Theory This consists of a set of simple maths sets, and then performs different combinations on them, such as AND, NOT, OR and XOR's expressed in different ways. The idea is to see if they can handle sets of data, which transposes to their ability to understand things like sets of customers or indeed more abstract things like problems (where symptoms may be in various sets)</em>
  • Web Programming This aims to see if they understand the principles behind reasonable web programming, security issues, interface issues and the technology behind simple request - response - request - response style programming.
  • OO Programming Possibly my favorite question, which aims to see if they understand references properly.

    Elvaston

    Was great fun - official webpage here. But it was not as good as previous years. I last visited three years ago, and there where at least two more large tents. Everything seemed scaled down - even the Icom tent, which last time bristled with cool equipment was now a shared tent with Kenwood - each having perhaps only two tables of their gear.

    The most interesting thing there has to be the WiNRADiO which is a receiver that does its signal processing on the computer. The basic premisis being that it provides a much better control of the listening range then any of the hardware solutions currently out.

    This product, and several others on display all seemed to point at a trend of moving back to having dual sets - one to listen and one to transmit, which of course leads to dangers of having them set on different frequencies, it does mean that you can have specialist gear for each one.

    The WiNRADiO was seductive though - its got a really nice USB interface to a nice plastic shielded case. It's specs look great, and its has 'alternative' interfaces for selecting listening frequencies. One of these (as well as the tradional dial) was a graph of the entire frequency range, with a selection of the frequencies that are being amplified. The user can spot what frequencies are in use by seeing where the spikes are, and then just drags their selection over the top.

    Seductive yes. Expensive yes - remember, this is only the receiver. With prices at £400 (£450 GBP for the better demodulator) you need to have a (Windows) PC allready (which lets face it, I do). But then if you want to transceive you also need to have another peice of gear to transmit from...

    However if you wanted to do something like.. ooh I dont know, Amature Radio Astronomy, it might provide an exellent base to start from.

  • 72 older entries...

    New Advogato Features

    New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

    Keep up with the latest Advogato features by reading the Advogato status blog.

    If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!