Older blog entries for salmoni (starting at number 571)

I just wasted most of a day trying to sort out the data import GUI and problems with sizers. It was quite frustrating, but I managed to get most of the problems sorted out finally. It is now connecting to various databases and showing a sample of data which users can browse and select what they want to import from.

Oh, and it imports the variables too which is good. It is so nice when problems eventually finish. I have lots more work to do tomorrow (csv importing - I wrote my own csv module to deal with little problems like missing data in the middle of a row) but I am also going to my wife's family's village for a fiesta. It's been raining all day, so here's hoping the weather improves. Here's a picture of the village in sunnier times.

Either way, the work is coming along really nicely now. The project is not yet 50% finished (my estimate), but it already imports data from databases, allows a range of operations on them, and can produce even complex descriptive analysis. It's looking good so far.

I've been busy playing with Python 3K at home. It seems to be nice though I haven't dug deep enough / far enough to notice real changes outside of the 'print' statement changes.

In other work, I'm managing to tame wxPython again and am producing a consistent and simple interface for importing data from different sources (databases, spreadsheets, text files). It could form the basis of a data manager, but it's all for the statistics program which is itself coming along.

The program has an interactive interpreter which is fun: it's all based on Python's 'code' module and I've organised it so that users can import data with awkward field names (like: 'Variable (1) & Variable (2) mixed'), and they can still be used on the command line, thus:

Variable (1) & Variable (2) mixed.mean

Not a big change, but it's one less thing to explain to demanding users. The work on the main GUI is still ongoing (choosing a test is the hardest thing) but we're getting there.

The thing will be released under the Affero GPL license so it's even relevant to this site.

In administration things, I managed to get some more marketing research done (all promising but lots of things to think about), and the company is getting closer to being officially founded. It's all very exciting stuff.

I have a questionnaire here if anyone feels like completing it: it should take about 5-10 minutes and concerns people who use computers to perform statistical analysis. I cannot offer any money in return (we have zero investment - any offers will be carefully considered!), but it would be extremely helpful in getting open source to the top.

The questionnaire is at Survey Monkey. TIA to anyone who completes it.

Phew, am tired. It's hard working in these temperatures, but nicer than freezing my bits off. We will be getting air-con in the room soon in preparation for the baby's arrival but right now it's hot!

The stats program is coming on very well thank you! It now handles different types of data with ease and total transparency for the user. The output is nicely formatted and looks good. The architecture seems to be about right and we're looking to release the first version (obviously a beta) maybe next week. It will be under the AGPL because we intend to put in networking capabilities. Complex data analysis through a web-browser? Heh, why not?

Uraeus - I've tried noise-cancelling headphones on long flights and found them to be quite good. If they're the "tighter" ones (ie, the ones that fit tight around the ears) they can also help preventing ears from popping.

So I guess this is the closest thing to an official announcement that I can make.

My business partner and I are going to form a company which will concentrate upon statistics software. Our product will be called Ecstatistics which is a seriously good update of my old project SalStat. It differs in that:

a) It will be able to read data from CSV files, databases (a whole range), and spreadsheets. We plan to import SPSS and SAS files too as well as any other format we can code for;

b) It will output to a range of formats (PDF, OOo, databases, MS Office, HTML). The HTML is interesting because it will allow online analysis;

c) It will have a nice range of tests;

d) It will have a great graphing / charting capability;

e) It will be modular and easy to upgrade;

f) It will be far more usable than existing programs;

g) And of course, it will be open source;

Our plan is to get the product working (the database browser does already quite nicely) and produce a version for the OLPC project. Some people have asked for a stats program that works there already and it makes sense to equip students with (possibly) the most useful tool in scientific research: statistical analysis. So far, it can import from a range of databases and analyse the data descriptively. Output is only text for now and interaction is via a custom interactive interpreter, but it's early days yet. From what we've read, the important thing is to get something released and we hope to do that very soon.

Ecstatistics is coded in Python with NumPy, SQLAlchemy, SQLite and lots of other stuff. Because of this, we can code the OLPC version down to about 200k which competes extremely well with the opposition like R, SPSS and SAS. The interface is designed not just to be useful but also to instill good statistical practice, so it's educational too.

The interface will be designed with non-expert users in mind, particularly students. We aren't aiming at calloused statisticians; they have their favourite tools (and often write them for themselves anyway). We are aiming at all those people who have to do stats but don't like it.

In other news, I saw a couple of laptops here in the Phils in a major chain of electrical stores. They came with Linux preinstalled which was nice to see.

Finally, but most importantly, my wife had her scan earlier this week - we're expecting a little baby girl! The due date is the end of July and we're both very excited.

It's fun. I've been coding a database exploration tool - it's a simple thing designed to explore databases in order to retrieve data for analysis. Python, wxPython & SQLAlchemy are the main bits used. It's fun to see the GUI fire up and pause as it downloads all the information from my remote DB. It does not have a clean GUI yet so there is a lot of work left in making it professional (or should that be presentable? I've seen an awful lot of poor professional interfaces in my time) but that is minor stuff now.

Right now, it's still in the mid-30s as far as temperature is concerned. It's mid-afternoon and a little too hot to work any more. I'll have a nap underneath a large fan and try to get back to work later. Did I mention that I have no air-con in my room? =:-O

I've been busy unit-testing the stats algorithms lately and using R to get "known" values. All seems to be going quite well though there is a modicum of error because mine have greater specificity. The testing showed that an error exists in one of the quantile calculations and the trimmed mean is borked (I mean really borked), but I had guessed that anyway. Still, it's good to have the accuracy of the tests documented.

It's extremely hot here in Laguna. The temperature is 33 degrees but more than that, it's extremely humid and still (little wind). It's hard to be bent over a laptop.

I've been here for 5 weeks now and am enjoying my time.

Strewth! One of my photographs was accepted at the Pentax Photo Gallery. That's surprising because I understand that an awful lot of very good pictures are declined.

And of course, I had 5 others declined too! ;-)

Anyway, this is it: a shot from Trondheim in Norway.

It's hot here in the Philippines - about 34 degrees all told. Everyone is out on the terrace chatting while I'm slaving over a hot computer trying to write an administrative database for a new (FOSS) program. I'll be able to release full details about it soon but it will take a bit of time to get it functional. Okay, that's not the purpose of FOSS, but I'm trying to get the design and architecture right first time.

I had fun on the weekend writing an event-handler from scratch which I managed to do in just a few lines of Python. Progress is going well and I hope to get something that works released by the end of May.

Still, it's a relatively peaceful though busy time. I'm working almost constantly on this software, but I have few other real pressures. It's a pleasure to be able to work full-time on a FOSS project and do it the way I want to do it. I'm happy because this whole time reminds me why I love programming so much - the constant challenges and beating them to take more steps toward a final goal of actually having built something worthwhile.

I'm taking on a lot of roles with this work: interaction design, programmer, tester, all-round monkey etc. I like the different colours and flavours that this work brings me - and if I ever get tired, relaxation here is good, honest and easy. I'm a lucky man indeed.

Looks like I've spent some time away from here! But I have been extremely busy.

So what's happened since my last entry? Non-FOSS, my wife came and lived with me in the UK for a few months, but we've had enough and are now both living in the Philippines together! I have my residence visa approved and should be picking it up soon (next week). Jell is pregnant too - the due date is the end of July. We're not sure if it's a girl or a boy yet, but should be having the scan soon to find out. My job contract finished which made things easier for parting so I'm no longer and academic. Life is good!

In FOSS stuff, I am revamping SalStat - or rather, I'm dumping it altogether and with a partner in New Zealand, we're launching a much better program. It will be open source and we want to make a business around it. The whole program is quite ambitious but quite possible. As always, we're using Python to code it. We're gunning for SPSS and SAS in a serious way. Investment ideas/leads are welcomed.

And how come I'm a master now? I still think of myself as a journeyer and probably will for ever ;-)

I'm back from PyCon UK which was probably the best conference I have ever been to. It was large enough so that there was always someone new to talk to (and new things to hear about), but not too large (unlike the med ed conference in Trondheim with ~1,800 delegates). People were friendly, it was well organised, interesting and varied, and I had a great time listening to things, having ideas and chatting away about my own stuff.

Isn't putting a business plan together really boring? I know this type of thing is usually done and it can help with the thought process of defining exactly what (and what not) a business should focus on; but it's a dry topic even then.

Somehow I think that passing around lots of hand-written notes to potential investors will not be too successful!