Name: David Welton
Member since: 2000-02-16 03:01:49
Last Login: 2008-04-04 07:13:59
Homepage: http://www.welton.it/davidw/
Notes:
3 Feb 2010 (updated 3 Feb 2010 at 23:06 UTC) »
Italy vs Google
I'm starting to notice a pattern here:
One way of seeing things is that politicians and businessmen in Italy noticed Google was actually making quite a bit of money, and even if they don't quite understand this internet thing, they want some of the loot.
And while Google certainly is becoming big enough to be cause for worry and discussion, the moves against them in Italy do not seem anything like a rational response calculated to offset severe failures in the market.
In any case, it will be interesting to see what happens. Maybe, after China, we'll see Google quit Italy as well?
Syndicated 2010-02-03 21:26:25 (Updated 2010-02-03 22:26:20) from David's Computer Stuff Journal
Flippa experiment
I decided to try a little experiment with Flippa.com, a site where you can auction off domains or web sites.
I put http://www.innsbruck-apartments.com up for auction:
http://flippa.com/auctions/83341/Innsbruck-Austria-rental-listing-site---Ski-Season
We'll see how it goes and whether the site is worth using for other sites that I'd like to sell on.
It's a good test case, because it's a site I threw together years ago simply to aid our search for a new apartment in Innsbruck, and then requested by friends.
Syndicated 2010-01-25 09:34:08 (Updated 2010-01-25 09:36:38) from David's Computer Stuff Journal
12 Jan 2010 (updated 12 Jan 2010 at 15:08 UTC) »
Rough Estimates of the Dollar Cost of Scaling Web Platforms - Part I
I have been pondering the idea behind this article for a while, and finally had a bit of time to implement it.
The basic idea is this: certain platforms have higher costs in terms of memory per concurrent connection. Those translate into increased costs in dollar terms.
Nota Bene: Having run LangPop.com for some time, I'm used to people getting hot and bothered about this or that aspect of statistics that are rough in nature, so I'm going to try and address those issues from the start, with more detail below.
All tests were run like so: my new laptop with two cores and four gigs of memory was used as a server, and my older laptop was used to run the ab (Apache Benchmark) program - they're connected via ethernet. I built up to successive levels of concurrency, running first 1 concurrent connection, 2, 10, and so on and so forth. The "server" computer is running Ubuntu 9.10, "karmic". <h3> Platforms</h3>
The platforms I tested:
As stated above, it's pretty obvious that using Rails or Django for something so simple is overkill: <h3> Better Tests for the Future</h3>
I would like to run similar tests with a more realistic application, but I simply don't have the time or expertise to sit down and write a blog, say, for all of the above platforms. If I can find a few volunteers, I'd be happy to discuss some rough ideas about what those tests ought to look like. Some ideas:
With that out of the way, I do think the results are meaningful, and reflect something of what I've seen on various platforms in the real world.
First of all, here we look at the total "VSZ" (as ps puts it) or Virtual Size of the process(es) in memory. Much of this might be shared, between libraries, and "copy on write" where applicable.
The results are impressive: Rails, followed by Django and PHP eats up a lot of memory for each new concurrent connection. Rails, which I know fairly well, most likely suffers from several problems: 1) it includes a lot of code. That's actually a good thing if you're building a reasonably sized app that makes use of all it has to offer. 2) Its garbage collector doesn't play well with "copy on write". Which is what "Enterprise Ruby" aims to fix. Django and PHP are also fairly large, capable platforms when compared to something small and light like mochiweb.
That said, excuses aside, Erlang and Mochiweb are very impressive in how little additional memory they utilize when additional concurrent connections are thrown at them. I was also impressed with Jetty. I don't have a lot of experience with Java on the web (I work more with J2ME for mobile phones), so I expected something a bit more "bloated", which is the reputation Java has. As we'll see below, Jetty does take up a lot of initial memory, but subsequent concurrent connections appear to not take up much. Of course, this is also likely another 'apples and oranges' comparison and it would be good to utilize a complete Java framework, rather than just a tiny web app with one JSP file.
So what's this mean in real world terms of dollars and cents? As your Rails application gets more popular, you're going to have to invest relatively more money to make it scale, in terms of memory.
For this comparison, I utilized the bytes/dollar that I'm getting for my Linode, which works out to 18,889,040.85 ($79.95 for 1440 MB a month).
As we can see, to have a similar amount of concurrent users is essentially free for Mochiweb, whereas with Rails, it has a significant cost. This information is particularly relevant when deciding how to monetize a site: with Erlang and Jetty it would appear that scaling up to lots of users is relatively cheap, so even a small amount of revenue per user per month is going to be a profit, whereas with Rails, scaling up to huge numbers of users is going to be more expensive, so revenue streams such as advertising may not be as viable. It's worth noting that 37 signals, the company that created Rails, is a vocal supporter of charging money for products.
There's another interesting statistic that I wanted to include as well. The previous graph shows the average cost per additional concurrent user, but this one shows how much the platform costs (using when there is just one user, so it acts as a sort of baseline:
As we can see, Jetty is particularly expensive from this point of view. The default settings (on Ubuntu) seem to indicate that, for instance, the basic $20 a month Linode package would not be sufficient to run Jetty, plus a database, plus other software. I think that the Apache Worker number is off a bit, and may reflect settings made to handle a large number of connections, or perhaps a different MPM would make sense. <h3> Source Code / Spreadsheet</h3>
The spreadsheet I put together is here: http://spreadsheets.google.com/ccc?key=0An76R90VwaRodElEYjVYQXpFRmtreGV3MEtsaWYzbXc&hl=en
And the source code (admittedly not terribly well organized) is here: http://github.com/davidw/marginalmemory/
Syndicated 2010-01-12 11:39:37 (Updated 2010-01-12 14:20:27) from David's Computer Stuff Journal
Detecting BlackBerry JDE Version
Recently, I went back and added some preprocessor code (it's pretty much necessary in the world of J2ME) to ensure that Hecl would compile with older versions of the BlackBerry JDE. However, I also faced a problem: how to figure out what version of the JDE we're using. It could be my latest cold clouding my mind, but I couldn't find a simple way to do this. It never seems to be simple with the BlackBerry platform, unfortunately.
I did, however, finally find a nice way to obtain this information programmatically: the bin/rapc.jar file, which ships with the JDE, contains a file called app.version, which, indeed, contains the version of the JDE in use. I hacked up this code to read it and print it out: <script src="http://gist.github.com/266010.js"></script>
Syndicated 2009-12-30 12:15:55 from David's Computer Stuff Journal
File Selector for Java ME
I recently did some work to make Hecl read files, which also means that it can execute Hecl scripts from the phone's memory. This is especially important for environments like Blackberry, where we will be distributing a signed version of the Hecl application. To create your own Hecl applications, instead of simply replacing the script in the .jar file, you can point to a script on the device's file system. This is also available for normal Java ME phones, but unfortunately, for an open source project, the cost of a code signing certificate are prohibitive (on Blackberry, it's only $20, so I bought one with my own money).
In any case, as part of this effort, I developed a very simple 'file browser', which is used in Hecl to select a script to execute.
The results are, like all of Hecl, available under an Apache license, which means that you can use it pretty much wherever you want:
http://github.com/davidw/hecl/blob/master/files/org/hecl/files/FileFinder.java
http://github.com/davidw/hecl/blob/master/files/org/hecl/files/FileFinderCallback.java
Of course, if you spot any ways to improve it or fix problems with it, I'd appreciate it if you sent patches back.
Syndicated 2009-12-14 12:31:46 from David's Computer Stuff Journal
davidw certified others as follows:
Others have certified davidw as follows:
[ Certification disabled because you're not logged in. ]
FOAF updates: Trust rankings are now exported, making the data available to other users and websites. An external FOAF URI has been added, allowing users to link to an additional FOAF file.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!