[ Home | Articles | Account | People | Projects | FAQ ] PersistJS: Cross Browser Client-Side Persistent Storage Without Cookies
I just released PersistJS, a client-side JavaScript persistent storage library. Features include:
If you already know why this is awesome, then you can skip straight to the download. If you're scratching your head, then read on... <h2>Why This is Awesome</h2>
Why use PersistJS? What's the problem with using cookies directly or simply requiring Flash?
Currently the only reliable cross-platform and cross-browser mechanism for storing data on the client side are cookies. Unfortunately, using cookies to store persistent data has several problems:
Modern web browsers have addressed these issues by adding non-Cookie mechanisms for saving client-side persistent data. Each of these solutions are simpler to use than cookies, can store far more data, and are not transmitted along with HTTP requests. Unfortunately, each browser has addressed the problem in a different and incompatible way. There are currently 4 different client side persistent data solutions:
Some developers have attempted to address the client side storage issue with the following browser plugins:
The problem with relying on plugins, of course, is that users without the plugin installed miss out on the feature in question, and your application is dependent on software from a particular vendor. Google Gears, for example, is not widely deployed. Flash is, but it has problems of its own:
Anyway, if we include Gears and Flash, that means there are no less than 6 incompatible solutions for storing client-side persistent data.
The most notable attempt at addressing this problem is probably Dojo Storage. Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based browsers at all (at least, not without Flash). Also, Dojo Storage is not standalone; it requires a several other Dojo components in order to operate.
PersistJS addresses all of the issues above. It currently supports persistent client-side storage through the following backends:
Each backend exploses the exact same interface, which means you don't have to know or care which backend is being used. <h2>Examples</h2>
Here are some simple examples of PersistJS in use:
// create a new client-side persistent data store
var store = new Persist.Store('My Data Store');
// pretend data
var data = "pretend this is really long data that won't fit in a cookie";
// save data in store
store.set('saved_data', data);
That's all there is to creating a persistent store and adding some data to it. Fetching data back from the store uses a callback function (to support asyncronous backends), but it's still pretty simple to use:
// get data back from store, and prompt user with it
store.get('saved_data', function(ok, val) {
if (ok)
alert('saved data = ' + val);
});
Removing data is pretty easy too:
// remove data from store
store.remove('saved_data');
Although it isn't necessary, you can also get some information about the
detected backend using the Persist.type and Persist.size attributes:
// build alert message
var info = [
'Backend: ',
Persist.type || 'none',
', ',
'Approximate Size Limit: ',
(Persist.size < 0) ? 'unknown' : Persist.size
].join('');
// prompt user with information
alert(info);
Finally, if you don't want a particular backend used under any
circumstances, you can disable it using the Persist.remove() function:
// never use cookies for persistent storage
Persist.remove('cookie');
This is the initial release, so there are bound to be some bugs. PersistJS has been tested with FireFox 2.0, FireFox 3.0rc1, IE7, and Safari 3.1. The Gears and Flash backends work where Gears and Flash 8 are supported.
The most notable omission here is IE6; it should work, but I don't have IE6 handy at the moment (MultipleIEs is temporarily busted).
Commenting is still busted around here, so any comments should sent via email or posted on the Reddit thread.
Don't Use ExtJS
<style type='text/css'> blockquote.quote { font-family: arial, verdana, helvetica, sans-serif; border-left: 2px solid #09d; padding: 10px; background-color: #efefef; } </style> A couple of years ago I recommended YUI and Ext (formerly YUI-Ext). I've changed my mind. Don't use Ext at all. The Ext license has changed four times since its inception, and each time the license has become more restrictive. <h2>History</h2>
Ext was originally created as an extension to YUI. It was BSD-licensed, just like YUI. YUI-Ext added several sorely-needed features to YUI. The most notable additions were a layout system, a Tree View widget and a Data Grid widget (YUI has since added each of these, although the YUI widgets are still less flexible than their Ext counterparts). Eventually support was added for jQuery and Prototype as well. The team dropped the "YUI-" prefix, and YUI-Ext became Ext.
Ext 1.0 was relicensed under the LGPL. Although switching from the BSD license to the LGPL is relatively innocuous, it is still significant because the LGPL is more restrictive than the BSD license.
Eventually the Ext team changed the license again. The new license was a custom license that granted conditional LGPL usage rights. Basically the LGPL usage clauses applied, but only if you weren't trying to develop a library or an Ext clone.
Confused? Yeah, me too. Here's the text from the old Ext "Open Source License":
Ext is also licensed under the terms of the Open Source LGPL 3.0 license. You may use our open source license if you:
- Want to use Ext in an open source project that precludes using non-open source software
- Plan to use Ext in a personal, educational or non-profit manner
- Are using Ext in a commercial application that is not a software development library or toolkit, you will meet LGPL requirements and you do not wish to support the project
A lot of open source developers were understandably confused by this hybrid license. The biggest problem was that it wasn't clear whether this license was compatible with other Open Source licenses. It also wasn't clear whether Ext could be legally distributed with Open Source software, since the license only granted LGPL usage rights, and not LGPL distribution rights.
To address these complaints, the Ext team changed the license again: the latest version of Ext is licensed under the GPLv3. This latest change complicates things quite a bit for many users, as we'll see in the next section. <h2>Problems</h2>
The GPL is far more restrictive than the BSD license and LGPL. It is rarely used for libraries, because the viral clause would effectively the library from being used for any non-GPL software. In fact, these problem were addressed over 16 years ago by creating the LGPL:
By 1990, it was becoming apparent that a less restrictive license would be strategically useful for some software libraries; when version 2 of the GPL (GPLv2) was released in June 1991, therefore, a second license - the Library General Public License (LGPL) was introduced at the same time and numbered with version 2 to show that both were complementary. The version numbers diverged in 1999 when version 2.1 of the LGPL was released, which renamed it the GNU Lesser General Public License to reflect its place in the GNU philosophy.
Who is affected by this change? In no particular order:
Things get even murkier when you consider linking and distribution. Does generating a dynamic page count as linking to Ext? Does any public web application automatically count as distribution? What about applications which use Ext to and access a common APi, such as a SOAP endpoint or RSS feed?
These questions were posted in a
thread about the license change on the ExtJS forums.
Here's how Jack Slocum, the primary Ext developer, responded:
If you are generating any markup or javascript code via the server in a page SPECIFICALLY designed for Ext, then that server code will have to be GPL as well.
For example:
- Suppose you have an index.php that includes Ext JS. According to the FSF, in that case index.php would be also under GPL since it is using ext. Since it must be GPL, it's source must be distributed. Since it is GPL, the "viral" effect of GPL is now in effect and any thing that uses index.php (if anything) on the serverside would also fall under the GPL. (Note: Note this is a pretty gray area)
- Suppose you are using server-side code to generate javascript that interacts with Ext JS. That code must also be GPL.
Like MySql and other GPL software the way to use GPL code without having to license under GPL is to not bundle or distribute the GPL code with your application. If you instead have the end user (developer?) download and install ext js on their own, they are then bound to the license and not you or your software.
For those seeking an FAQ, we have defined and explained some of the reasoning and license implications under these 2 pages:
http://extjs.com/products/license.php
http://extjs.com/company/dual.php
It's worth noting that the examples given at the beginning of this post are just my opinion and it is impossible for us to analyze everyone's usage and say whether or not someone "complies" with the GPL. That really is a task for an attorney or even someone with better knowledge of your application and how Ext JS is used.
In the end, we want Ext JS to be open source friendly and still have a good business model in place to grow. The old Ext License was not open source friendly and pretty much killed all options for use in open source projects. That wasn't our goal so we had to address it.
There are several problems with the statements above. The biggest one is that the original BSD license and subsequent LGPL license had none of this ambiguity.
In other words, the problem the Ext team is trying to fix is one they created themselves. If that wasn't bad enough, the solution actually hurts many Open Source developers far more than it helps.
In an attempt to clarify the situation for non-GPL Open Source developers, I posted several questions in the Ext license thread. I also created a post on Reddit about the license change and summarized my questions there:
As of page 8 of the thread on the license change I have yet to receive a response, simple or otherwise, to any of my comments:
There's this comment:
The new license prevents Open Source software that is using a license other than the GPL from using Ext. Applications which use popular Open Source licenses like the LGPL license, BSD license, MIT license, and the Artistic license would be required to either re-license under the GPL, carefully design their application to meet the requirements in your post, use an older LGPL-licensed version of Ext, or move to another library entirely.And this one:
What about authors who which to provide their software under a license that is more permissive than the GPL, such as the MIT or BSD licenses?And this one, which was directly in response to Jack Slocum, the primary author of ExtJS:
Hi Jack,
I can see how switching to a license without exceptions would make things simpler, but what about those of us who release Open Source software under non-GPL licenses such as the BSD, MIT, and Artistic licenses?
I've been an Ext user since its inception as YUI-Ext, but the fact that I cannot seem to get a straight answer to a simple question makes me wary and extremely skeptical.
John Resig, the author of jQuery and Processing.js, responded:
It's important to understand that OSS developers are not their target audience at all. I'm 100% certain that we'll never get a clear response. They're using 'open source' as a buzzword selling point to lure companies in, befuddle them with confusing viral licensing, and obligate them (through the obvious balking that the corporate lawyers will do) to get them to buy a full, corporate, license. It's very sneaky, quite disingenuous, and paints a bad picture for open source development as a whole.
It's been over three weeks since these this exchange on Reddit. None of my questions have been answered on the Ext license pages or in the 68-page license thread on the Ext forums. <h2>Rationale</h2>
According to the Ext license page, Ext licensing is based on the principle of "Quid Pro Quo", or "something for something":
Dual Licensing is based on the principle of Quid Pro Quo - "something for something". In return for the advantages you realize from using an Ext product to create your application, we require that you do one of the following:
- Contribute to the continued development of the product by purchasing commercial licenses from Ext. This option secures you the right to distribute your application under the license terms of your choice
- Contribute to the Open Source community by placing your application under an Open Source license (e.g. GPL v3). This option secures all users the rights to obtain the application's full source code, modify it, and redistribute it.
The justification for using the GPLv3 instead of the LGPL is addressed on the Ext license FAQ page:
We considered once again releasing under straight LGPL but it was not an option as a business. We tried that with version 1.0 and found out quickly that it enabled others (e.g. large commercial entities) to take our work, wrap it up and sell it as their own. With no mention of us at all. We, as a business with a full time team of talented developers, can not exist under those circumstances. We would quickly become diluted and competing with ourselves.
The concern about others taking their work and selling it without attribution is particularly ironic, considering:
What do the projects above have in common? That's right, they are all in the public domain or available under extremely permissive Open Source licenses.
The Ext team is certainly entitled to license and sell their software any way they see fit. However, it is hypocritical and dishonest to complain about other people taking your work and selling it as their own when you take other peoples' work and either sell it as your own or relicense it under an extremely restrictive license.
It is tempting to attribute this entire fiasco to a simple misunderstanding on the part of Jack Slocum and the Ext team. Here's what I had to say on Reddit:
It is a bit disconcerting that Ext has such strong roots in existing Open Source software, and yet the project seems at best partially indifferent, and at worst, outright hostile to the Open Source community.
Unfortunately, according to John Resig, this isn't the first time that there have been problems with the Ext team:
We (the jQuery project) worked hard with them to try and fix bugs and add features for an ExtJS integration layer. They turned around and built their own, specialized, library (removing the need for any of our work) and then mutated the licensing into this bizzaro scheme that they have now. We can't, in good consciousness, even recommend their library anymore due to its very nature. On top of this they ended up hiring our lead evangelist to promote their work. I can't speak for everyone on the team but I feel quite frustrated and used.
They're providing a great disservice to the Open Source community in general. They consume with reckless abandon, it's impossible to even hope to borrow code from them, and they turn it all into a money-making machine. No aspect of that sits well with me.
Jack Slocum did respond to this comment on a separate blog. He also wrote a post on his blog. Neither adequately addresses John Ressig's main points or my questions from the Ext forums, so I won't bother quoting his mostly vacuous responses here. <h2>Conclusion</h2>
To recap, the reasons I recommend against using Ext are:
Some suggestions for the Ext team:
Finally, here are a list of Ext alternatives. None are as nice as Ext, but they are all available under permissive licenses and they each have an active and enthusiastic user community:
Comments are still broken at the moment. I've posted this article on Reddit, so feel free to comment there.
EasingJS, a JavaScript Easing Library
I just released EasingJS 0.1.1, a port Robert Penner's ActionScript Easing library to JavaScript. EasingJS allows you to easily generate smooth and stylish animation or color transitions. For some examples, check out the test page.
Custom Firmware on the PSP
Last night I installed a custom firmware (CFW) on my PSP. The custom firmwares allow you to run unsigned homebrew (e.g. non-Sony sanctioned) applications. <h2>Applications</h2>
The PSP homebrew community has been pretty active; there are several useful applications and emulators for lots of older systems. There are even emulators for obscure systems like the ColecoVision and Neo Geo.
Here's what works for me so far, in no particular order:
I also tried the Genesis, NES, and N64 emulators, but they aren't working yet. Here's a picture of the SNES emulator at work:
|
| Playing SNES on a PSP |
The custom firmware also allows you to dump UMDs and run them from a memory stick. Since it's pretty much impossible to fit a PSP and 8 UMDs in the your pockets without looking like a complete tool, I'm going to offload as many UMDs as I can into the 3GB remaining on my memory stick.
The next section explains the firmware installation process. If you don't have a PSP, you may still find my creatively ominous safety warnings entertaining. <h2>Installation</h2>
Installing the custom firmware varies in complexity depending on the model of PSP model and version of the original firmware. If you're fortunate enough to have an older "phat" PSP (e.g. the larger black model) that's running firmware 1.00 or 1.50, then installing the custom firmware is fairly straightforward.
If you know someone with a PSP who already has the custom firmware installed, then the installation process is still easy enough, because they can use their PSP to help you with yours.
If you've got a newer PSP Slim (the smaller white model, like the one in the picture above) and/or are running a newer firmware, then there are no easy options left, so get ready for the comically unpleasant experience below.
In order to install the custom firmware, you'll need a spare battery and a spare memory stick. For the love of Douglas Adams, please do not use this post as a guide! There are several web sites (here and here) that cover the entire installation process in far more detail and with the appropriate safety precautions. If you mess this up you will turn your PSP into a lifeless and possibly explosive plastic brick.
The basic, high-level steps are as follows:
The hardest part of this process is creating the Pandora's battery. If you know someone with a PSP that already has the custom firmware installed, then they can run an application on their PSP to temporarily "soft-mod" a regular battery into Pandora's battery.
If you don't know anyone with a PSP that already has the custom firmware installed, then the only way to convert a regular battery into a Pandora's battery is to "hard-mod" it; that means cutting open the battery casing and disconnecting one of the leads on the internal circuitry.
The guides I read (see above) have plenty of pictures, but I was still surprised by how small the pieces actually were. Here's a picture I just took of my Pandora's battery, including a ruler and quarter as size references:
|
| Inside a Pandora's Battery |
Creating a Magic Memory Stick is much simpler. Basically you:
mspformat)
mspinst)
If you're using Windows, the "TotalNewbi Installer" and "Pandora Easy GUI" tools can automate this process. In theory, anyway. When I tried to use them in my Windows XP VMWare instance, they both had problems. The TotalNewbi Installer simply refused to work, and the Pandora Easy GUI blue-screened XP each time I ran it.
Here's what finally worked:
mspinst.idl)
dd in Linux to copy mspinst.ibl into the first sector of the
Memory Stick
The good news is that creating the Pandora's Battery and Magic Memory Stick are the hardest steps in the process. Once you get past them, everything else is relatively straightforward. Even better, the process can be used to install custom firmware on any PSP, regardless of hardware model or firmware version.
That's it for me. If you're interested in the history and technical details of PSP homebrew, check out this extremely detailed PSP homebrew Wikipedia entry.
...as if things weren't broken enough around here
Hello from the new server!
I managed to glom things together just enough to get the bit-rot that is this site's code running on the new server. All the usual caveats about brokenness apply.
Reddit Content Filter 0.4
Works with the latest changes to Reddit.
Local copy:
There was a suggestion in the comments on User Scripts about auto-downmodding blocked articles. I kind of like the idea. Does anyone have any opinions on that? Comments are broken at the moment, but feel free to email me.
HTPC Shot
I got the LCD power adapter, power supply, and new CPU fan for my HTPC. Here's a picture of it in my entertainment center:
I've been planning to build an HTPC for quite a while now, and I'm
finally doing it. I got an Antec Fusion Black case for my
birthday, and I went out and bought the rest of the parts yesterday
evening. Here's the hardware I'm using:
The other pieces were all spare parts and aren't particularly
interesting. This post is about the Antec Fusion Black. Let's start
with the pros:
And the cons:
Fortunately I've already decided to buy a Harmony remote, so the MCE
remote requirement is a minor inconvenience and an excuse to go buy a
new gadget.
The most irritating problem so far is that the power supply fan is
extremely loud. I think my PSU is probably defective, because it
doesn't make any sense to design a sound-isolating case with extremely
quiet case fans and then put a power supply that sounds like an air raid
in there. Alternatively, the noise might be related to the non-existent
3-pin power supply fan signal connector cable mentioned above.
Anyway, I don't really want to send the case back, so I ran to
Microcenter and picked up a new silent power supply. That's when I
noticed the non-standard cable that powers the LCD/IR receiver, and the
following blurb tucked away in the manual:
Note: If you choose to swap the included power supply with another
power supply, please call Antec Customer Service to purchase a special
24-pin Extender with the 3-pin connector to power the display.
I've already ordered the adaptor from Antec's web site; I figure even if
I figure out how to quiet the existing power supply it's worth having so
I don't end up without the LCD and IR port if the power supply failure
in the future.
Next up, Antec Fusion Linux support. The LCD is not officially
supported in Linux by Antec. The good news is that the LCD in the
Fusion V2/Black is better than the VFD display included in the original
V1 Antec Fusion case. Unfortunately it is much harder to find
documentation on the newer display. The newer display also requires a
couple of patches and some config file twiddling.
Here's what you need to do, as of today:
After you patch LCDproc, you need to do the following or LCDproc won't
pick up the changes from the patch:
Compile and install both LIRC and LCDproc.
Find the Add this section to Make sure the LIRC modules are loaded:
Start up If you're using That's all for tonight. It looks like configuring the motherboard in
will be a battle too; I'll have more details as events warrant.
aclocal && automake && autoconf
./configure --enable-drivers=imonlcd
[server] section in /etc/LCDd.conf and add the following
lines:
Driver=imonlcd
DriverPath=/usr/local/lib/lcdproc/
/etc/LCDd.conf:
[imonlcd]
Device=/dev/lcd0
Contrast=300
# do NOT set the Size, if you do it won't work
#Size=16x2
modprobe lirc_dev && modprobe lirc_imon
lircd and LCDd, and things should be working.
2.6.23.1 or newer the patches above will not work;
you'll need these updated patches against the CVS version of LIRC.
You'll still need to follow all the steps above, even with the updated
patches.
WELCOME TO THE NEXT GENERATION
I got sick of the page being so slow, so I this morning I made some fairly destructive slash-and-burn style changes under the hood. Things should be mostly working again, with the following exceptions:
There may also be other stuff broken, too. I've tried to keep URLs for articles and whatnot the same, but your mileage mamy vary. Today is the first of several iterations away from the almost decade-old code this site is running.
EasyCookie: Easy Javascript Cookie Management
Ever tried to read and write cookies in Javascript? If you have, then I'll wait until you've stopped frothing at the mouth and pounding your keyboard.
...
Feeling better? Good. I just released the first public version of EasyCookie, a simple cookie library for Javascript. Using EasyCookie is, well, easy. For example, here's how you get a cookie:
// get a cookie
val = EasyCookie.get('my_cookie');
And here's how you set one:
// set a cookie
val = 'a random value that i want to save as a cookie';
EasyCookie.set('my_cookie', val);
And, if you haven't already guessed, here's how your remove a cookie:
// remove a cookie
EasyCookie.remove('my_cookie');
But what about all the extra crap you usually have to fight to get
working, like the domain, path, and expiration? Don't panic!
EasyCookie.set takes a hash of additional attributes as an optional
third argument. Here's another example of EasyCookie.set, this time
with the optional hash:
// value to set
val = '99 bottles of beer on the wall, 99 bottles of beer!';
// set a cookie that expires in 10 days, and limit the scope to
// "https://foo.example.com/some/path"
EasyCookie.set('my_cookie', val, {
// expires in 10 days
expires: 10,
// limit cookie to domain 'foo.example.com'
domain: 'foo.example.com',
// limit cookie to path '/some/path'
path: '/some/path',
// restrict cookie to secure pages only
secure: true
});
Checking to see if cookies are enabled just got a whole lot simpler, too:
// are cookies enabled?
enabled = EasyCookie.enabled;
// harass user with annoying dialog about their cookie status
alert('Cookies are ' + (enabled ? 'enabled' : 'not enabled'));
Best of all, EasyCookie is BSD licensed, and the minified version weighs in at a measly 1873 bytes. Anyway, here ya go:
PS. Yes, I realize the link above are to version 0.2.1 of EasyCookie, even though this is the first public release. In internet years the backend to my page is older than your grandmother. Imagine trying to explain how to release software to your grandmother.
FOAF updates: Trust rankings are now exported, making the data available to other users and websites. An external FOAF URI has been added, allowing users to link to an additional FOAF file.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!