Persistent State
Last updated at 2005/04/19 04:51:43 PDT by Temporal

Evlan programs are persistent. This means that when you start the Evlan VM, any programs running on top of it are continued from where they were when the VM was last shut down. The data each program had stored in RAM is saved to disk on shutdown and restored on startup. Furthermore, as Evlan runs, the data in RAM is backed up to the hard disk on a regular basis to prevent significant data loss on a system failure.

At first, this may sound somewhat inefficient. You might think that backing up the complete state of a program's memory would be slow -- possibly causing a pause in execution -- and use a lot of hard drive space. As it turns out, though, because of the nature of purely functional languages, it is trivial for the virtual machine to identify which data has changed since the last time a backup was made and only write that data. Thus, persistence is in fact quite efficient.

The implication of this is that it is perfectly reasonable to write an Evlan program which stores all of its persistent state -- configuration options, databases, etc. -- in RAM. Depending on the nature of the software, this could mean a drastic reduction in development costs while at the same time making the software far easier to maintain.

For example, take Io Community Manager (IoCM), the content management system which runs the evlan.org web site (among others). Io Community Manager was originally written for fateofio.org, a collaborative internet project aimed at creating a role-playing computer game. It includes features like content management, forums, file submissions, chat, an award system, and more.

Before IoCM was written, fateofio.org ran on a similar content management system written using Java servlets and an SQL back-end which we called "v4 Project Manager". About half of V4PM's code was dedicated solely to translating data structures to and from their database representations and other database maintenance tasks. This code tended to make it very difficult to add new features to the software. Every time I wanted to add a new attribute to some table in the database, about ten separate places in the code had to be modified to reflect this change. This basically ended up meaning that I simply didn't add new features because it was too difficult.

IoCM, on the other hand, simply stores all data in RAM, which eliminated the need for any sort of database translation code, cutting out a huge amount of coding time. And, as I have found, adding new features to IoCM is far, far easier than it was with V4PM, as usually only one or two places need to be updated. IoCM and V4PM both took about a month to develop, but IoCM is far more powerful, with many more useful features.

Ironically, despite the fact that IoCM stores everything in RAM, and despite the fact that this server hosts many more sites than the old V4PM server did, the new server has yet to break 20MB of system memory usage while the old server was easily over 75MB. And that's just for the respective VMs, not counting the memory usage of the old server's database process. And, again, the Evlan VM has the potential to use far less RAM with proper optimization.

Now, this is not to say that Evlan necessarily makes databases obsolete. For very large systems, they may still be useful. But, the majority of software will get by just fine without them, and development costs can be reduced greatly by dropping the extra code.

Implementation Status: Evlan v0.3.0 implements persistent state, but would benefit from more work. The current implementation only reads the state file on startup, and only writes after that. A better implementation would try to keep memory usage low by freeing rarely-used data and restoring it from the state file as necessary. Also, the current implementation only updates the state file when explicitly requested, although it is trivial to write a loop which makes this request periodically. Incidentally, the current implementation is fault-tolerant.

References: EROS and Unununium implement automatic persistence at the OS level. Some programming languages probably implement it too.

evlan.org © Copyright 2003-2005 Kenton Varda
Powered by Io Community Manager, Evlan, and FreeBSD