Just to get that straight: I’m an avid db4o user and really love it. These issues are not critical and they don’t stop me from using or evangelizing db4o. However, I think there is some lack of awareness of some issues.

Also, I’d like to spawn some discussion about the issues below. Unfortunately, due to their changes to the forum system, most of the original discussions on the db4o forum are hard to find or possibly lost. You may want to vote on the issues you deem most pressing, which you can easily do in their issue tracker! I’m very interested in what you think about this little selection.

Selected Issues

» Don’t run SODA when no more constraints are present
I blogged about this already, because you experience this in very common scenarios, namely whenever you query a small subset of a larger candidate set. For example, consider selecting the last 50 posts on a blog/qa-site/etc. What will happen is that db4o runs the BTREE query for the sort operation (blazing), then hydrates (?) all objects, the returns the first 50 of them and throws away the rest. Thing is that there is no need to further inspect the items, and activating all them is basically a linear operation. Thus, this common type of query currently runs in O(n) instead of O(log n) which is an incredibly dramatic difference.

» LINQ-Implementation is not ‘thread’-safe
A very similar issue has been on LINQ-to-SQL’s todo list some time ago.
I’m not sure whether this is so much the typical use case. For the db4o case, it teaches us two things right now:

• Container reuse is non-trivial and should be approached with extreme care. You don’t want to run into this kind of byzantine error in a live app.
• The object identity problem is similar in both Object-Relational Mapping and OODBMS

» Scalable server architecture: multiple readers against the same file, transactional files
This sounds daunting, and it’s probably a huge one, as you can see from its age. I also believe this might be politically challenging, because this moves into the direction of Versant’s large-scale object database. However, there is a lot of movement into that direction from the user-side it seems – people are asking for features of this kind more and more lately, largely due to the ultra-cool LINQ integration db4o has. I suppose I’d be wise to focus on this kind of scenario as it could really become the preferred way of writing web applications: it’s extremely agile, supports rapid development, is flexible in that the same (LINQ) code could be used for different persistence layers if that should ever be needed and leads to clean, compile-time checked, type-safe code.

» Sanitize reflector design – remove core dependencies on generic reflector
Being able to get rid of the generic reflector seems important, I’m already building my own code for this. Here we have conflicting requirements: The GenericReflector makes db4o very easy to use and may help beginners. It is also required in client-server scenarios where the server doesn’t have the necessary model dlls, but for most applications I think you should try to avoid it. Storing data in a generic manner is slow and requires a lot more space, making it highly inefficient.

Current attack vector: Throw a Listener on the object created event on the server and make sure the server knows the type.

» Allow immediate TCP port reuse
When opening a server, the TCP port will be blocked in case the server crashes, the app is terminated, etc. Since that might happen quite often when you use the ‘integrated server’ where the server is actually created in your web application, a restart of the web application will fail because the TCP port is blocked. In a client-server scenario, on the other hand, a simple restart wouldn’t be possible because you need to assign a new port to clients or wait 6 minutes. This should be fairly simple, but I don’t know if that comes with any side-effects.

» Fast Collections
This is a huge one. The cool thing is that this could allow much more complicated queries to be executed in reasonable timeframes. However, I’m a bit worried about the issue “FastCollections: Inside BTree List implementation”, because that sounds really important, but is in state “Won’t fix”.

» Locking
Optimistic locking would be a nice-to-have thingie, but you do this yourself rather easy I think.

» A new object is stored upon value type updates
This is rated critical, so it’s not an item for a “wishlist”. I’m not sure if I understand its implications and I rarely use value types apart from Guids, and updating Guids is pointless – still a db4o user should probably know this and keep this in mind, so I felt it should go here ☠.

db4o Configuration

This one is not really in the issue tracker as a single item, and it’s more of a general remark. One of the rather messy things in db4o is configuration. Even with the new configuration interface, there is quite a bit of confusion among users. The reason, in my eyes, is mostly a combination of incomplete documentation and unexpected behaviour. Examples:

• The indexation setting for fields is the only configuration setting that is persistent. Everything else, including unique constraints, needs to be re-set when (or – more precisely – before) opening the ObjectContainer.
• Applying an option to a field that doesn’t exist will not trigger a warning or an Exception
• Some settings simply won’t have any effect when you perform them after opening the ObjectContainer, but they do not warn you.
• Certain settings must be applied on the server, a few must be applied on the client and with some … well, you just set them on both just to make sure. Here, db4o does throw exceptions, however!
• Several options need to be set before creating the object container (e.g. string encoding) and cannot be reset afterwards, again being completely silent about the ineffectiveness of the respective settings.
• Some settings, such as field-based cascade-on-activate, simply don’t seem to work at all

This leads to lots and lots of confusion. Most importantly, it is often hard to determine whether a certain setting was successfully applied, or not. Also, some defaults are unexpected:

• Default ActivationDepth is (completely random) 5. Why not 8? Or 2? This troubles beginners a lot. Either set it to infinity, or to zero. Everything else feels just random. You can still include a line ActivationDepth = 5; in beginner’s samples, thereby showing them that the setting is there and that they need to be aware of it.
• Default string encoding seems to be UTF-16 or UCS-2, probably the most useless encodings around, despite the Windows Kernel working with it. UTF-8 would come in as a reasonable default, but with UTF-16 half of your database is probably zeros, because even in non-english environments, there is still a lot of mostly ASCII-data to be stored (such as URLs, Email addresses, base64 encoded information, SHA-hashes, etc.). Also many languages have non-ASCII characters only sparingly, German being one example.

I think it’d be really cool if the configuration interface was a little more explicit and would throw Exceptions instead of silently ignoring requests that cannot be fulfilled.

Lately, I have fiddled around with Versant’s excellent dual-license object database db4o a little and I think object databases are very neat. In fact, I read a few pages in PoEAA [Patterns of Enterprise Application Architecture] today, and here and there Fowler writes how cool object databases are. The Fowler knows!

Well, anyhow when you download db4o (sign up and go for the latest release) there is no standalone server application included. That makes perfect sense, because the server needs access to your domain model / data model dll. In an object database, there are no tables, but there are objects, so you have to compile them into the application that runs the actual db4o server. There is server code which handles practically everything that is tricky, but it’s in a dll so you’ll have to write a few lines to build an application out of it. I figured this would be a great time to write my first windows service and so I mashed up some code I found on the net.

I initially thought the VS 2008 designer would set up pretty much everything, but it turned out that things are a little more complicated.

The Installer

Since a service will run in a different manner, potentially using a different user account, etc. it needs to be installed first. There is a tool called installutil.exe, but using it is a little awkward. Also, you’d have to deploy it along your application. Matt Davis posted a cool guide on stackoverflow and some additional information here on how to write a windows service that is esentially able to install itself!

Unexpected Features

One of the most peculiar features is that -should you prefer to have your own event log instead of writing to the ApplicationLog- you have to search through a tree of installers, find the EventLogInstaller and modify it instead of simply adding a new one because the ServiceInstaller will come along with the EventLogInstaller as a child:

EventLogInstaller installer = FindInstaller(this.Installers);
if (installer != null)
    installer.Log = ServiceConfiguration.LogName;

We also need to take care of spawning the actual worker thread, but fortunately there is a comprehensive guide on MSDN explaining this.

Using the Sample

First things first: Of course, this is not nearly production-safe code! Also, I did not include the db4o dll’s because I’m not sure about the licence terms.

The installer in 'action'. Note the Administrator prompt

1. Download the sample, extract
2. Download db4o, install
3. Copy Db4objects.Db4o.dll and Db4objects.Db4o.CS.dll from the db4o install directory to the samples’ lib directory
4. Compile
5. Open an Administrator command prompt and find the sample’s bin folder. There is no feedback in case anything goes wrong (at best, the program crashes)
6. Call the install routine by executing emphess.db4oServer.exe -install. This will install the db4o server and the associated event log. The server will not be started automatically.
7. Head over to services.msc or your EventLog, where you should find a new entry!
8. Edit the configuration file, ready to run!
9. Don’t use this for anything production…

The App.Config

The configuration file looks like this

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <appSettings>
    <add key="UseLocalFile" value="false" />
    <!-- <add key="DatabaseLocalFileName" value="irrelevant"/> -->
    <add key="DatabaseHost" value="localhost" />
    <add key="DatabasePort" value="52354" />
    <add key="DatabaseServerFileName" value="C:/Development/db4oServer/db/server.db4o" />
    <add key="DatabaseUser" value="test" />
    <add key="DatabasePassword" value="test" />
  </appSettings>
</configuration>

The UseLocalFile setting is meant for clients so you can easily switch between file-based and network-based configurations. Since db4oServer.Shared contains a small class that holds this configuration data, you can easily reuse the code in a different app, e.g. in the client.

Todo-List

There are some things I’d like to add over the next weeks, so I put the list here to give me some ‘pseudo-extrinsic motivation’… (yes, I like complicated words)

• Simple sample client
• Out-of-band data for backup, defragmentation commands, etc.
• Tests / Benchmarks

I hope you have any use for this.