Tuesday, August 30, 2005

Database Resource Abstraction Layer

I've just put together a sample database abstraction layer to a MySQL database, for demonstration purposes:

The idea (not all these features are working yet) is to allow read only access via the above methodology. _ENTIRE_CONTENTS will return a CSV representation of the data (assuming no large objects or equivalent). If the table has a SINGLE primary key, records will be listed as resources with names as those keys - these have no content but all attribute fields are available as live properties of these resources.

Trying to implement these highlighted some weaknesses in the server - it is too file oriented. Properties like modified/creation date may not mean anything to a record or table, and contentlength is a problem - there is no file size, so the only way to calculate the content length for a table.... is to retrieve ALL the content.

The solution here is to tweak the server to make these optional, such that if contentlength or modified date support is not provided, server will not use them (no last modified, no IF_(UN)MODIFIED_SINCE matching for example) or return the corresponding headers, and to support some other properties that were previously left out since they did not have too much to do with files, like getcontentlanguage.

Not all of this will appear by the SoC deadline, but most likely by mid-September (next release).


Ian Bicking said...

I think in some cases you can make an abstract superclass for the resource providers. These can do things like calculate content length the hard way (read the content, give the lenght). Then you can get something quick and dirty (but complete) up quickly, and later override some of those other methods to be faster.

I'm not sure this is really right for content-length -- if it means reading the database twice for every access (for instance) it's probably a bad idea. But something to keep in mind.

cwho said...

I managed to get in the support<>() methods so it doesn't retrieve the content length if the resource layer does not support it.

The database is done - but in general I am left with the taste that it is being inefficient. Example, it should be simple to retrieve all the properties of a record (SELECT * from table where...), but the resource layer API breaks it up into 10 getSingleProperty calls which results in 10 (SELECT property from table where....) queries.

Should the resource layer API be made broader (which makes it harder for all developers to make their custom components), or leave it to the database-interface developer to do some sort of optimization caching (retrieving all the properties at once and returning it when required).