Friday, July 22, 2005

Galore of Errors

Yesterday I finished work on the MOVE and COPY commands and was ready to start on testing this application as integrated. Only the LOCK mechanisms are lacking and I will implement that, prefering to do a midway test. The litmus library http://www.webdav.org/neon/litmus/ was perfect for this. I installed and compiled it over cygwin.

Extending the server:

After several failures on the first test (MKCOL) and some telnetting of my own, I realise that paster was using wsgiServer, based off SimpleHTTPHandler, which filters off all the requests except HEAD, GET, PUT. There was not much choice but to duplicate a paste server plug-in that allowed these methods. I copied code off wsgiServer and wsgiutils_server, and borrowed a pattern from http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307618 , which uses __getattr__ to allow me to specify a list of supported methods rather than define a separate method to do the same thing - pass request to application. It works now.

Reading the request body

The next thing I found was that reading off environ['wsgi.input'] would end up blocking, since it is a stream that is still open. I will have to check the Content Length variable to get the number of bytes to read, but is there another option to not block_read, in case Content Length is not given? My unfixed code is:

requestbody = ''
if 'wsgi.input' in environ:
....inputstream = environ['wsgi.input']
....readbuffer = inputstream.read(BUF_SIZE)
....while len(readbuffer) != 0:
........requestbody = requestbody + readbuffer
........readbuffer = inputstream.read(BUF_SIZE)


Catching Iterator Code Exceptions
Some of the exceptions thrown were not being caught by my error processing middleware, it was because some of my sub-apps return a iterator object, which is iterated out of the scope of my error processor. for example:


def A():
....for v in iter(B()):
........print v

def B():
....try:
........return C()
....except:
........return ['exception']

def C():
....raise Exception
....yield '1'
....yield '2'


From my old functional perspective, B() should be able to catch the exception in C(), since C is within its scope. In reality, when B is executed, B returns C() immediately as a generator/iterator object, without executing C() itself. B then exits its scope. When A tries to use the iterator object returned, C() runs and throws the exception on A which thought it was protected. So instead I went to look at paste.errormiddleware on how it was catching iterator exceptions and implemented it accordingly.


Current Test Status
-> running `basic':
0. init.................. pass
1. begin................. pass
2. options............... pass
3. put_get............... pass
4. put_get_utf8_segment.. pass
5. mkcol_over_plain...... pass
6. delete................ pass
7. delete_null........... pass
8. delete_fragment....... WARNING: DELETE removed collection resource with Request-URI including fragment; unsafe...................... pass (with 1 warning)
9. mkcol................. pass
10. mkcol_again........... pass
11. delete_coll........... pass
12. mkcol_no_parent....... pass
13. mkcol_with_body....... pass
14. finish................ pass
<- summary for `basic': of 15 tests run: 15 passed, 0 failed. 100.0%
-> 1 warning was issued.

This is the basic test set, following which are sets for copymove, props, locks and http. As of this moment I am still sorting out the copymove errors identified.

2 comments:

Ian Bicking said...

Errors in the iterator do make finalization and error-catching code harder to implement. If you look at, for instance, this example: http://blog.ianbicking.org/more-perfect-app-server-wsgi-transactions.html you'll see how finalization code can be implemented. This might be useful for some of the places where WebDAV wants atomic operations, and so you should probably roll back locks or deletes or whatnot if there's an error.

But then that raises the question -- why did you implement your own error middleware instead of using Paste's? I think it's good to keep this project somewhat separate, since it clarifies and encapsulates your progress, but something like error catching isn't central to what you are doing, and it's better to use the code that is already out there.

cwho said...

Transaction Manager
That's a great suggestion. I had a look at the template and it works great into some transaction situations which I had seen before :)

I would try to put the server operations in this form, perhaps it would put more clearly and deal better with the operation requiring atomicity - MOVE.

Error middleware
I think I described it wrongly using "error" middleware. it does not catch server exceptions, rather, if the application wanted to return a HTTP error code back to the client, it could raise ProcessRequestError(HTTP_NOT_FOUND) and the middleware would catch that and send a proper 404 response back to the client. I did this so that I could use error-checking functions in generator functions more easily (the error checking function throws an exception that causes program control to jump out of the generator scope straight to the error-catcher, rather than the generator checking the return result and then yielding the error message if required).

I probably should have used HttpException instead of defining my own Exception class. Again something I could improve on after I get LOCKing working.