TransactionManager
In Openstack we have a lot of code that is creating/allocating some resources (networks, volumes, quotas, etc.) but we don't have a clear way to release such resources in case of failure. Release is currently done manually in "except" block. Such approach leads to more complicated error handling and potential resource leak. Currently Nova and other projects can use such system to improve error handling in many places.
Examples: https://bugs.launchpad.net/nova/+bug/1173413 https://bugs.launchpad.net/nova/+bug/1161657
We could implement a utility that will help us to manage such resources and will allow us to simplify existing error handling and fix resource leak issues. I propose to implement a "transaction" system that will manage (release) such resources in transparent way:
1) Create a decorator that will be used to mark functions that operate with such resources. Example:
@managed_resource(rollback=deallocate_network) def allocate_network(...
Such decorator will make sure that such function is called inside of the function that is marked with 2nd decorator - @transactional
2) transactional decorator will automatically call rollback function for each resource in case of exception in decorated function.
Such transaction system can also be used to explicitly manage db transactions.
Please see implementation draft below:
#!/usr/bin/env python import functools class TransactionManager: CONTEXTS = [] def __init__(self, rollback_for, no_rollback_for): self._rollbacks = [] self._rollback_for = rollback_for self._no_rollback_for = no_rollback_for or set() def __enter__(self): TransactionManager.CONTEXTS.append(self) return self def __exit__(self, exc_type, exc_value, traceback): try: if (exc_value is None or exc_type in self._no_rollback_for or (self._rollback_for is not None and not isinstance(exc_value, tuple(self._rollback_for)))): return # everything is OK for rollback, args, kwargs in self._rollbacks: rollback(*args, **kwargs) except Exception as e: print e finally: TransactionManager.CONTEXTS.pop() def add_rollback(self, rollback, args, kwargs): self._rollbacks.append((rollback, args, kwargs)) @staticmethod def current(): if not TransactionManager.CONTEXTS: raise ValueError('Trying to call method without transaction context') return TransactionManager.CONTEXTS[-1] def transactional(rollback_for=None, no_rollback_for=None): def decorator(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): with TransactionManager(rollback_for, no_rollback_for): return fn(*args, **kwargs) return wrapper return decorator def rollback_required(rollback): def decorator(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): TransactionManager.current().add_rollback(rollback, args, kwargs) return fn(*args, **kwargs) return wrapper return decorator def deallocate_stuff(name, *args): print 'Deallocating stuff - %s' % name @rollback_required(deallocate_stuff) def allocate_stuff(name, other, stuff): print 'Allocating important stuff named - %s' % name @transactional() def run_instance(name): print 'Running %s' % name allocate_stuff('network', 'foo', 'bar') raise TypeError print 'End run' try: run_instance('World') except Exception as e: pass