harald/blog
1
0
Fork 0
Code Activity

initial commit

Browse source
This commit is contained in:
Harald Hoyer 2020-05-19 18:20:18 +02:00
commit 429a176559
59 changed files with 4097 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
content/2015/10/13/_index.md Normal file
View file

@ -0,0 +1,5 @@
+++
transparent = true
sort_by = "date"
paginate_by = 5
+++

357
content/2015/10/13/a-python-transaction-class.md Normal file
View file

@ -0,0 +1,357 @@
+++
title = "A Python Transaction Class"
date = 2015-10-13T10:20:43+00:00
+++
This is a repost of a blog post of 2008. Just for the reference :-)
This class allows sub-classes to commit changes to an instance to a history, and rollback to previous states.
<!-- more -->
The final class with an extension for `__setstate__` and `__getstate__` can be found here:
[transaction.py](/files/transaction.py)
and [transaction_test.py](/files/transaction_test.py).
Now to the story, that led to it: I had the need for a transaction class in python and browsing the python cookbook led me to a [small Transaction class](http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/551788).
```python
$ python
Python 2.5.1 (r251:54863, Oct 30 2007, 13:45:26)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Transaction(object):
... def __init__(self):
... self.log = []
... def commit(self):
... self.log.append(self.__dict__.copy())
... def rollback(self):
... try:
... self.__dict__.update(self.log.pop(-1))
... except IndexError:
... pass
...
```
Ok, lets have some fun with it.
```python
>>> class A(Transaction):
... pass
...
>>> a = A()
>>> a.test = True
>>> a.commit()
>>> a.test = False
>>> a
'self.__dict__ = {'test': False, 'log': [{'test': True, 'log': [...]}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': True, 'log': []}'
```
Nice. Let's see if we can commit and rollback several times.
```python
>>> a = A()
>>> a.test = 1
>>> a.commit()
>>> a.test = 2
>>> a.commit()
>>> a.test = 3
>>> a
'self.__dict__ = {'test': 3, 'log': [{'test': 1, 'log': [...]}, {'test': 2, 'log': [...]}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': 2, 'log': [{'test': 1, 'log': [...]}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': 1, 'log': []}'
```
Ok.. works :) Let's try some lists.
```python
>>> a = A()
>>> a.test = [ 0, 1 ]
>>> a.commit()
>>> a.test.append(2)
>>> a
'self.__dict__ = {'test': [0, 1, 2], 'log': [{'test': [0, 1, 2], 'log': [...]}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': [0, 1, 2], 'log': []}'
```
Doh! Ok, someone mentioned that already in the comments. copy.deepcopy() is the key.
```python
>>> import copy
>>>
>>> class Transaction2(Transaction):
... def commit(self, **kwargs):
... self.log.append(copy.deepcopy(self.__dict__))
...
>>> class A(Transaction2):
... pass
...
>>> a = A()
>>> a.test = [ 0, 1 ]
>>> a.commit()
>>> a.test.append(2)
>>> a
'self.__dict__ = {'test': [0, 1, 2], 'log': [{'test': [0, 1], 'log': []}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': [0, 1], 'log': []}'
```
Ah, works. Very good. Now another check:
```python
>>> a = A()
>>> a.test = 1
>>> a.commit()
>>> a
'self.__dict__ = {'test': 1, 'log': [{'test': 1, 'log': []}]}'
>>> a.other = 2
>>> a
'self.__dict__ = {'test': 1, 'other': 2, 'log': [{'test': 1, 'log': []}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': 1, 'other': 2, 'log': []}'
>>> a.other
2
```
Oh, a leftover... seems like `self.__dict__` has to be cleared, before the update.
```python
>>> class Transaction3(Transaction2):
... def rollback(self, **kwargs):
... try:
... state = self.log.pop(-1)
... self.__dict__.clear()
... self.__dict__.update(state)
... except IndexError:
... pass
...
>>> class A(Transaction3):
... pass
...
>>> a = A()
>>> a.test = 1
>>> a.commit()
>>> a
'self.__dict__ = {'test': 1, 'log': [{'test': 1, 'log': []}]}'
>>> a.other = 2
>>> a
'self.__dict__ = {'test': 1, 'other': 2, 'log': [{'test': 1, 'log': []}]}'
>>> a.rollback()
>>> a
'self.__dict__ = {'test': 1, 'log': []}'
>>> a.other
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'A' object has no attribute 'other'
>>>
```
Ah, works. Very good. Ok, more tests...
```python
>>> b = a.ncls
>>> a.ncls.test = True
>>> a.commit()
>>> a.ncls.test = False
>>> a.rollback()
>>> a
'self.__dict__ = {'ncls': 'self.__dict__ = {'test': True, 'log': []}', 'log': []}'
>>> b
'self.__dict__ = {'test': False, 'log': []}'
```
Oh, what if we work with "b", which stills holds the old value? Maybe we should commit() all our attributes also, traversing through them all? Ok, here is such a beast:
```python
>>> class TransactionNew1(object):
... def _docommit(self):
... if "log" not in self.__dict__:
... self.__dict__["log"] = list()
...
... self.__dict__["log"].append(copy.deepcopy(self.__dict__))
...
... def _dorollback(self):
... if "log" not in self.__dict__:
... return
... try:
... state = self.__dict__["log"].pop(-1)
... self.__dict__.clear()
... self.__dict__.update(state)
... except IndexError:
... pass
...
... def commit(self, **kwargs):
... # commit ourselves, then our childs
... self._docommit()
... if kwargs.get("deep", True):
... for child in self.__dict__.values():
... if isinstance(child, self.__class__):
... child.commit()
...
... def rollback(self, **kwargs):
... # rollback our childs, then ourselves
... if kwargs.get("deep", True):
... for child in self.__dict__.values():
... if isinstance(child, self.__class__):
... child.rollback()
... self._dorollback()
...
... def __repr__(self):
... return "'self.__dict__ = %s'" % self.__dict__
...
>>> class A(TransactionNew1):
... pass
...
>>> a = A()
>>> a.ncls = A()
>>> b = a.ncls
>>> a.ncls.test = True
>>> a.commit()
>>> a.ncls.test = False
>>> a.rollback()
>>> a
'self.__dict__ = {'ncls': 'self.__dict__ = {'test': True}', 'log': []}'
>>> b
'self.__dict__ = {'test': True, 'log': []}'
```
Ok... looks good, but we lost the reference. id(b) != id(a.ncls) ... ( update: this is fixed in the final version ) Working with it revealed also:
```python
>>> a = A()
>>> b = a
>>> for i in xrange(3):
... b.n = A()
... b.t = "test"
... b = b.n
... a.commit()
...
>>> a
'self.__dict__ = {'log': [{'n': 'self.__dict__ = {}', 'log': [], 't': 'test'}, {'n': 'self.__dict__ = {'log': [{'log': []}], 't': 'test', 'n': 'self.__dict__ = {}'}', 'log': [{'t': 'test', 'log': [], 'n': 'self.__dict__ = {}'}], 't': 'test'}, {'n': 'self.__dict__ = {'log': [{'log': []}, {'log': [{'log': []}], 't': 'test', 'n': 'self.__dict__ = {}'}], 't': 'test', 'n': 'self.__dict__ = {'log': [{'log': []}], 't': 'test', 'n': 'self.__dict__ = {}'}'}', 'log': [{'t': 'test', 'log': [], 'n': 'self.__dict__ = {}'}, {'t': 'test', 'log': [{'n': 'self.__dict__ = {}', 't': 'test', 'log': []}], 'n': 'self.__dict__ = {'t': 'test', 'log': [{'log': []}], 'n': 'self.__dict__ = {}'}'}], 't': 'test'}], 't': 'test', 'n': 'self.__dict__ = {'t': 'test', 'log': [{'log': []}, {'n': 'self.__dict__ = {}', 't': 'test', 'log': [{'log': []}]}, {'n': 'self.__dict__ = {'log': [{'log': []}], 't': 'test', 'n': 'self.__dict__ = {}'}', 't': 'test', 'log': [{'log': []}, {'log': [{'log': []}], 't': 'test', 'n': 'self.__dict__ = {}'}]}], 'n': 'self.__dict__ = {'t': 'test', 'log': [{'log': []}, {'n': 'self.__dict__ = {}', 't': 'test', 'log': [{'log': []}]}], 'n': 'self.__dict__ = {'log': [{'log': []}]}'}'}'}'
>>> len(str(a))
1192
```
Hmm... seems strange.. Ah, self.log was also copied with copy.deepcopy(). So, we have multiple useless copies. Let's "pop()" the state from self.__dict__ before the deepcopy.
```python
>>> class TransactionNew2(TransactionNew1):
... def _docommit(self):
... if "log" in self.__dict__:
... oldstate = self.__dict__.pop("log")
... else:
... oldstate = None
... state = copy.deepcopy(self.__dict__)
... if oldstate:
... state["log"] = oldstate
... self.__dict__["log"] = state
... def _dorollback(self):
... if "log" not in self.__dict__:
... return
... try:
... state = self.__dict__["log"]
... self.__dict__.clear()
... self.__dict__.update(state)
... except IndexError:
... pass
...
>>> class A(TransactionNew2):
... pass
...
>>>
>>> a = A()
>>> b = a
>>> for i in xrange(3):
... b.n = A()
... b.t = "test"
... b = b.n
... a.commit()
...
>>> a
'self.__dict__ = {'log': {'log': {'log': {'t': 'test', 'n': 'self.__dict__ = {}'}, 't': 'test', 'n': 'self.__dict__ = {'log': {}, 't': 'test', 'n': 'self.__dict__ = {}'}'}, 't': 'test', 'n': 'self.__dict__ = {'log': {'t': 'test', 'n': 'self.__dict__ = {}'}, 't': 'test', 'n': 'self.__dict__ = {'log': {}, 't': 'test', 'n': 'self.__dict__ = {}'}'}'}, 't': 'test', 'n': 'self.__dict__ = {'t': 'test', 'log': {'log': {'t': 'test', 'n': 'self.__dict__ = {}'}, 't': 'test', 'n': 'self.__dict__ = {'log': {}, 't': 'test', 'n': 'self.__dict__ = {}'}'}, 'n': 'self.__dict__ = {'t': 'test', 'log': {'t': 'test', 'n': 'self.__dict__ = {}'}, 'n': 'self.__dict__ = {'log': {}}'}'}'}'
>>> len(str(a))
671
```
Ok, saved us a bit of state length. The final version has:
```python
>>> a
'self.__dict__ = {'__l': {'__l': {'__l': {'t': 'test'}, 't': 'test'}, 't': 'test'}, 't': 'test', 'n': 'self.__dict__ = {'__l': {'__l': {'t': 'test'}, 't': 'test'}, 't': 'test', 'n': 'self.__dict__ = {'__l': {'t': 'test'}, 't': 'test', 'n': 'self.__dict__ = {'__l': {}}'}'}'}'
>>> len(str(a))
275
```
Now another thing:
```python
>>> a = A()
>>> a.n = a
>>> a.commit()
File "/usr/lib64/python2.5/copy.py", line 162, in deepcopy
y = copier(x, memo)
RuntimeError: maximum recursion depth exceeded
```
... Oh, oh! Recursion in commit()... Now we have to check, if we have been there.
```python
>>> class TransactionNew3(TransactionNew2):
... def _checksetseen(self, seen):
... if id(self) in seen:
... import sys
... sys.stderr.write("Recursion detected... \n")
... return True
... seen.add(id(self))
... return False
...
... def commit(self, **kwargs): # pylint: disable-msg=W0613
... seen = kwargs.get("_commit_seen", set())
... if self._checksetseen(seen):
... return
... # commit ourselves, then our childs
... self._docommit()
... if kwargs.get("deep", True):
... for child in self.__dict__.values():
... if isinstance(child, self.__class__):
... child.commit(_commit_seen = seen)
...
... def rollback(self, **kwargs):
... seen = kwargs.get("_rollback_seen", set())
... if self._checksetseen(seen):
... return
... # rollback our childs, then ourselves
... if kwargs.get("deep", True):
... for child in self.__dict__.values():
... if isinstance(child, self.__class__):
... child.rollback(_rollback_seen = seen)
... self._dorollback()
...
>>>
>>> class A(TransactionNew3):
... pass
...
>>> a = A()
>>> a.n = a
>>> a.commit()
Recursion detected...
>>>
```
The final class with an extension for `__setstate__` and `__getstate__` can be found here: [transaction.py](https://harald.hoyer.xyz/files/transaction.py) and [transaction_test.py](/files/transaction_test.py). Have fun with it :-)