[Twisted-Python] questions about twisted usage

[Tim Allen]

On Thu, Mar 22, 2012 at 02:34:41PM +0200, Uri Okrent wrote:
> Maybe I should clarify my question:
> 
> thing = 0
> def a():
>     thing += 1
>     thing += 1
>     thing += 1
>     thing += 1
>     thing += 1
> 
> def b():
>     thing = 0
> 
> def show_thing():
>     print thing
> 
> a.addCallback(show_thing)
> b.addCallback(show_thing)
> 
> given the two deferreds 'a' and 'b', with 'a' being called first, it
> is possible for twisted to jump to 'b' in the middle of 'a' correct?

No.

The situation above doesn't actually work because a and b don't return
Deferreds, but during the execution of a, it's the only thing running
(unless you have other threads running... but since Twisted is largely
a way of avoiding thread-programming, that's not often the case).

In the real world, things aren't so simple, because nobody writes
Deferreds for code that can just execute in a straight line. People use
Deferreds when they want to wait for some external event before going
further, and you generally don't have control over the order those
events occur in.

For example, here we have a function that does some work, then does
a deferred operation, then does some more work in the callback:

    thing = 0

    def a():
	global thing

	# Do some work.
	thing += 1

	# Cause some_db to do some work on another machine, or in
	# another thread, or something.
	d = some_db.do_work("a")

	def add_more(res):
	    global thing

	    thing += 1
	    thing += 1
	    thing += 1

	    return res

	# When some_db is done, call this callback.
	d.addBoth(add_more)

	# Do more work.
	thing += 1

	return d

    def b():
	d = some_db.do_work("b")

	def clear_thing(res):
	    global thing
	    thing = 0
	    return res

	d.addBoth(clear_thing)
	return d

The pattern of events would go like this:

- Initially, thing is 0.
- Somebody calls a(), which increments thing twice (under the comment
  "do some work" and "do more work", and launches a database call.
- thing is now 2.
- Somebody calls b(), which launches a different database call.
- Sometime later, the database gets back to us with an answer, and
  either a or b's callback will be fired first.
- If a's callback is fired before b's callback, thing will be
  incremented three more times then (some time later) set to 0.
- IF b's callback is fired before a's callback, thing will be set to 0,
  then (some time later) incremented 3 more times, resulting in 3.

So the only possible results are 0 and 3, because the 'synchronisation
points' are at the edges of callbacks. There's no way to get a result of
1 or 2, because a callback can't be interrupted while it's running
(again, unless threads are involved).


Tim.