[Twisted-Python] Deferred in C++

Amaury Forgeot d'Arc amauryfa at gmail.com
Tue Oct 21 12:14:43 EDT 2008


On Sat, Oct 18, 2008 at 22:34, Jamu Kakar <jkakar at kakar.ca> wrote:
> Hi,
> I've implemented Deferred in C++.  There is room for improvement and
> optimization, but the basic core works.

Wonderful! Awesome!
So I am not the only one who tries to use Twisted concepts in C++.

I did write some kind of a Deferred class in C++ myself, together with
a re-implementation of a Twisted framework (two reactors featuring TCP
client & server, callLater, callInThread...; a selectReactor and a
native MFC reactor that merges nicely in a Windows application, yes,
there are some; Protocols, LineReceiver, HttpClient & server...)
and I already have a running business application that uses all of it
on Windows.

The Deferred part was really hard to implement, and may be the reason
why it was never done before. It really pushed me out of my C++
I tried to convert my code to use your class, which seems easier to
use than mine. Great job!

> I'm not sure how practical this component would be in a non-trivial
> application.

Here are random comments about the changes I had to make, when
confronted to a real application using nested deferreds, DeferredList,
various error handling...

- First, good news, your code almost compiles on Windows (VS 2005). I
had to typedef int32_t (why don't you simply use int?), comment out
the StackFrameFactory, and declare some functions/classes with
__declspec(dllexport) (with the help of a macro) so we can build a

- In my eyes, the notation "addCallback" is more readable than
"add_callback" (and it's the one used by twisted python), but that's a
matter of taste. On the other hand I found disturbing at first to fire
deferreds with the methods "succeed()" and "fail()" (instead of
callback() and errback()

- I found that my callbacks often have a void result, and deferred are
sometimes fired without a value. For example, a twisted DeferredList
often just waits for its children to finish. The results may be
gathered somewhere, but the callbacks return None.
I took the liberty to change your code and add *a lot* of template
specializations for the "void" ReturnType or ResultType.
I suggest that when a callback takes no parameter, it can be added to
any Deferred, whatever its current state (and the current value is
simply discarded).

- In some cases my callbacks take additional arguments (the twisted
"callbackArgs"). To make it possible I used the
boost::bind functions, and a wrapper class that converts any
boost::function to a ion::ICallback.

- Several times I had a "Can't add invalid callback" error, and I
found it difficult to figure which type was in the deferred state (the
previous callback returned a Deferred, and so on...) So I extended
ITypeInfo with a "const char* typename" member, that I fill with
"typeid(Object).name()". Much easier to see with the debugger...

- I tried to implement a DeferredList. I almost succeeded, except that
it simply inherit from Deferred and it seems a bad idea: when the
DeferredList is returned as a plain Deferred, all state is lost... and
the application crashes.
Maybe Deferred should be a kind of "shared pointer" to the real
object: DeferredState or DeferredListState

- Errr... I'm not sure I understood the "trule" concept. When do you
use it? What are its semantics regarding ownership? is it similar to

- How is one supposed to get the content of a Failure? I use code like
this, but it seems too long for this simple task:
  void MyProcess::displayError(ion::trule<ion::Failure> failure)
    std::string message = ion::ptr<ion::Failure>(failure)->get_message();
    throw ion::UnhandledFailureError(failure);

- likewise, it should be possible to simply
    throw Failure("Some error message");

> The reliance on templates would negatively affect
> compile time, for one thing.  More importantly, I've found
> deciphering the error messages produced by simple mistakes can be
> tricky.

I confirm: it is.

> Nonetheless, my main motivation was to see if it was
> possible at all and so I'm pleased that it works in the end.

I think my program can be considered as a non-trivial application (I
cannot show its code, though, except for the non-business framework)
It is definitely possible to implement a Twisted framework in C++. The
hard part is to make it nice for the developer. Programming with
callbacks already twists your mind, it becomes very difficult if the
language syntax hides the important part of the code.

The "ion" project seems promising! What extend do you intend to give it?

Amaury Forgeot d'Arc

More information about the Twisted-Python mailing list