|Version 17 (modified by exarkun, 8 years ago) (diff)|
Here are a list of Frequently Asked Questions regarding Twisted.
- I run mktap (from site-packages/twisted/scripts/mktap.py) and nothing happens!
- Why do the Debian packages for Alphas and Release Candidates have weird versions containing old version numbers?
- Why am I getting ImportErrors for Twisted subpackages on OS X 10.5?
- Why am I getting ImportErrors for Twisted subpackages on my 64bit installation of RedHat/Fedora/CentOS?
- Why does Twisted depend on Zope?
- How can I access self.factory from my Protocol's __init__?
- Where can I find out how to write Twisted servers?
- How do I make Twisted talk to multiple clients / connect to multiple servers?
- How do I make input on one connection result in output on another?
- When I try to install my reactor, I get errors about a reactor already being installed. What gives?
- twistd won't load my .tap file! What's this Ephemeral nonsense?
- I get Interrupted system call errors when I use os.popen2. How do I read results from a sub-process in Twisted?
- Why don't my spawnProcess programs see my environment variables?
- My Deferred or DeferredList never fires, so my program just mysteriously hangs! What's wrong?
- My exceptions and tracebacks aren't getting printed!
- How do I use Deferreds to make my blocking code non-blocking?
- I get exceptions.ValueError: signal only works in main thread when I try to run my Twisted program! What's wrong?
- I'm trying to stop my program with sys.exit(), but Twisted seems to catch it! How do I exit my program?
- How do I find out the IP address of the other end of my connection?
- Why don't Twisted's network methods support Unicode objects as well as strings?
- Why can't the Twisted's reactor be restarted?
- What's the best way of integrating Twisted with pygame?
- Perspective Broker
- Requests and Contributing
- Communicating with us
What is Twisted?
See the Twisted Project.
Why should I use Twisted?
See the TwistedAdvantage.
I have a problem getting Twisted.
Did you check the HOWTO collection? There are so many documents there that they might overwhelm you... try starting from the index, reading through the overviews and seeing if there seems to be a chapter which explains what you need to. You can try reading the PostScript or PDF formatted books, inside the distribution. And, remember, the source will be with you... always.
Why are there so many parts and subprojects? Isn't Twisted just Twisted?
As of version 2.0, Twisted was split up into many subprojects, because it was getting too much to handle in a monolithic release, and we believe breaking the project into smaller chunks will help people understand the things they need to understand (There used to be a FAQ entry here asking Why is Twisted so big?). More information is available in the Split FAQ.
Why did the version number go from 2.5 to 8.x?
Twisted has moved to a date based version number scheme. The major version number now indicates the year the release was made in. So all releases made in 2008 are 8.x, all releases in 2009 will be 9.x and so on.
This is similar to, but less confusing than, Ubuntu's date based versioning scheme of YEAR.MONTH. The Twisted versions still counts up from 0 for the minor number and patches.
Does the 1.0 release mean that all of Twisted's APIs are stable?
No, only specific parts of Twisted are stable, i.e. we only promise backwards compatibility for some parts of Twisted. While these APIs may be extended, they will not change in ways that break existing code that uses them.
While other parts of Twisted are not stable, we will however do our best to make sure that there is backwards compatibility for these parts as well. In general, the more the module or package are used, and the closer they are to being feature complete, the more we will concentrate on providing backwards compatibility when API changes take place.
Which parts of Twisted are stable?
Only modules explictily marked as such can be considered stable. Semi-stable modules may change, but not in a large way and some sort of backwards-compatibily will probably be provided. If no comment about API stability is present, assume the module is unstable.
In Twisted 1.1, most of twisted.internet, .cred and .application are completely stable (excepting of course code marked as deprecated).
But as always, the only accurate way of knowing a module's stability is reading the module's docstrings.
I run mktap (from site-packages/twisted/scripts/mktap.py) and nothing happens!
Don't run scripts out of site-packages. The Windows installer should install executable scripts to someplace like C:\Python22\scripts\, *nix installers put them in $PREFIX/bin, which should be in your $PATH.
Why do the Debian packages for Alphas and Release Candidates have weird versions containing old version numbers?
An example: 1.0.6+1.0.7rc1-1
In Debian versioning, 1.0.7rc1 is greater than 1.0.7. This means that if you install a package with Version: 1.0.7rc1, and then that package gets a new version 1.0.7, apt will not upgrade it for you, because 1.0.7 looks like an older version. So, we prefix the previous version to the actual version. 1.0.6+1.0.7rc1 is less than 1.0.7.
Why am I getting ImportErrors for Twisted subpackages on OS X 10.5?
OS X 10.5 comes with a partial installation of Twisted. This version is found first by Python and will hide any other version of Twisted you install. In order to use a different version of Twisted, you need to modify your PYTHONPATH environment variable so that the location of the alternate install is inspected first.
Why am I getting ImportErrors for Twisted subpackages on my 64bit installation of RedHat/Fedora/CentOS?
Red Hat and derived Linux distributions arrange to have certain Python libraries installed in /usr/lib/ and other parts installed in /usr/lib64/. This is problematic for Twisted, since it results in some of Twisted being installed in one directory and the rest installed in another. Python will load a package from only one directory. You can override this default behavior when you install Twisted by supplying the --install-lib parameter when running Twisted's setup.py.
Why does Twisted depend on Zope?
Actually, Twisted depends on Zope Interface, an interface package which is also bundled independently from the main Zope distribution. Twisted uses Zope Interface to define and document APIs. Zope Interface is also packaged by Debian.
How can I access self.factory from my Protocol's __init__?
You can't. A Protocol doesn't have a Factory when it is created. Instead, you should probably be doing that in your Protocol's connectionMade method.
Similarly you shouldn't be doing real work, like connecting to databases, in a Factory's __init__ either. Instead, do that in startFactory.
Where can I find out how to write Twisted servers?
Try Writing Servers.
How do I make Twisted talk to multiple clients / connect to multiple servers?
Twisted already does this. If you want to connect to multiple servers, call connectTCP (or similar) multiple times, and each call will result in a single outgoing connection. listenTCP will result in your factory (and therefore your protocol) being invoked for each incoming connection.
Sometimes people ask this question when they write a function that calls connectTCP, then reactor.run(). You don't usually need to call reactor.run() yourself; let twistd do it. If you do need to call it yourself, call it just once after your initial setup. When reactor.run() exits, your program should too.
How do I make input on one connection result in output on another?
This seems like it's a Twisted question, but actually it's a Python question. Each Protocol object represents one connection; you can call its transport.write to write some data to it. These are regular Python objects; you can put them into lists, dictionaries, or whatever other data structure is appropriate to your application.
As a simple example, add a list to your factory, and in your protocol's connectionMade and connectionLost, add it and remove it to that list. Here's the Python code:
from twisted.internet.protocol import Protocol, Factory from twisted.internet import reactor class MultiEcho(Protocol): def connectionMade(self): self.factory.echoers.append(self) def dataReceived(self, data): for echoer in self.factory.echoers: echoer.transport.write(data) def connectionLost(self, reason): self.factory.echoers.remove(self) class MultiEchoFactory(Factory): protocol = MultiEcho def __init__(self): self.echoers =  reactor.listenTCP(4321, MultiEchoFactory()) reactor.run()
When I try to install my reactor, I get errors about a reactor already being installed. What gives?
Here's the rule - installing a reactor should always be the first thing you do, and I do mean first. Importing other stuff before you install the reactor can break your code.
Tkinter and wxPython support, as they do not install a new reactor, can be done at any point, IIRC.
twistd won't load my .tap file! What's this Ephemeral nonsense?
When the pickled application state cannot be loaded for some reason, it is common to get a rather opaque error like so:
% twistd -f test2.tap Failed to load application: global name 'initRun' is not defined
The rest of the error will try to explain how to solve this problem, but a short comment first: this error is indeed terse -- but there is probably more data available elsewhere -- namely, the twistd.log file. Open it up to see the full exception.
The error might also look like this:
Failed to load application: <twisted.persisted.styles.Ephemeral instance at 0x82450a4> is not safe for unpickling
To load a .tap file, as with any unpickling operation, all the classes used by all the objects inside it must be accessible at the time of the reload. This may require the PYTHONPATH variable to have the same directories as were available when the application was first pickled.
A common problem occurs in single-file programs which define a few classes, then create instances of those classes for use in a server of some sort. If the class is used directly, the name of the class will be recorded in the .tap file as something like __main__.MyProtocol. When the application is reloaded, it will look for the class definition in __main__, which probably won't have it. The unpickling routines need to know the module name, and therefore the source file, from which the class definition can be loaded.
The way to fix this is to import the class from the same source file that defines it: if your source file is called myprogram.py and defines a class called MyProtocol, you will need to do a from myprogram import MyProtocol before (and in the same namespace as) the code that references the MyProtocol class. This makes it important to write the module cleanly: doing an import myprogram should only define classes, and should not cause any other subroutines to get run. All the code that builds the Application and saves it out to a .tap file must be inside an if __name__ == '__main__' clause to make sure it is not run twice (or more).
When you import the class from the module using an external name, that name will be recorded in the pickled .tap file. When the .tap is reloaded by twistd, it will look for myprogram.py to provide the definition of MyProtocol.
Here is a short example of this technique:
# file dummy.py from twisted.internet import protocol class Dummy(protocol.Protocol): pass if __name__ == '__main__': from twisted.application import service, internet a = service.Application("dummy") import dummy f = protocol.Factory() f.protocol = dummy.Dummy # Note! Not "Dummy" internet.TCPServer(2000, f).setServiceParent(a) a.save()
I get Interrupted system call errors when I use os.popen2. How do I read results from a sub-process in Twisted?
Why don't my spawnProcess programs see my environment variables?
spawnProcess defaults to clearing the environment of child processes as a security feature. You can either provide a dictionary with exactly the name-value pairs you want the child to use, or you can simply pass in os.environ to inherit the complete environment.
My Deferred or DeferredList never fires, so my program just mysteriously hangs! What's wrong?
It really depends on what your program is doing, but the most common cause is this: it is firing -- but it's an error, not a success, and you have forgotten to add an errback, so nothing happens. Always add errbacks!
The reason Deferred can't automatically show your errors is because a Deferred can still have callbacks and errbacks added to it even after a result is available -- so we have no reasonable place to put a logging call that wouldn't result in spurious tracebacks that are handled later on. There is a facility for printing tracebacks when the Deferreds are garbage collected -- call defer.setDebugging(True) to enable it.
My exceptions and tracebacks aren't getting printed!
See previous question.
How do I use Deferreds to make my blocking code non-blocking?
You don't. Deferreds don't magically turn a blocking function call into a non-blocking one. A Deferred is just a simple object that represents a deferred result, with methods to allow convenient adding of callbacks. (This is a common misunderstanding; suggestions on how to make this clearer in the Deferred Execution howto are welcome!)
If you have blocking code that you want to use non-blockingly in Twisted, either rewrite it to be non-blocking, or run it in a thread. There is a convenience function, deferToThread, to help you with the threaded approach -- but be sure to read Using Threads in Twisted.
I get exceptions.ValueError: signal only works in main thread when I try to run my Twisted program! What's wrong?
The default reactor, by default, will install signal handlers to catch events like Ctrl-C, SIGTERM, and so on. However, you can't install signal handlers from non-main threads in Python, which means that reactor.run() will cause an error. Pass the installSignalHandlers=0 keyword argument to reactor.run to work around this.
I'm trying to stop my program with sys.exit(), but Twisted seems to catch it! How do I exit my program?
Use reactor.stop() instead. This will cleanly shutdown the reactor.
How do I find out the IP address of the other end of my connection?
The .transport object (which implements the ITransport interface) offers a pair of methods named getPeer and getHost. getPeer will give you a tuple that describes the address of the system at the other end of the connection. For example:
class MyProtocol(protocol.Protocol): def connectionMade(self): print "connection from", self.transport.getPeer()
Why don't Twisted's network methods support Unicode objects as well as strings?
In general, such methods (e.g. FileDescriptor's write) are designed to send bytes over the network. These methods use non-Unicode string objects as a container for the bytes that they send and receive.
Unicode objects are not byte-based and are an abstraction used for representing strings of human readable text. In order to send Unicode strings using these methods, you should explicitly specify a byte-based encoding for them, for example: s.encode("utf-8") and explicitly decode them at the receiving end.
Twisted cannot choose an encoding for you at this level: your encoding choice will be protocol specific and may need to be specified in the message you send (for example, HTTP headers include a encoding specification).
For a more complete discussion of the distinction between Unicode strings and specific encodings of Unicode strings, see the following articles:
- Dan Sugalski's What the heck is: A string; and
- Joel Spolsky's The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!).
Why can't the Twisted's reactor be restarted?
There is no interesting reason. This is mainly a limitation of the current implementation, and a lack of interest on this particular feature. It's just a matter of someone spending the time on it.
What's the best way of integrating Twisted with pygame?
Take a look at game to see an example of the world's best practice for integrating pygame and Twisted.
How can I get the reference to a client from a Perspective?
Firstly, the client must send a reference when it connects to the perspective broker. This can be done by passing the reference as a parameter to pb.connect.
At the server end, you must override the Perspective.attach, which is called when a client attaches to a perspective. The first argument of this method is a remote reference to the client object that was passed to pb.connect.
Requests and Contributing
Twisted is cool, but I need to add more functionality.
Great! Read our docs, and if you're feeling generous, contribute patches.
I have a patch. How do I maximize the chances the Twisted developers will include it?
There are several steps you can take to increase the chances of inclusion into Twisted:
- Be sure you have read and are familiar with the information and linked material here and here.
- Open a ticket for the feature you wish to add.
- Get feedback from others on your ideas and thoughts.
- Write trial tests for your code!
- When providing a patch, use unified diff. Either use svn diff or, better yet, make a clean checkout and use diff -urN between them. Make sure your patch applies cleanly.
- If you post the patch to the mailing list, make sure it is inlined and without any word wrapping.
And to whom do I send it?
My company would love to use Twisted, but it's missing feature X, can you implement it?
You have 3 options:
- Pay one of the Twisted developers to implement the feature.
- Implement the feature yourself.
- Add a feature request to our bug tracker. We will try to implement the feature, but there are no guarantees when and if this will happen.
Twisted really needs documentation for X, Y or Z - how come it's not documented?.
Twisted's documentation is a work in progress, and one that we would appreciate assistance with. If you notice a gap or flaw in the documentation, please file a bug in the Twisted issue tracker and mark it as having topic 'documentation'. Patches appreciated. Unit tests even more so.
Wow the Twisted documentation is nice! I want my docs to look like that too!
Now you can, with Lore.
Communicating with us
There's a bug in Twisted. Where do I report it?
Unless it is a show-stopper bug, we usually won't roll out a new release with a fix it if it's already fixed in [source:trunk Subversion], so check if it is fixed there. If it is not fixed in Subversion, you should add it to the issue tracker, including pertinent information about the bug (hopefully as much information needed to reproduce it: OS, Subversion versions of any important files, Python version, code you wrote or things you did to trigger the bug, etc. If the bug appears to be severe, you should also raise it on the mailing list, with a pointer to the issue already filed in the bug tracker.
Where do I go for help?
Ask for help where the Twisted team hangs out.
How do I e-mail a Twisted developer?
First, note that in many cases this is the wrong thing to do: if you have a question about a part of Twisted, it's usually better to e-mail the mailing list. However, the preferred e-mail addresses for all Twisted developers are listed in the file CREDITS in the Subversion repository.