Part of twisted.web2 View Source
The stream module provides a simple abstraction of streaming data. While Twisted already has some provisions for handling this in its Producer/Consumer model, the rather complex interactions between producer and consumer makes it difficult to implement something like the CompoundStream object. Thus, this API.
The IStream interface is very simple. It consists of two methods: read, and close. The read method should either return some data, None if there is no data left to read, or a Deferred. Close frees up any underlying resources and causes read to return None forevermore.
IByteStream adds a bit more to the API: 1) read is required to return objects conforming to the buffer interface. 2) .length, which may either an integer number of bytes remaining, or None if unknown 3) .split(position). Split takes a position, and splits the stream in two pieces, returning the two new streams. Using the original stream after calling split is not allowed.There are two builtin source stream classes: FileStream and MemoryStream. The first produces data from a file object, the second from a buffer in memory. Any number of these can be combined into one stream with the CompoundStream object. Then, to interface with other parts of Twisted, there are two transcievers: StreamProducer and ProducerStream. The first takes a stream and turns it into an IPushProducer, which will write to a consumer. The second is a consumer which is a stream, so that other producers can write to it.
|Interface||IStream||A stream of arbitrary data.|
|Interface||IByteStream||A stream which is of bytes.|
|Interface||ISendfileableStream||No interface docstring; 1/1 methods documented|
|Class||SimpleStream||Superclass of simple streams with a single buffer and a offset and length into that buffer.|
|Function||mmapwrapper||Python's mmap call sucks and ommitted the "offset" argument for no discernable reason. Replace this with a mmap module that has offset.|
|Class||FileStream||No class docstring; 1/3 methods documented|
|Class||MemoryStream||A stream that reads data from a buffer object.|
|Class||CompoundStream||A stream which is composed of many other streams.|
|Function||readStream||Pass a stream's data to a callback.|
|Function||readAndDiscard||Read all the data from the given stream, and throw it out.|
|Function||readIntoFile||Read a stream and write it into a file.|
|Function||connectStream||Connect a protocol constructed from a factory to stream.|
|Class||ProducerStream||Turns producers into a IByteStream. Thus, implements IConsumer and IByteStream.|
|Class||StreamProducer||A push producer which gets its data by reading a stream.|
|Class||ProcessStreamer||Runs a process hooked up to streams.|
|Function||generatorToStream||Converts a generator function into a stream.|
|Class||BufferedStream||A stream which buffers its data to provide operations like readline and readExactly.|
|Class||_StreamReader||Process a stream's data using callbacks for data and stream finish.|
Pass a stream's data to a callback.Returns Deferred which will be triggered on finish. Errors in reading the stream or in processing it will be returned via this Deferred.
Read all the data from the given stream, and throw it out.Returns Deferred which will be triggered on finish.
Read a stream and write it into a file.Returns Deferred which will be triggered on finish.
Connect a protocol constructed from a factory to stream.
Returns an output stream from the protocol.The protocol's transport will have a finish() method it should call when done writing.
Converts a generator function into a stream.
The function should take an iterator as its first argument, which will be converted *from* a stream by this wrapper, and yield items which are turned *into* the results from the stream's 'read' call.One important point: before every call to input.next(), you *MUST* do a "yield input.wait" first. Yielding this magic value takes care of ensuring that the input is not a deferred before you see it.
>>> from twisted.web2 import stream >>> from string import maketrans >>> alphabet = 'abcdefghijklmnopqrstuvwxyz' >>> >>> def encrypt(input, key): ... code = alphabet[key:] + alphabet[:key] ... translator = maketrans(alphabet+alphabet.upper(), code+code.upper()) ... yield input.wait ... for s in input: ... yield str(s).translate(translator) ... yield input.wait ... >>> encrypt = stream.generatorToStream(encrypt) >>> >>> plaintextStream = stream.MemoryStream('SampleSampleSample') >>> encryptedStream = encrypt(plaintextStream, 13) >>> encryptedStream.read() 'FnzcyrFnzcyrFnzcyr' >>> >>> plaintextStream = stream.MemoryStream('SampleSampleSample') >>> encryptedStream = encrypt(plaintextStream, 13) >>> evenMoreEncryptedStream = encrypt(encryptedStream, 13) >>> evenMoreEncryptedStream.read() 'SampleSampleSample'