[Twisted-Python] yielding from within deferToThread

[Naveen Michaud-Agrawal]

I'm trying to use sqlalchemy from a twisted application (by running all
blocking queries using deferToThread). Is it possible to yield from within
the function running in deferToThread? For example:


def threadRunQuery(engine, query):
    conn = engine.connect()
    res = conn.execute(query)

    while True:
        results = res.fetchmany(1000)
        if not results:
            break
        yield results


@defer.inlineCallbacks
def stream_results():
    engine = sqlalchemy.create_engine(...)
    query = "select * from table"
    result_iter = yield threads.deferToThread(threadRunQuery, engine, query)
    for results in result_iter:
        print results


It seems that the thread returns a generator, and so everything within
threadRunQuery is actually running on the main reactor thread. Is there
anyway to stream back results from a deferToThread?

Regards
Naveen Michaud-Agrawal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20151203/59cffe83/attachment.html>