[Twisted-Python] Twisted and SNMP (was: Advice on doing thousands of simultaneous UDP queries with Twisted...)
Thomas.Favier at accelance.fr
Tue Feb 10 04:03:56 EST 2004
Mike C. Fletcher wrote:
> If for no other reason than that it convinced me there wasn't a simple
> and elegant solution readily available to be dropped in for a UDP system
> :) . In the final analysis, what was happening is that individual
> queries were timing out *before the queries were even sent* because it
> was taking so incredibly long to format and send all of the queries.
You bring my point with SNMP : I've tried to build a SNMP poller based
on twisted and pySNMP. Main trouble i've had is the decoding time of the
pure python ASN parser. It spent about 1 full second decoding (packets
are about 7ko, as i get all values and tables with a GET-BULK, there is
room for improvement by splitting requests, but that's not significant
for that matter). So i switched to a threaded implementation using
yapsnmp (and ugly deferToThreads). Decoding times are about 0.02 seconds
for the same requests. Load on the server was more than halved.
Seems that you are hit by the same kind of problems...
I've dug the source of UCD-SNMP but was not able to see a simple hack to
integrate the UCD packet decoding and the twisted loop, especially with
the parts dealing with SNMPv3 (UCD- and NET-snmp almost forces you to
use their select wrapper...) Moreover, the last yapsnmp release is based
on v4.3 so a little out of date. As my solution works for my (smaller)
needs, i currently have no plan to improve my solution, but if anyone is
interested in bringing a full high performance ASN coder-decoder to
twisted, i'd be willing to help.
Tel. +33 4 26 29 12 22
More information about the Twisted-Python