[Twisted-Python] Consensus on how to handle "MySQL server has gone away"

Glyph glyph at twistedmatrix.com
Fri Oct 5 04:30:04 EDT 2012

Le Oct 4, 2012 à 12:26 PM, E S <electric.or.sharp at gmail.com> a écrit :

> On Wed, Oct 3, 2012 at 3:39 PM, Glyph <glyph at twistedmatrix.com> wrote:
>> Le Oct 3, 2012 à 6:36 AM, Augusto Mecking Caringi <augustocaringi at gmail.com> a écrit :
>>> On Wed, Oct 3, 2012 at 8:27 AM, Itamar Turner-Trauring
>>> <itamar at itamarst.org> wrote:
>>>> On 10/03/2012 04:28 AM, Phil Mayers wrote:
>>>>> I think the behaviour it should be aiming for is clear:
>>>>> 1. Test each connection with "good_sql" before beginning the user
>>>>> interaction/query
>>>> The problem is that this adds latency; this can add up to quite a
>>>> slowdown if your database server is on a remote server and you're doing
>>>> lots of single queries (as opposed to runInteraction).
>>> Hi!
>>> Sometime ago I faced some related problems and I found this code,
>>> "ReconnectingConnectionPool":
>>> http://www.gelens.org/2009/09/13/twisted-connectionpool-revisited/
>>> What are your opinion about it?
>> If there are bugs in Twisted, or important missing features, step 1 should be to file a bug – first, of course, searching for duplicates – at <http://twistedmatrix.com/trac/newticket>.  Especially if the workaround involves calling or overriding some private, internal implementation detail.  If you're going to blog about a hack that fixes the problem for you, that post should link to the ticket, so that Twisted can move forward and provide a good experience for people getting started with it.
>> I hate the idea that step 1 for some poor new Twisted user would be to go searching around a couple dozen external websites to apply undocumented hacks to try to just get something basic, like relational database communication, to work acceptably for their application.
>> So, my opinion is that either this isn't a real problem, in which case you shouldn't use it, or it is a real problem, in which case Jeffrey Gelens and powdahound should open a ticket and explain why it's necessary :).
>> -glyph
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> Well, I did reference two bugs already in the twisted bug tracker that
> I believe are related to this issue (4964 definitely is anyway).

Aah, I'm terribly sorry.  #4964 is indeed this exact issue; I missed the link.

> These issues have been in the bug tracking system for years now so I'm not holding my breath on fixes any time soon.

Well, someone who cares will then need to fix the issue for real :).

> I appreciate your desire to make twisted as good as it can be, but we all have schedules to meet and hacks, documented or not, are sometimes necessary to move the project along. So I definitely will consider input from people who have run into this situation and found a solution that has worked for them.

Of course.  But you should be aware that this is how issues remain open in the bug tracker for years - they only get fixed when people fix them :).

> If you're not convinced it's a real problem, it's pretty easy to recreate. Assuming your MySQL wait_timeout = 2, you can run the following code and get the (2006, 'MySQL server has gone away') error:

No, I'm perfectly convinced it's a real problem.  The description on #4964 was very helpful.

> 4. Man up and fix adbapi myself

While I might disagree with the wording, this is what I'd suggest :).

> I'm leaning towards #1 since it seems to be least invasive, but I am open to other opinions.

I think you may be overestimating the difficulty involved in just fixing the bug for real.  The conflict between schedule and fixing bugs in the right place - i.e. in Twisted - is that there is some waiting involved, not that fixing the bug is super hard.  You have to wait for a code review, then wait for a release.  But, assuming your fix is not terribly invasive, you can just do this:

Fix the bug.
Submit your fix for review.
While you're waiting, of course, you need to ship some software, so ship a modified copy of the modified module with your package and hot-patch it at runtime, assuming the fix is not already applied.  You can see examples of other projects (that I've worked on) doing that here:<http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Epsilon/epsilon/hotfix.py> and here:<http://trac.calendarserver.org/browser/CalendarServer/trunk/twext/patches.py>.
Respond promptly to any review feedback, when it becomes available, and update your local copy.

When a new version of Twisted ships with the bug fixed, and you can update to depend on it, just delete your local copy.  Assuming that the final fix is reasonably close to the one that you implemented previously, even older versions of your software should keep working.

I hope you do elect to help out with this, we can always use more contributors :).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20121005/b068496c/attachment.htm 

More information about the Twisted-Python mailing list