twisted.python._initgroups import causes segmentation fault in embedded system

[alberto]

the following line from twisted.web import xmlrpc, server, resource causes the python interpreter segmentation faults

Python 2.6.6 (r266:84292, Feb 15 2011, 17:03:16) 
[GCC 4.2.4 (STMicroelectronics/Linux Base 4.2.4-71)] on linux2
>>>
>>> from twisted.web import xmlrpc, server, resource
Segmentation fault
root@spear_tih01:/workarea/test# 

[exarkun]

Can you track this down to a more specific cause? It's not very likely that anyone else has access to such a system to try to reproduce this.

[alberto]

Replying to exarkun:

Can you track this down to a more specific cause? It's not very likely that anyone else has access to such a system to try to reproduce this.

What can I do ? Feel free to ask for more specific test and I will provide whatever you want. The system is a Spear300 board from STMicroelectronics, the python interpreter is 2.6.6 cross compiled by me for the board. I have installed twisted 10.1.0 on the board and the result is the bug I've commented at the beginning of this ticket.

[michaelnt]

If you only import python code and this causes a segfault then this sounds like a problem with python, your compilation of python, or your platform.

This link has some info on debugging python  http://wiki.python.org/moin/DebuggingWithGdb.

Does this platform have sufficient RAM?

[alberto]

Replying to michaelnt:

If you only import python code and this causes a segfault then this sounds like a problem with python, your compilation of python, or your platform. This link has some info on debugging python  http://wiki.python.org/moin/DebuggingWithGdb. Does this platform have sufficient RAM?

This platform has 128 MB of RAM

[alberto]

This platform has 128 MB of RAM Which is the minimal amount of RAM you need to run ?

[itamar]

In particular, a gdb stack trace from the core file would be helpful.

[alberto]

Replying to itamar:

In particular, a gdb stack trace from the core file would be helpful.

0x00000584 in ?? () (gdb) bt #0 0x00000584 in ?? () #1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbe847d24,

env=0xbe847d30) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3f4b38, argc=-1098613468,

argv=0xbe847d30, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516 #4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbe831ef0,

mallocedp=0xbe831eec, operate=0xbe831ef7, args=0xbe831ec8) at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2, argv=0xbe847d24, env=0xbe847d30) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2 #7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x178bcc,

mallocedp=0x178bd0, operate=0x178bc8, args=0xbe832130) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2 #9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2 #10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0, pathname=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", fp=0x3f49c8) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

[alberto]

0x00000584 in ?? () (gdb) bt #0 0x00000584 in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbe847d24,

env=0xbe847d30) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3f4b38, argc=-1098613468,

argv=0xbe847d30, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbe831ef0,

mallocedp=0xbe831eec, operate=0xbe831ef7, args=0xbe831ec8) at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2, argv=0xbe847d24, env=0xbe847d30) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

#7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x178bcc,

mallocedp=0x178bd0, operate=0x178bc8, args=0xbe832130) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2

#9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2

#10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0, pathname=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", fp=0x3f49c8) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

[michaelnt]

_initgroups made it into python 2.7 here  http://hg.python.org/cpython/rev/a54c19802acc/ so might be an idea to try that version.

Perhaps your platform defines gid as a different size than the code in twisted expects

You could try compiling and running this code to check

#include <grp.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>


int main() {
  unsigned int igid;
  gid_t gid;

  printf("sizeof(igid) = %d\n", sizeof(igid));
  printf("sizeof(gid) = %d\n", sizeof(gid));

  return 0;
}

[alberto]

Replying to michaelnt:

_initgroups made it into python 2.7 here  http://hg.python.org/cpython/rev/a54c19802acc/ so might be an idea to try that version. Perhaps your platform defines gid as a different size than the code in twisted expects You could try compiling and running this code to check {{{ #include <grp.h> #include <sys/types.h> #include <unistd.h> #include <stdio.h> int main() { unsigned int igid; gid_t gid; printf("sizeof(igid) = %d\n", sizeof(igid)); printf("sizeof(gid) = %d\n", sizeof(gid)); return 0; } }}}

this piece of software does not compile on my platform

[alberto]

Sorry my fault:

this piece of software compiles and executes:

root@spear_tih01:/workarea/test# ./testtwisted sizeof(igid) = 4 sizeof(gid) = 4

[alberto]

root@spear_tih01:/workarea/test# ./testtwisted

sizeof(igid) = 4

sizeof(gid) = 4

[exarkun]

Does just importing twisted.python._initgroups cause the segfault? Or does a function in it need to be called before there is a problem? (It sounds like just importing it is enough of a problem, but I'm not sure)

[alberto]

import twisted.python._initgroups

Segmentation fault

It is enough import twisted.python._initgroups to have the segmentation fault

[itamar]

1. I've updated the summary.

2. Note that as a workaround you could just delete the C module.

3. Python 2.7 includes a version of this code - does importing the os module in Python 2.7 cause this problem? If it imports, does os.initgroups exist? If it imports and the function exists, we should take a look at how that code is different than ours as a potential source for a fix.

[alberto]

2. Where can I just delete the C module ?

3. I can't cross compile 2.7 for the board so this test couldn't be performed. Unfortunately the cross-compilation process does not support python 2.7

Cheers

Alberto

[alberto]

If I move _initgroups.c out and then I run > python setup.py install this is the message i obtainfrom the native compiler: arm-linux-gcc: twisted/python/_initgroups.c: No such file or directory

[itamar]

You'll want to delete twisted/python/_initgroups.so from the installed version.

[alberto]

from twisted.web import xmlrpc, server, resource

causes again segmentation faults also if I use the workaround of deleting twisted/python/_initgroups.so from the installed version

[michaelnt]

Can you provide a strack trace of this new segfault.

[alberto]

the new back trace:

#0 0x00000658 in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbef2ed34,

env=0xbef2ed40) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x570768, argc=-1091375820,

argv=0xbef2ed40, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbef15b90,

mallocedp=0xbef15b8c, operate=0xbef15b97, args=0xbef15b68) at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbef160af "/workarea/_install/lib/python2.6/site-packages/twisted/internet/_sigchld.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2, argv=0xbef2ed34, env=0xbef2ed40) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

[michaelnt]

Is it time for

find . -iname '*.so' -exec rm {} \;

It looks like you have a problem with the way you're compiling these extension modules are you sure you're using the correct compiler?

For the record I can run 'from twisted.web import xmlrpc, server, resource' on my N900 without a problem, which has an ARMv7.

[michaelnt]

Checked to see what Twisted C modules are compiled on the N900 and the module in question isn't included, which is presumably because its running Twisted 8.2.0. So there might be a problem on ARM, (contrary to my previous comment).

[alberto]

If I use Twisted 8.2.0 I can't remove the segmentation fault. I still obtain Segmentation Fault and this is the Backtrace:

#0 0x000006ec in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbedbcd34,

env=0xbedbcd40) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3347b8, argc=-1092891340,

argv=0xbedbcd40, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbedb3c70,

mallocedp=0xbedb3c6c, operate=0xbedb3c77, args=0xbedb3c48) at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbedb418f "/workarea/_install/lib/python2.6/site-packages/twisted/protocols/_c_urlarg.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc argv=0xbedbcd34, env=0xbedbcd40) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

#7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x1cb1b4,

mallocedp=0x1cb1b8, operate=0x1cb1b0, args=0xbedb3eb0) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2

#9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2

#10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0,

pathname=0xbedb418f "/workarea/_install/lib/python2.6/site-packages/twisted/protocols/_c_urlarg.so", fp=0x32eec0) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

[itamar]

Given that all extension modules you're compiling are failing, this suggests the problem is with the compilation process... Since you also built Python, which has its own extension modules that do appear to work, you should compare the compiler/linker flags being used there as opposed to what is being used when twisted is built.

[alberto]

Where can I change some flags used building Twisted ? I mean which file of your distribution specify the flags to be used by the compiler/linker building Twisted ?

[itamar]

When you run the appropriate command with setup.py (e.g. "setup.py build_ext" or "setup.py install") it calls gcc. You should make sure that is using the cross compiler, for one thing! But if it is, you can probably see what gcc calls it is doing, and do those manually. Also there's some command line arguments or env variables you can set to override compilation:

 http://docs.python.org/install/index.html#building-extensions-tips-and-tricks