Opened 4 years ago

Last modified 4 years ago

#4931 defect new

twisted.python._initgroups import causes segmentation fault in embedded system

Reported by: alberto Owned by: alberto
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch:
Author: Launchpad Bug:

Description (last modified by exarkun)

the following line
from twisted.web import xmlrpc, server, resource
causes the python interpreter segmentation faults

Python 2.6.6 (r266:84292, Feb 15 2011, 17:03:16) 
[GCC 4.2.4 (STMicroelectronics/Linux Base 4.2.4-71)] on linux2
>>>
>>> from twisted.web import xmlrpc, server, resource
Segmentation fault
root@spear_tih01:/workarea/test# 

Change History (28)

comment:1 follow-up: Changed 4 years ago by exarkun

  • Owner set to alberto

Can you track this down to a more specific cause? It's not very likely that anyone else has access to such a system to try to reproduce this.

comment:2 Changed 4 years ago by exarkun

  • Description modified (diff)

comment:3 in reply to: ↑ 1 ; follow-up: Changed 4 years ago by alberto

Replying to exarkun:

Can you track this down to a more specific cause? It's not very likely that anyone else has access to such a system to try to reproduce this.

What can I do ? Feel free to ask for more specific test and I will provide whatever you want.
The system is a Spear300 board from STMicroelectronics, the python interpreter is 2.6.6 cross compiled by me for the board. I have installed twisted 10.1.0 on the board and the result is the bug I've commented at the beginning of this ticket.

comment:4 in reply to: ↑ 3 ; follow-up: Changed 4 years ago by michaelnt

If you only import python code and this causes a segfault then this sounds like a problem with python, your compilation of python, or your platform.

This link has some info on debugging python http://wiki.python.org/moin/DebuggingWithGdb.

Does this platform have sufficient RAM?

comment:5 in reply to: ↑ 4 ; follow-up: Changed 4 years ago by alberto

Replying to michaelnt:

If you only import python code and this causes a segfault then this sounds like a problem with python, your compilation of python, or your platform.

This link has some info on debugging python http://wiki.python.org/moin/DebuggingWithGdb.

Does this platform have sufficient RAM?

This platform has 128 MB of RAM

comment:6 in reply to: ↑ 5 Changed 4 years ago by alberto

This platform has 128 MB of RAM
Which is the minimal amount of RAM you need to run ?

comment:7 follow-up: Changed 4 years ago by itamar

In particular, a gdb stack trace from the core file would be helpful.

comment:8 in reply to: ↑ 7 ; follow-up: Changed 4 years ago by alberto

Replying to itamar:

In particular, a gdb stack trace from the core file would be helpful.

0x00000584 in ?? ()
(gdb) bt
#0 0x00000584 in ?? ()
#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbe847d24,

env=0xbe847d30) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3f4b38, argc=-1098613468,

argv=0xbe847d30, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516
#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbe831ef0,

mallocedp=0xbe831eec, operate=0xbe831ef7, args=0xbe831ec8)
at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2,
argv=0xbe847d24, env=0xbe847d30) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2
#7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x178bcc,

mallocedp=0x178bd0, operate=0x178bc8, args=0xbe832130) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2
#9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2
#10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0,
pathname=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", fp=0x3f49c8) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

comment:9 in reply to: ↑ 8 Changed 4 years ago by alberto

0x00000584 in ?? ()
(gdb) bt
#0 0x00000584 in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbe847d24,

env=0xbe847d30) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3f4b38, argc=-1098613468,

argv=0xbe847d30, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbe831ef0,

mallocedp=0xbe831eec, operate=0xbe831ef7, args=0xbe831ec8)
at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2,
argv=0xbe847d24, env=0xbe847d30) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

#7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x178bcc,

mallocedp=0x178bd0, operate=0x178bc8, args=0xbe832130) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2

#9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2

#10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0,
pathname=0xbe83240f "/workarea/_install/lib/python2.6/site-packages/twisted/python/_initgroups.so", fp=0x3f49c8) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

comment:10 follow-up: Changed 4 years ago by michaelnt

_initgroups made it into python 2.7 here http://hg.python.org/cpython/rev/a54c19802acc/ so might be an idea to try that version.

Perhaps your platform defines gid as a different size than the code in twisted expects

You could try compiling and running this code to check

#include <grp.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>


int main() {
  unsigned int igid;
  gid_t gid;

  printf("sizeof(igid) = %d\n", sizeof(igid));
  printf("sizeof(gid) = %d\n", sizeof(gid));

  return 0;
}

comment:11 in reply to: ↑ 10 ; follow-up: Changed 4 years ago by alberto

Replying to michaelnt:

_initgroups made it into python 2.7 here http://hg.python.org/cpython/rev/a54c19802acc/ so might be an idea to try that version.

Perhaps your platform defines gid as a different size than the code in twisted expects

You could try compiling and running this code to check

#include <grp.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>


int main() {
  unsigned int igid;
  gid_t gid;

  printf("sizeof(igid) = %d\n", sizeof(igid));
  printf("sizeof(gid) = %d\n", sizeof(gid));

  return 0;
}

this piece of software does not compile on my platform

comment:12 in reply to: ↑ 11 Changed 4 years ago by alberto

Sorry my fault:

this piece of software compiles and executes:

root@spear_tih01:/workarea/test# ./testtwisted
sizeof(igid) = 4
sizeof(gid) = 4

comment:13 Changed 4 years ago by alberto

root@spear_tih01:/workarea/test# ./testtwisted

sizeof(igid) = 4

sizeof(gid) = 4

comment:14 follow-up: Changed 4 years ago by exarkun

Does just importing twisted.python._initgroups cause the segfault? Or does a function in it need to be called before there is a problem? (It sounds like just importing it is enough of a problem, but I'm not sure)

comment:15 in reply to: ↑ 14 Changed 4 years ago by alberto

import twisted.python._initgroups

Segmentation fault

It is enough import twisted.python._initgroups to have the segmentation fault

comment:16 follow-up: Changed 4 years ago by itamar

  • Keywords python removed
  • Summary changed from twisted responsible for segmentation fault in embedded systems to twisted.python._initgroups import causes segmentation fault in embedded system
  1. I've updated the summary.
  1. Note that as a workaround you could just delete the C module.
  1. Python 2.7 includes a version of this code - does importing the os module in Python 2.7 cause this problem? If it imports, does os.initgroups exist? If it imports and the function exists, we should take a look at how that code is different than ours as a potential source for a fix.

comment:17 in reply to: ↑ 16 ; follow-up: Changed 4 years ago by alberto

  1. Where can I just delete the C module ?
  1. I can't cross compile 2.7 for the board so this test couldn't be performed. Unfortunately the cross-compilation process does not support python 2.7

Cheers

Alberto

comment:18 in reply to: ↑ 17 Changed 4 years ago by alberto

If I move _initgroups.c out and then I run > python setup.py install
this is the message i obtainfrom the native compiler:
arm-linux-gcc: twisted/python/_initgroups.c: No such file or directory

comment:19 follow-up: Changed 4 years ago by itamar

You'll want to delete twisted/python/_initgroups.so from the installed version.

comment:20 in reply to: ↑ 19 ; follow-up: Changed 4 years ago by alberto

from twisted.web import xmlrpc, server, resource

causes again segmentation faults also if I use the workaround of deleting twisted/python/_initgroups.so from the installed version

comment:21 Changed 4 years ago by michaelnt

Can you provide a strack trace of this new segfault.

comment:22 in reply to: ↑ 20 Changed 4 years ago by alberto

the new back trace:

#0 0x00000658 in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbef2ed34,

env=0xbef2ed40) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x570768, argc=-1091375820,

argv=0xbef2ed40, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbef15b90,

mallocedp=0xbef15b8c, operate=0xbef15b97, args=0xbef15b68)
at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbef160af "/workarea/_install/lib/python2.6/site-packages/twisted/internet/_sigchld.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc=2,
argv=0xbef2ed34, env=0xbef2ed40) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

comment:23 Changed 4 years ago by michaelnt

Is it time for

find . -iname '*.so' -exec rm {} \;

It looks like you have a problem with the way you're compiling these extension modules are you sure you're using the correct compiler?

For the record I can run 'from twisted.web import xmlrpc, server, resource' on my N900 without a problem, which has an ARMv7.

comment:24 follow-up: Changed 4 years ago by michaelnt

Checked to see what Twisted C modules are compiled on the N900 and the module in question isn't included, which is presumably because its running Twisted 8.2.0. So there might be a problem on ARM, (contrary to my previous comment).

comment:25 in reply to: ↑ 24 Changed 4 years ago by alberto

If I use Twisted 8.2.0 I can't remove the segmentation fault. I still obtain Segmentation Fault and this is the Backtrace:

#0 0x000006ec in ?? ()

#1 0x4000cb7c in call_init (l=<value optimized out>, argc=2, argv=0xbedbcd34,

env=0xbedbcd40) at dl-init.c:85

#2 0x4000cc8c in _dl_init (main_map=0x3347b8, argc=-1092891340,

argv=0xbedbcd40, env=0x4) at dl-init.c:134

#3 0x40010ea4 in dl_open_worker (a=<value optimized out>) at dl-open.c:516

#4 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0xbedb3c70,

mallocedp=0xbedb3c6c, operate=0xbedb3c77, args=0xbedb3c48)
at dl-error.c:178

#5 0x40010338 in _dl_open (

file=0xbedb418f "/workarea/_install/lib/python2.6/site-packages/twisted/protocols/_c_urlarg.so", mode=-2147483646, caller_dlopen=0xcae58, nsid=-2, argc
argv=0xbedbcd34, env=0xbedbcd40) at dl-open.c:596

#6 0x40040ac8 in dlopen_doit () from /lib/libdl.so.2

#7 0x4000c7c0 in _dl_catch_error (objname=0x4001bd64, errstring=0x1cb1b4,

mallocedp=0x1cb1b8, operate=0x1cb1b0, args=0xbedb3eb0) at dl-error.c:178

#8 0x40040f74 in _dlerror_run () from /lib/libdl.so.2

#9 0x400409e0 in dlopen@@GLIBC_2.4 () from /lib/libdl.so.2

#10 0x000cae58 in _PyImport_GetDynLoadFunc (fqname=<value optimized out>,

shortname=0x0,

pathname=0xbedb418f "/workarea/_install/lib/python2.6/site-packages/twisted/protocols/_c_urlarg.so", fp=0x32eec0) at Python/dynload_shlib.c:130

#11 0x000b77dc in _PyImport_LoadDynamicModule (

comment:26 follow-up: Changed 4 years ago by itamar

Given that all extension modules you're compiling are failing, this suggests the problem is with the compilation process... Since you also built Python, which has its own extension modules that do appear to work, you should compare the compiler/linker flags being used there as opposed to what is being used when twisted is built.

comment:27 in reply to: ↑ 26 Changed 4 years ago by alberto

Where can I change some flags used building Twisted ? I mean which file of your distribution specify the flags to be used by the compiler/linker building Twisted ?

comment:28 Changed 4 years ago by itamar

When you run the appropriate command with setup.py (e.g. "setup.py build_ext" or "setup.py install") it calls gcc. You should make sure that is using the cross compiler, for one thing! But if it is, you can probably see what gcc calls it is doing, and do those manually. Also there's some command line arguments or env variables you can set to override compilation:

http://docs.python.org/install/index.html#building-extensions-tips-and-tricks

Note: See TracTickets for help on using tickets.