Opened 2 years ago

Closed 13 months ago

#5728 defect closed fixed (fixed)

sendmsg.c does not compile on Solaris

Reported by: toddmowen Owned by: itamar
Priority: normal Milestone:
Component: core Keywords:
Cc: xaka2004@…, zooko@… Branch: branches/solaris-build-5728
(diff, github, buildbot, log)
Author: itamarst Launchpad Bug:

Description

We are trying to upgrade from version 11 to 12, on a Solaris 5.10 box with Python 2.7.

The Twisted-12.0.0 release builds without error. However, the later Twisted-12.1.0 release produces the following error during python setup.py build:

twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:450: warning: cast to pointer from integer of different size
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: command 'gcc' failed with exit status 1

It appears that on this platform it is necessary to define the _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED macros in order to gain access to the msg_control, msg_controllen, and msg_flags members that the errors mention (according to http://stackoverflow.com/questions/1034587).

Attachments (3)

my-twisted-patch.patch (507 bytes) - added by nibalizer 2 years ago.
This patch allows twitsted to build on solaris.
twisted-5728-fix-build-on-sunos.patch (626 bytes) - added by xaka 21 months ago.
twisted-5728-fix-build-on-sunos.2.patch (1016 bytes) - added by xaka 21 months ago.

Download all attachments as: .zip

Change History (28)

comment:1 Changed 2 years ago by therve

  • Type changed from regression to defect

comment:2 Changed 2 years ago by exarkun

Just a note about defects vs regressions. Twisted is "supported" on a number of platforms. Only on those platforms do we consider new defects in old functionality (such as building) "regressions". You can find the list of supported platforms on our continuous integration system. There are various Windows, Linux, and even BSD configurations. However, no Solaris. If you're interested in Twisted on Solaris, consider helping out with that effort.

Changed 2 years ago by nibalizer

This patch allows twitsted to build on solaris.

comment:3 Changed 2 years ago by nibalizer

  • Keywords review added

This patch should allow for Solaris to build. I'm also trying to learn the development process so if there are procedures I'm not following please let me know.

comment:4 Changed 2 years ago by itamar

I emailed the person who recently setup a Solaris build machine, but which is currently offline (http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6). Hopefully he'll put it back up so we can test the patch.

comment:5 Changed 2 years ago by itamar

OK, the Solaris builder is up. It doesn't have a build error, but the sendmsg.so it builds does seem broken:

http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6/builds/9/steps/select/logs/problems

comment:6 Changed 2 years ago by exarkun

Inferring from the slave name, we have a Solaris 11 builder? This bug report seems to be about SunOS' 5.10 (as I understand it, there is no Solaris 5.10, but perhaps Solaris 10 is like SunOS 5.10?). That seems like a significant version skew. Though SunOS 5.10 was released 7 years ago, apparently support is planned for another 6 years by ... Oracle. So perhaps someone could contribute a Solaris 10 slave to complement the Solaris 11 slave.

comment:7 Changed 2 years ago by nibalizer

If buildbot can run as my unprivledged user at my university then I can set that up to run. I'll just nice all the processes and have to restart it when the system goes down for patching.

SunOS 5.10 is Solaris 10. Somewhere around SunOS 4 Sun renamed it Solairs. Uname continues to be SunOS.

This is the full uname:

uname -a
SunOS chandra.cs.pdx.edu 5.10 Generic_147440-19 sun4v sparc SUNW,SPARC-Enterprise-T5220

comment:8 Changed 2 years ago by nibalizer

While trying to easy_install buildbot I was unable to build twisted. Pasting here. Since buildbot depends on twisted... should I try with Twisted 12.0.0 ? Should I apply my patch?

Best match: Twisted 12.1.0
Downloading http://pypi.python.org/packages/source/T/Twisted/Twisted-12.1.0.tar.bz2#md5=f396f1d6f5321e869c2f89b2196a9eb5
Processing Twisted-12.1.0.tar.bz2
Running Twisted-12.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-hTDI_i/Twisted-12.1.0/egg-dist-tmp-Z2xXLG
conftest.c:1:23: sys/epoll.h: No such file or directory
twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: Setup script exited with error: command 'gcc' failed with exit status 1

comment:9 Changed 2 years ago by itamar

Any of those. Or just comment out the line in setup.py about sendmsg.c, it's not at all necessary.

comment:10 Changed 2 years ago by exarkun

Thanks for the offer, but this doesn't sound like the kind of environment that is suited to be used as a build slave. Slaves need to be reliably online 24/7. Do you expect this user account on shared university resources will meet that requirement?

comment:11 Changed 2 years ago by itamar

Can anyone (besides original author of patch) verify that the patch fixes the build?

comment:12 Changed 2 years ago by dannygoulder

Confirmed this does fix the build:

-bash-3.2$ patch < ~goulderd/build/sendmsg.c.patch

Looks like a unified context diff.

done
-bash-3.2$ bash ../mysetuppy.sh .
Processing .
Running setup.py -q bdist_egg --dist-dir /var/tmp/graphite/build/Twisted-12.2.0/egg-dist-tmp-N0owxZ
gcc: unrecognized option `-KPIC'
gcc: unrecognized option `-KPIC'
conftest.c:1:23: sys/epoll.h: No such file or directory
gcc: unrecognized option `-KPIC'
gcc: unrecognized option `-KPIC'
In file included from /usr/include/python2.6/Python.h:8,

from twisted/internet/_sigchld.c:9:

/usr/include/python2.6/pyconfig.h:978:1: warning: "_FILE_OFFSET_BITS" redefined
In file included from /usr/include/signal.h:18,

from twisted/internet/_sigchld.c:6:

/usr/include/sys/feature_tests.h:187:1: warning: this is the location of the previous definition
Adding Twisted 12.2.0 to easy-install.pth file
Installing pyhtmlizer script to /prod/systems/graphite/bin
Installing twistd script to /prod/systems/graphite/bin
Installing tap2deb script to /prod/systems/graphite/bin
Installing ckeygen script to /prod/systems/graphite/bin
Installing tap2rpm script to /prod/systems/graphite/bin
Installing conch script to /prod/systems/graphite/bin
Installing tapconvert script to /prod/systems/graphite/bin
Installing trial script to /prod/systems/graphite/bin
Installing tkconch script to /prod/systems/graphite/bin
Installing cftp script to /prod/systems/graphite/bin
Installing mailmail script to /prod/systems/graphite/bin
Installing manhole script to /prod/systems/graphite/bin
Installing lore script to /prod/systems/graphite/bin

Installed /prod/systems/graphite/lib/python2.6/site-packages/Twisted-12.2.0-py2.6-solaris-2.10-i86pc.egg
Processing dependencies for Twisted==12.2.0
Finished processing dependencies for Twisted==12.2.0
-bash-3.2$ uname -a
SunOS sysappsx1.london.daiwa.global 5.10 Generic_147441-24 i86pc i386 i86pc

comment:13 Changed 2 years ago by itamar

  • Keywords review removed

Sigh. The linked info suggests we want to set _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED, but the patch sets something else, so we need a new patch setting those and then a retest.

comment:14 Changed 2 years ago by exarkun

Also, still need a buildslave (otherwise effectively no one can work on this or test it).

Changed 21 months ago by xaka

comment:15 Changed 21 months ago by xaka

  • Cc xaka2004@… added

Sorry for messing up with patches. The right one is twisted-5728-fix-build-on-sunos.2.patch.

comment:16 Changed 20 months ago by enmand

If the only hold-up on this bug is that it needs a buildbot to test on it, I can offer a SmartOS buildbot for Python 2.6, 2.7, 3.2 and 3.3. Email me off-topic and we can organize it.

Thanks,
Dan

P.S. the patched worked for me.

comment:17 Changed 20 months ago by itamar

  • Keywords review added

enmand, could you email tom.prince@… about setting up a buildbot?

Personally I'm fine accepting a patch given independent confirmation of compilation, but a buildbot would be much better if we want things to continue working.

Adding review keyword.

comment:18 Changed 18 months ago by zooko

  • Cc zooko@… added
  • Keywords review removed
  • Owner set to xaka

I reviewed attachment:twisted-5728-fix-build-on-sunos.2.patch . Thank you, xaka, for contributing the patch!

Here are my comments:

  • The patch doesn't come with a NEWS entry. Please add a NEWS entry as described here: wiki:ReviewProcess#Newsfiles so that Twisted users will be informed about the fix.
  • The comment seems to be some text from a man page, and it is describing what the X/Open CAE is in general instead of describing why Twisted needs to use X/Open CAE. Please replace that text with an explanation of why Twisted needs this, plus a reference to the X/Open CAE doc, for example like this:
    # On Solaris 10, we need to define the _XOPEN_SOURCE and
    # _XOPEN_SOURCE_EXTENDED macros to build in order to gain access to
    # the msg_control, msg_controllen, and msg_flags members in
    # sendmsg.c. (according to
    # http://stackoverflow.com/questions/1034587).  See the documentation
    # of X/Open CAE in the standards(5) man page of Solaris.
    

Otherwise the patch looks good to me. Please attach a new patch with these changes.

By the way, I did not apply the patch, build it (I don't have Solaris), run unit tests or pyflakes, use buildbot, or do any other steps besides read the patch and write the review comments above.

comment:19 Changed 18 months ago by s0undt3ch

I confirm that attachment:twisted-5728-fix-build-on-sunos.2.patch allowed me to compile 0.13.0 under SmartOS.

comment:20 Changed 18 months ago by s0undt3ch

For those with an urgency in this fix, while it's not included into twisted:

pip install git+git://github.com/s0undt3ch/twisted.git#egg=Twisted

I just created this patched twisted clone because I need it for an automated install procedure.

comment:21 Changed 14 months ago by exarkun

#6787 was a duplicate of this.

comment:22 Changed 14 months ago by itamarst

  • Author set to itamarst
  • Branch set to branches/solaris-build-5728

(In [40559]) Branching to 'solaris-build-5728'

comment:23 Changed 14 months ago by itamar

  • Keywords review added; Solaris removed
  • Owner xaka deleted

I've added news file, and improved setup.py comment. Started buildbot build.

My thought is that given passing tests on supported platforms, and people reporting this fixes the problem, I'm fine merging this. The news file explicitly mentions that Solaris is not a supported platform until we have a buildslave.

comment:24 Changed 13 months ago by thijs

  • Keywords review removed
  • Owner set to itamar

Thanks, please merge.

comment:25 Changed 13 months ago by itamarst

  • Resolution set to fixed
  • Status changed from new to closed

(In [40859]) Merge solaris-build-5728.

Author: xaka
Reviewers: zooko, itamar, thijs
Fixes: #5728

sendmsg.c now builds on Solaris.

Note: See TracTickets for help on using tickets.