Opened 5 years ago

Closed 3 years ago

#5728 defect closed fixed (fixed)

sendmsg.c does not compile on Solaris

Reported by: Todd Owen Owned by: Itamar Turner-Trauring
Priority: normal Milestone:
Component: core Keywords:
Cc: Pavel Strashkin, zooko@… Branch: branches/solaris-build-5728
branch-diff, diff-cov, branch-cov, buildbot
Author: itamarst

Description

We are trying to upgrade from version 11 to 12, on a Solaris 5.10 box with Python 2.7.

The Twisted-12.0.0 release builds without error. However, the later Twisted-12.1.0 release produces the following error during python setup.py build:

twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:450: warning: cast to pointer from integer of different size
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: command 'gcc' failed with exit status 1

It appears that on this platform it is necessary to define the _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED macros in order to gain access to the msg_control, msg_controllen, and msg_flags members that the errors mention (according to http://stackoverflow.com/questions/1034587).

Attachments (3)

my-twisted-patch.patch (507 bytes) - added by Spencer Krum 4 years ago.
This patch allows twitsted to build on solaris.
twisted-5728-fix-build-on-sunos.patch (626 bytes) - added by Pavel Strashkin 4 years ago.
twisted-5728-fix-build-on-sunos.2.patch (1016 bytes) - added by Pavel Strashkin 4 years ago.

Download all attachments as: .zip

Change History (28)

comment:1 Changed 5 years ago by therve

Type: regressiondefect

comment:2 Changed 5 years ago by Jean-Paul Calderone

Just a note about defects vs regressions. Twisted is "supported" on a number of platforms. Only on those platforms do we consider new defects in old functionality (such as building) "regressions". You can find the list of supported platforms on our continuous integration system. There are various Windows, Linux, and even BSD configurations. However, no Solaris. If you're interested in Twisted on Solaris, consider helping out with that effort.

Changed 4 years ago by Spencer Krum

Attachment: my-twisted-patch.patch added

This patch allows twitsted to build on solaris.

comment:3 Changed 4 years ago by Spencer Krum

Keywords: review added

This patch should allow for Solaris to build. I'm also trying to learn the development process so if there are procedures I'm not following please let me know.

comment:4 Changed 4 years ago by Itamar Turner-Trauring

I emailed the person who recently setup a Solaris build machine, but which is currently offline (http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6). Hopefully he'll put it back up so we can test the patch.

comment:5 Changed 4 years ago by Itamar Turner-Trauring

OK, the Solaris builder is up. It doesn't have a build error, but the sendmsg.so it builds does seem broken:

http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6/builds/9/steps/select/logs/problems

comment:6 Changed 4 years ago by Jean-Paul Calderone

Inferring from the slave name, we have a Solaris 11 builder? This bug report seems to be about SunOS' 5.10 (as I understand it, there is no Solaris 5.10, but perhaps Solaris 10 is like SunOS 5.10?). That seems like a significant version skew. Though SunOS 5.10 was released 7 years ago, apparently support is planned for another 6 years by ... Oracle. So perhaps someone could contribute a Solaris 10 slave to complement the Solaris 11 slave.

comment:7 Changed 4 years ago by Spencer Krum

If buildbot can run as my unprivledged user at my university then I can set that up to run. I'll just nice all the processes and have to restart it when the system goes down for patching.

SunOS 5.10 is Solaris 10. Somewhere around SunOS 4 Sun renamed it Solairs. Uname continues to be SunOS.

This is the full uname:

uname -a
SunOS chandra.cs.pdx.edu 5.10 Generic_147440-19 sun4v sparc SUNW,SPARC-Enterprise-T5220

comment:8 Changed 4 years ago by Spencer Krum

While trying to easy_install buildbot I was unable to build twisted. Pasting here. Since buildbot depends on twisted... should I try with Twisted 12.0.0 ? Should I apply my patch?

Best match: Twisted 12.1.0
Downloading http://pypi.python.org/packages/source/T/Twisted/Twisted-12.1.0.tar.bz2#md5=f396f1d6f5321e869c2f89b2196a9eb5
Processing Twisted-12.1.0.tar.bz2
Running Twisted-12.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-hTDI_i/Twisted-12.1.0/egg-dist-tmp-Z2xXLG
conftest.c:1:23: sys/epoll.h: No such file or directory
twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: Setup script exited with error: command 'gcc' failed with exit status 1

comment:9 Changed 4 years ago by Itamar Turner-Trauring

Any of those. Or just comment out the line in setup.py about sendmsg.c, it's not at all necessary.

comment:10 Changed 4 years ago by Jean-Paul Calderone

Thanks for the offer, but this doesn't sound like the kind of environment that is suited to be used as a build slave. Slaves need to be reliably online 24/7. Do you expect this user account on shared university resources will meet that requirement?

comment:11 Changed 4 years ago by Itamar Turner-Trauring

Can anyone (besides original author of patch) verify that the patch fixes the build?

comment:12 Changed 4 years ago by danny

Confirmed this does fix the build:

-bash-3.2$ patch < ~goulderd/build/sendmsg.c.patch

Looks like a unified context diff.

done -bash-3.2$ bash ../mysetuppy.sh . Processing . Running setup.py -q bdist_egg --dist-dir /var/tmp/graphite/build/Twisted-12.2.0/egg-dist-tmp-N0owxZ gcc: unrecognized option `-KPIC' gcc: unrecognized option `-KPIC' conftest.c:1:23: sys/epoll.h: No such file or directory gcc: unrecognized option `-KPIC' gcc: unrecognized option `-KPIC' In file included from /usr/include/python2.6/Python.h:8,

from twisted/internet/_sigchld.c:9:

/usr/include/python2.6/pyconfig.h:978:1: warning: "_FILE_OFFSET_BITS" redefined In file included from /usr/include/signal.h:18,

from twisted/internet/_sigchld.c:6:

/usr/include/sys/feature_tests.h:187:1: warning: this is the location of the previous definition Adding Twisted 12.2.0 to easy-install.pth file Installing pyhtmlizer script to /prod/systems/graphite/bin Installing twistd script to /prod/systems/graphite/bin Installing tap2deb script to /prod/systems/graphite/bin Installing ckeygen script to /prod/systems/graphite/bin Installing tap2rpm script to /prod/systems/graphite/bin Installing conch script to /prod/systems/graphite/bin Installing tapconvert script to /prod/systems/graphite/bin Installing trial script to /prod/systems/graphite/bin Installing tkconch script to /prod/systems/graphite/bin Installing cftp script to /prod/systems/graphite/bin Installing mailmail script to /prod/systems/graphite/bin Installing manhole script to /prod/systems/graphite/bin Installing lore script to /prod/systems/graphite/bin

Installed /prod/systems/graphite/lib/python2.6/site-packages/Twisted-12.2.0-py2.6-solaris-2.10-i86pc.egg Processing dependencies for Twisted==12.2.0 Finished processing dependencies for Twisted==12.2.0 -bash-3.2$ uname -a SunOS sysappsx1.london.daiwa.global 5.10 Generic_147441-24 i86pc i386 i86pc

comment:13 Changed 4 years ago by Itamar Turner-Trauring

Keywords: review removed

Sigh. The linked info suggests we want to set _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED, but the patch sets something else, so we need a new patch setting those and then a retest.

comment:14 Changed 4 years ago by Jean-Paul Calderone

Also, still need a buildslave (otherwise effectively no one can work on this or test it).

Changed 4 years ago by Pavel Strashkin

Changed 4 years ago by Pavel Strashkin

comment:15 Changed 4 years ago by Pavel Strashkin

Cc: Pavel Strashkin added

Sorry for messing up with patches. The right one is twisted-5728-fix-build-on-sunos.2.patch.

comment:16 Changed 4 years ago by Daniel Enman

If the only hold-up on this bug is that it needs a buildbot to test on it, I can offer a SmartOS buildbot for Python 2.6, 2.7, 3.2 and 3.3. Email me off-topic and we can organize it.

Thanks, Dan

P.S. the patched worked for me.

comment:17 Changed 4 years ago by Itamar Turner-Trauring

Keywords: review added

enmand, could you email tom.prince@… about setting up a buildbot?

Personally I'm fine accepting a patch given independent confirmation of compilation, but a buildbot would be much better if we want things to continue working.

Adding review keyword.

comment:18 Changed 4 years ago by zooko

Cc: zooko@… added
Keywords: review removed
Owner: set to Pavel Strashkin

I reviewed attachment:twisted-5728-fix-build-on-sunos.2.patch . Thank you, xaka, for contributing the patch!

Here are my comments:

  • The patch doesn't come with a NEWS entry. Please add a NEWS entry as described here: wiki:ReviewProcess#Newsfiles so that Twisted users will be informed about the fix.
  • The comment seems to be some text from a man page, and it is describing what the X/Open CAE is in general instead of describing why Twisted needs to use X/Open CAE. Please replace that text with an explanation of why Twisted needs this, plus a reference to the X/Open CAE doc, for example like this:
    # On Solaris 10, we need to define the _XOPEN_SOURCE and
    # _XOPEN_SOURCE_EXTENDED macros to build in order to gain access to
    # the msg_control, msg_controllen, and msg_flags members in
    # sendmsg.c. (according to
    # http://stackoverflow.com/questions/1034587).  See the documentation
    # of X/Open CAE in the standards(5) man page of Solaris.
    

Otherwise the patch looks good to me. Please attach a new patch with these changes.

By the way, I did not apply the patch, build it (I don't have Solaris), run unit tests or pyflakes, use buildbot, or do any other steps besides read the patch and write the review comments above.

comment:19 Changed 4 years ago by s0undt3ch

I confirm that attachment:twisted-5728-fix-build-on-sunos.2.patch allowed me to compile 0.13.0 under SmartOS.

comment:20 Changed 4 years ago by s0undt3ch

For those with an urgency in this fix, while it's not included into twisted:

pip install git+git://github.com/s0undt3ch/twisted.git#egg=Twisted

I just created this patched twisted clone because I need it for an automated install procedure.

comment:21 Changed 3 years ago by Jean-Paul Calderone

#6787 was a duplicate of this.

comment:22 Changed 3 years ago by itamarst

Author: itamarst
Branch: branches/solaris-build-5728

(In [40559]) Branching to 'solaris-build-5728'

comment:23 Changed 3 years ago by Itamar Turner-Trauring

Keywords: review added; Solaris removed
Owner: Pavel Strashkin deleted

I've added news file, and improved setup.py comment. Started buildbot build.

My thought is that given passing tests on supported platforms, and people reporting this fixes the problem, I'm fine merging this. The news file explicitly mentions that Solaris is not a supported platform until we have a buildslave.

comment:24 Changed 3 years ago by Thijs Triemstra

Keywords: review removed
Owner: set to Itamar Turner-Trauring

Thanks, please merge.

comment:25 Changed 3 years ago by itamarst

Resolution: fixed
Status: newclosed

(In [40859]) Merge solaris-build-5728.

Author: xaka Reviewers: zooko, itamar, thijs Fixes: #5728

sendmsg.c now builds on Solaris.

Note: See TracTickets for help on using tickets.