sendmsg.c does not compile on Solaris

[toddmowen]

We are trying to upgrade from version 11 to 12, on a Solaris 5.10 box with Python 2.7.

The Twisted-12.0.0 release builds without error. However, the later Twisted-12.1.0 release produces the following error during python setup.py build:

twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:450: warning: cast to pointer from integer of different size
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: command 'gcc' failed with exit status 1

It appears that on this platform it is necessary to define the _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED macros in order to gain access to the msg_control, msg_controllen, and msg_flags members that the errors mention (according to  http://stackoverflow.com/questions/1034587).

[exarkun]

Just a note about defects vs regressions. Twisted is "supported" on a number of platforms. Only on those platforms do we consider new defects in old functionality (such as building) "regressions". You can find the list of supported platforms on  our continuous integration system. There are various Windows, Linux, and even BSD configurations. However, no Solaris. If you're interested in Twisted on Solaris, consider helping out with that effort.

[nibalizer]

This patch allows twitsted to build on solaris.

[nibalizer]

This patch should allow for Solaris to build. I'm also trying to learn the development process so if there are procedures I'm not following please let me know.

[itamar]

I emailed the person who recently setup a Solaris build machine, but which is currently offline ( http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6). Hopefully he'll put it back up so we can test the patch.

[itamar]

OK, the Solaris builder is up. It doesn't have a build error, but the sendmsg.so it builds does seem broken:

 http://buildbot.twistedmatrix.com/builders/solaris11-x86-py2.6/builds/9/steps/select/logs/problems

[exarkun]

Inferring from the slave name, we have a Solaris 11 builder? This bug report seems to be about SunOS' 5.10 (as I understand it, there is no Solaris 5.10, but perhaps Solaris 10 is like SunOS 5.10?). That seems like a significant version skew. Though SunOS 5.10 was released 7 years ago, apparently support is planned for another 6 years by ... Oracle. So perhaps someone could contribute a Solaris 10 slave to complement the Solaris 11 slave.

[nibalizer]

If buildbot can run as my unprivledged user at my university then I can set that up to run. I'll just nice all the processes and have to restart it when the system goes down for patching.

SunOS 5.10 is Solaris 10. Somewhere around SunOS 4 Sun renamed it Solairs. Uname continues to be SunOS.

This is the full uname:

uname -a
SunOS chandra.cs.pdx.edu 5.10 Generic_147440-19 sun4v sparc SUNW,SPARC-Enterprise-T5220

[nibalizer]

While trying to easy_install buildbot I was unable to build twisted. Pasting here. Since buildbot depends on twisted... should I try with Twisted 12.0.0 ? Should I apply my patch?

Best match: Twisted 12.1.0
Downloading http://pypi.python.org/packages/source/T/Twisted/Twisted-12.1.0.tar.bz2#md5=f396f1d6f5321e869c2f89b2196a9eb5
Processing Twisted-12.1.0.tar.bz2
Running Twisted-12.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-hTDI_i/Twisted-12.1.0/egg-dist-tmp-Z2xXLG
conftest.c:1:23: sys/epoll.h: No such file or directory
twisted/python/sendmsg.c: In function `sendmsg_sendmsg':
twisted/python/sendmsg.c:198: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:199: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:201: error: structure has no member named `msg_flags'
twisted/python/sendmsg.c:243: warning: implicit declaration of function `CMSG_SPACE'
twisted/python/sendmsg.c:268: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:269: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:274: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:276: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:285: warning: implicit declaration of function `CMSG_FIRSTHDR'
twisted/python/sendmsg.c:285: warning: initialization makes pointer from integer without a cast
twisted/python/sendmsg.c:306: warning: implicit declaration of function `CMSG_LEN'
twisted/python/sendmsg.c:316: warning: implicit declaration of function `CMSG_DATA'
twisted/python/sendmsg.c:316: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:322: warning: implicit declaration of function `CMSG_NXTHDR'
twisted/python/sendmsg.c:322: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:351: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:352: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:353: error: structure has no member named `msg_control'
twisted/python/sendmsg.c: In function `sendmsg_recvmsg':
twisted/python/sendmsg.c:414: error: structure has no member named `msg_control'
twisted/python/sendmsg.c:416: error: structure has no member named `msg_controllen'
twisted/python/sendmsg.c:429: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:432: warning: assignment makes pointer from integer without a cast
twisted/python/sendmsg.c:477: error: structure has no member named `msg_flags'
error: Setup script exited with error: command 'gcc' failed with exit status 1

[itamar]

Any of those. Or just comment out the line in setup.py about sendmsg.c, it's not at all necessary.

[exarkun]

Thanks for the offer, but this doesn't sound like the kind of environment that is suited to be used as a build slave. Slaves need to be reliably online 24/7. Do you expect this user account on shared university resources will meet that requirement?

[itamar]

Can anyone (besides original author of patch) verify that the patch fixes the build?

[dannygoulder]

Confirmed this does fix the build:

-bash-3.2$ patch < ~goulderd/build/sendmsg.c.patch

Looks like a unified context diff.

done -bash-3.2$ bash ../mysetuppy.sh . Processing . Running setup.py -q bdist_egg --dist-dir /var/tmp/graphite/build/Twisted-12.2.0/egg-dist-tmp-N0owxZ gcc: unrecognized option `-KPIC' gcc: unrecognized option `-KPIC' conftest.c:1:23: sys/epoll.h: No such file or directory gcc: unrecognized option `-KPIC' gcc: unrecognized option `-KPIC' In file included from /usr/include/python2.6/Python.h:8,

from twisted/internet/_sigchld.c:9:

/usr/include/python2.6/pyconfig.h:978:1: warning: "_FILE_OFFSET_BITS" redefined In file included from /usr/include/signal.h:18,

from twisted/internet/_sigchld.c:6:

/usr/include/sys/feature_tests.h:187:1: warning: this is the location of the previous definition Adding Twisted 12.2.0 to easy-install.pth file Installing pyhtmlizer script to /prod/systems/graphite/bin Installing twistd script to /prod/systems/graphite/bin Installing tap2deb script to /prod/systems/graphite/bin Installing ckeygen script to /prod/systems/graphite/bin Installing tap2rpm script to /prod/systems/graphite/bin Installing conch script to /prod/systems/graphite/bin Installing tapconvert script to /prod/systems/graphite/bin Installing trial script to /prod/systems/graphite/bin Installing tkconch script to /prod/systems/graphite/bin Installing cftp script to /prod/systems/graphite/bin Installing mailmail script to /prod/systems/graphite/bin Installing manhole script to /prod/systems/graphite/bin Installing lore script to /prod/systems/graphite/bin

Installed /prod/systems/graphite/lib/python2.6/site-packages/Twisted-12.2.0-py2.6-solaris-2.10-i86pc.egg Processing dependencies for Twisted==12.2.0 Finished processing dependencies for Twisted==12.2.0 -bash-3.2$ uname -a SunOS sysappsx1.london.daiwa.global 5.10 Generic_147441-24 i86pc i386 i86pc

[itamar]

Sigh. The linked info suggests we want to set _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED, but the patch sets something else, so we need a new patch setting those and then a retest.

[exarkun]

Also, still need a buildslave (otherwise effectively no one can work on this or test it).

[xaka]

Sorry for messing up with patches. The right one is twisted-5728-fix-build-on-sunos.2.patch.

[enmand]

If the only hold-up on this bug is that it needs a buildbot to test on it, I can offer a SmartOS buildbot for Python 2.6, 2.7, 3.2 and 3.3. Email me off-topic and we can organize it.

Thanks, Dan

P.S. the patched worked for me.

[itamar]

enmand, could you email tom.prince@… about setting up a buildbot?

Personally I'm fine accepting a patch given independent confirmation of compilation, but a buildbot would be much better if we want things to continue working.

Adding review keyword.