=== modified file 'doc/core/howto/endpoints.xhtml' --- doc/core/howto/endpoints.xhtml 2012-08-20 15:25:12 +0000 +++ doc/core/howto/endpoints.xhtml 2013-01-27 13:05:25 +0000 @@ -233,13 +233,13 @@ socket. lockfile enables use of a separate lock file to prove the server is still running. For example, unix:address=/var/run/web.sock:lockfile=1. -
  • systemd. Supported arguments: domain, index. domain indicates which - socket domain the inherited file descriptor belongs to (eg INET, INET6). - index indicates an offset into the array of file descriptors which have been - inherited from systemd. For - example, systemd:domain=INET6:index=3. +
  • systemd. Supported arguments: domain, index. domain indicates + which socket domain the inherited file descriptor belongs to (eg + INET, INET6). index indicates an offset into the array of file + descriptors which have been inherited from systemd. For + example, systemd:domain=INET6:index=3. See + also Deploying Twisted with systemd.
  • - === modified file 'doc/core/howto/index.xhtml' --- doc/core/howto/index.xhtml 2012-09-07 14:46:38 +0000 +++ doc/core/howto/index.xhtml 2013-01-27 13:05:25 +0000 @@ -159,6 +159,10 @@
  • Writing Twisted Application Plugins for twistd
    More powerful twistd deployment method.
  • + +
  • Deploying Twisted with + systemd
    Use systemd to launch and + monitor Twisted applications
  • === added directory 'doc/core/howto/listings/systemd' === added file 'doc/core/howto/listings/systemd/www.example.com.socket' --- doc/core/howto/listings/systemd/www.example.com.socket 1970-01-01 00:00:00 +0000 +++ doc/core/howto/listings/systemd/www.example.com.socket 2013-01-27 13:05:25 +0000 @@ -0,0 +1,5 @@ +[Socket] +ListenStream=0.0.0.0:80 + +[Install] +WantedBy=sockets.target === added file 'doc/core/howto/listings/systemd/www.example.com.socketactivated.service' --- doc/core/howto/listings/systemd/www.example.com.socketactivated.service 1970-01-01 00:00:00 +0000 +++ doc/core/howto/listings/systemd/www.example.com.socketactivated.service 2013-01-29 14:36:02 +0000 @@ -0,0 +1,17 @@ +[Unit] +Description=Example Web Server + +[Service] +ExecStart=/usr/bin/twistd \ + --nodaemon \ + --pidfile= \ + web --port systemd:domain=INET:index=0 --path . + +NonBlocking=true + +WorkingDirectory=/srv/www/www.example.com/static + +User=nobody +Group=nobody + +Restart=always === added file 'doc/core/howto/listings/systemd/www.example.com.static.service' --- doc/core/howto/listings/systemd/www.example.com.static.service 1970-01-01 00:00:00 +0000 +++ doc/core/howto/listings/systemd/www.example.com.static.service 2013-01-28 17:33:16 +0000 @@ -0,0 +1,18 @@ +[Unit] +Description=Example Web Server + +[Service] +ExecStart=/usr/bin/twistd \ + --nodaemon \ + --pidfile= \ + web --port 8080 --path . + +WorkingDirectory=/srv/www/www.example.com/static + +User=nobody +Group=nobody + +Restart=always + +[Install] +WantedBy=multi-user.target === added file 'doc/core/howto/systemd.xhtml' --- doc/core/howto/systemd.xhtml 1970-01-01 00:00:00 +0000 +++ doc/core/howto/systemd.xhtml 2013-01-29 18:18:20 +0000 @@ -0,0 +1,431 @@ + + + + + + + Deploying Twisted with systemd + + + +

    Deploying Twisted with systemd

    + +

    Introduction

    + +

    In this tutorial you will learn how to start a Twisted service + using systemd.

    + +

    You will also learn how to start the service using socket + activation.

    + +
    +

    The examples in this tutorial demonstrate how to + launch a Twisted web server, but the same techniques apply to any + Twisted service.

    +
    + +

    Prerequisites

    +
    +
    Twisted
    +
    You will need a version of Twisted >= 12.2 for the socket + activation section of this tutorial.
    +
    This tutorial was written on a Fedora 18 Linux operating system + with a system wide installation of Twisted and Twisted Web.
    +
    If you have installed Twisted locally eg in your home directory + or in a virtualenv, you will need to modify the paths in some of + the following examples.
    +
    +

    Test your Twisted installation by starting a twistd + web server on TCP port 8080 with the following + command:

    +
    +$ twistd --nodaemon web --port 8080 --path /srv/www/www.example.com/static
    +2013-01-28 13:21:35+0000 [-] Log opened.
    +2013-01-28 13:21:35+0000 [-] twistd 12.3.0 (/usr/bin/python 2.7.3) starting up.
    +2013-01-28 13:21:35+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
    +2013-01-28 13:21:35+0000 [-] Site starting on 8080
    +2013-01-28 13:21:35+0000 [-] Starting factory <twisted.web.server.Site instance at 0x7f57eb66efc8>
    +    
    +

    This assumes that you have the following static web page in the + following directory structure:

    +
    +# tree /srv/
    +/srv/
    +└── www
    +    └── www.example.com
    +        └── static
    +            └── index.html
    +    
    +
    +<!doctype html>
    +<html lang=en>
    +  <head>
    +    <meta charset=utf-8>
    +    <title>Example Site</title>
    +  </head>
    +  <body>
    +    <h1>Example Site</h1>
    +  </body>
    +</html>
    +    
    + +

    Now try connecting + to http://localhost:8080 in + your web browser.

    +

    If you do not see your web page or if twistd didn't start, you + should investigate and fix the problem before continuing.

    +
    +
    + +

    Basic Systemd Service Configuration

    +

    The essential configuration file for a systemd service is the + systemd.service + file.

    +

    Later in this tutorial, you will learn about some other types of + configuration file, which are used to control when and how your + service is started.

    +

    But we will begin by configuring systemd to start a Twisted web + server immediately on system boot.

    +
      +
    1. +

      Create + a systemd.service + file

      +

      Create the service file + at /etc/systemd/system/www.example.com.service with + the following content:

      + /etc/systemd/system/www.example.com.service + +

      This configuration file contains the following note worthy + sections:

      + +
      +
      ExecStart
      +
      Always include the full path to twistd in case + you have multiple versions installed.
      +
      The --nodaemon flag makes twistd + run in the foreground. Systemd works best with child processes + that remain in the foreground.
      +
      The --pidfile= flag + prevents twistd from writing a pidfile. A pidfile + is not necessary when Twisted runs as a foreground + process.
      +
      The --path flag to specifies the location of + the website files. In this example we use . which + makes twistd serve files from its current working + directory (see below).
      + +
      WorkingDirectory
      +
      Systemd can configure the working environment of its child + processes.
      +
      In this example the working directory of twistd + is set to that of the static website.
      + +
      User / Group
      +
      Systemd can also control the effective user and group of its + child processes.
      +
      This example uses an un-privileged user nobody and + un-privileged group nobody.
      +
      This is an important security measure which ensures that the + Twisted sub-process can not access restricted areas of the + file system.
      + +
      Restart
      +
      Systemd can automatically restart a child process if it + exits or crashes unexpectedly.
      +
      In this example the Restart option is set + to always, which ensures that twistd will be + restarted under all circumstances.
      +
      + +

      There are many more service options which are documented in the + systemd.service + man page.

      +
    2. + +
    3. +

      Reload systemd

      +
      +$ sudo systemctl daemon-reload
      +    
      +

      This forces systemd to read the new configuration + file.

      +

      Always run systemctl daemon-reload + after changing any of the systemd configuration files.

      +
    4. +
    5. +

      Start the service

      +
      +$ sudo systemctl start www.example.com
      +    
      + +

      twistd should now be running and listening on TCP + port 8080. You can verify this using the systemctl + command. eg

      +
      +$ sudo systemctl status www.example.com.service
      +www.example.com.service - Example Web Server
      +          Loaded: loaded (/etc/systemd/system/www.example.com.service; enabled)
      +          Active: active (running) since Mon 2013-01-28 16:16:26 GMT; 1s ago
      +        Main PID: 10695 (twistd)
      +          CGroup: name=systemd:/system/www.example.com.service
      +                  └─10695 /usr/bin/python /usr/bin/twistd --nodaemon --pidfile= web --port 8080 --path .
      +
      +Jan 28 16:16:26 zorin.lan systemd[1]: Starting Example Web Server...
      +Jan 28 16:16:26 zorin.lan systemd[1]: Started Example Web Server.
      +Jan 28 16:16:26 zorin.lan twistd[10695]: 2013-01-28 16:16:26+0000 [-] Log opened.
      +Jan 28 16:16:26 zorin.lan twistd[10695]: 2013-01-28 16:16:26+0000 [-] twistd 12.1.0 (/usr/bin/python 2.7.3) starting up.
      +Jan 28 16:16:26 zorin.lan twistd[10695]: 2013-01-28 16:16:26+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
      +Jan 28 16:16:26 zorin.lan twistd[10695]: 2013-01-28 16:16:26+0000 [-] Site starting on 8080
      +Jan 28 16:16:26 zorin.lan twistd[10695]: 2013-01-28 16:16:26+0000 [-] Starting factory <twisted.web.server.Site instance at 0x159b758>
      +    
      +

      The systemctl status command is convenient because + it shows you both the current status of the service and a short + log of the service output.

      +

      This is especially useful for debugging and diagnosing service + startup problems.

      +

      The twistd subprocess will log messages + to stderr and systemd will in turn log + these messages to syslog. You can verify this by monitoring the + syslog messages or by using the new journalctl tool + in Fedora.

      +

      See + the systemctl + man page for details of the service available options.

      +
    6. +
    7. +

      Enable the service

      +

      We've seen how to start the service manually, but now we need + to enable it so that it starts automatically at boot + time.

      +

      Enable the service with the following command:

      +
      +$ sudo systemctl enable www.example.com.service
      +ln -s '/etc/systemd/system/www.example.com.service' '/etc/systemd/system/multi-user.target.wants/www.example.com.service'
      +    
      +

      As you can see, this creates a symlink to the service file in + the systemd multi-user target directory.

      +

      The Twisted web server will now be started automatically at + boot time.

      +

      The default systemd boot target + is multi-user. Later in this tutorial you will learn how + to use the sockets target.

      +
    8. + +
    9. +

      Test that the service is automatically restarted

      +

      The Restart=always option in + the systemd.service file ensures that systemd will + restart the twistd process if and when it exits + unexpectedly.

      +

      You can read about other Restart options in + the systemd.service + man page.

      +

      Try killing the twistd process and then checking its status + again:

      +
      +$ sudo kill 12543
      +
      +$ sudo systemctl status www.example.com.service
      +www.example.com.service - Example Web Server
      +          Loaded: loaded (/etc/systemd/system/www.example.com.service; disabled)
      +          Active: active (running) since Mon 2013-01-28 17:47:37 GMT; 1s ago
      +        Main PID: 12611 (twistd)
      +    
      +

      The Active time stamp shows that the twistd process was + restarted within 1 second.

      +
    10. + +
    11. +

      Now stop the service before you proceed to the next + section.

      +
      +$ sudo systemctl stop www.example.com.service
      +
      +$ sudo systemctl status www.example.com.service
      +www.example.com.service - Example Web Server
      +          Loaded: loaded (/etc/systemd/system/www.example.com.service; enabled)
      +          Active: inactive (dead) since Mon 2013-01-28 16:51:12 GMT; 1s ago
      +         Process: 10695 ExecStart=/usr/bin/twistd --nodaemon --pidfile= web --port 8080 --path . (code=exited, status=0/SUCCESS)
      +    
      +
    12. +
    + +

    Socket Activation

    +

    First you need to understand what socket activation is. This + extract from + the systemd + daemon man page explains it clearly.

    +
    + In a socket-based activation scheme the creation and binding of the + listening socket as primary communication channel of daemons to + local (and sometimes remote) clients is moved out of the daemon code + and into the init system. Based on per-daemon configuration the init + system installs the sockets and then hands them off to the spawned + process as soon as the respective daemon is to be + started. Optionally activation of the service can be delayed until + the first inbound traffic arrives at the socket, to implement + on-demand activation of daemons. However, the primary advantage of + this scheme is that all providers and all consumers of the sockets + can be started in parallel as soon as all sockets are + established. In addition to that daemons can be restarted with + losing only a minimal number of client transactions or even any + client request at all (the latter is particularly true for + state-less protocols, such as DNS or syslog), because the socket + stays bound and accessible during the restart, and all requests are + queued while the daemon cannot process them. +
    +

    Twisted (since version 12.2) includes + a systemd endpoint API and a corresponding + string ports syntax, which allows a Twisted service to inherit a + listening socket from systemd.

    +

    The following example builds on the previous example, demonstrating + how to enable socket activation for a simple Twisted web server.

    +
    +

    Before continuing, stop the previous example service with the + following command:

    + +
    +$ sudo systemctl stop www.example.com.service
    +  
    +
    + +
      +
    1. +

      Create + a systemd.socket + file

      +

      Create the socket file + at /etc/systemd/system/www.example.com.socket with + the following content:

      + /etc/systemd/system/www.example.com.socket + +

      This configuration file contains the following important + sections:

      +
      +
      ListenStream=0.0.0.0:80
      +
      This option configures systemd to create a + listening TCP socket bound to all local IPv4 addresses on port + 80.
      +
      WantedBy=sockets.target
      +
      This is a special target used by all socket activated + services. systemd will automatically bind to all + such socket activation ports during boot up.
      +
      + +

      You also need to modify the systemd.service file as + follows:

      + /etc/systemd/system/www.example.com.service +

      Notice the following important changes:

      +
      +
      ExecStart
      +
      The domain=INET endpoint argument makes twistd + treat the inherited file descriptor as an IPv4 socket.
      +
      The index=0 endpoint argument + makes twistd adopt the first file descriptor + inherited from systemd.
      +
      Socket activation is also technically possible + with other socket families and types, but Twisted currently + only accepts IPv4 and IPv6 TCP + sockets. See limitations + below.
      + +
      NonBlocking
      +
      This must be set to true to ensure that systemd + passes non-blocking sockets to Twisted.
      +
      +

      Reload systemd so that it reads the updated + configuration files.

      +
      +$ sudo systemctl daemon-reload
      +    
      +
    2. + +
    3. +

      Start and enable the socket

      +

      You can now start systemd listening on the socket with the + following command:

      +
      +$ sudo systemctl start www.example.com.socket
      +    
      +

      This command refers specifically to the socket + configuration file, not the service file.

      +

      systemd should now be listening on port 80

      +
      +$ sudo systemctl status www.example.com.socket
      +www.example.com.socket
      +          Loaded: loaded (/etc/systemd/system/www.example.com.socket; disabled)
      +          Active: active (listening) since Tue 2013-01-29 14:53:17 GMT; 7s ago
      +
      +Jan 29 14:53:17 zorin.lan systemd[1]: Listening on www.example.com.socket.
      +    
      + +

      But twistd should not yet have started. You can verify this + using the systemctl command. eg

      +
      +$ sudo systemctl status www.example.com.service
      +www.example.com.service - Example Web Server
      +          Loaded: loaded (/etc/systemd/system/www.example.com.service; static)
      +          Active: inactive (dead) since Tue 2013-01-29 14:48:42 GMT; 6min ago
      +    
      + +

      Enable the socket, so that it will be started automatically + with the other socket activated services during boot up.

      +
      +$ sudo systemctl enable www.example.com.socket
      +ln -s '/etc/systemd/system/www.example.com.socket' '/etc/systemd/system/sockets.target.wants/www.example.com.socket'
      +    
      +
    4. +
    5. +

      Activate the port, start the service

      + +

      Now try connecting + to http://localhost:80 in + your web browser.

      + +

      systemd will accept the connection and + start twistd, passing it the listening socket. You + can verify this by using systemctl to report the status of the + service.. eg

      +
      +$ sudo systemctl status www.example.com.service
      +www.example.com.service - Example Web Server
      +          Loaded: loaded (/etc/systemd/system/www.example.com.service; static)
      +          Active: active (running) since Tue 2013-01-29 15:02:20 GMT; 3s ago
      +        Main PID: 25605 (twistd)
      +          CGroup: name=systemd:/system/www.example.com.service
      +                  └─25605 /usr/bin/python /usr/bin/twistd --nodaemon --pidfile= web --port systemd:domain=INET:index=0 --path .
      +
      +Jan 29 15:02:20 zorin.lan systemd[1]: Started Example Web Server.
      +Jan 29 15:02:20 zorin.lan twistd[25605]: 2013-01-29 15:02:20+0000 [-] Log opened.
      +Jan 29 15:02:20 zorin.lan twistd[25605]: 2013-01-29 15:02:20+0000 [-] twistd 12.1.0 (/usr/bin/python 2.7.3) starting up.
      +Jan 29 15:02:20 zorin.lan twistd[25605]: 2013-01-29 15:02:20+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
      +Jan 29 15:02:20 zorin.lan twistd[25605]: 2013-01-29 15:02:20+0000 [-] Site starting on 80
      +Jan 29 15:02:20 zorin.lan twistd[25605]: 2013-01-29 15:02:20+0000 [-] Starting factory <twisted.web.server.Site instance at 0x24be758>
      +    
      +
    6. +
    + +

    Conclusion

    +

    In this tutorial you have learned how to deploy a Twisted service + using systemd. You have also learned how the service can be started + on demand, using socket activation

    + +

    Limitations and Known Issues

    +
      +
    1. Twisted can not accept UNIX or datagram sockets + from systemd.
    2. +
    3. Twisted does not support listening for SSL connections on + sockets inherited from systemd.
    4. +
    +

    Further Reading

    + + + === added file 'twisted/topfiles/5601.feature' --- twisted/topfiles/5601.feature 1970-01-01 00:00:00 +0000 +++ twisted/topfiles/5601.feature 2013-01-29 15:20:26 +0000 @@ -0,0 +1,3 @@ +A new "Deploying Twisted with systemd" howto document which +demonstrates how to start a Twisted service using systemd socket +activation.