Changes between Version 10 and Version 11 of Infrastructure/AdminScripts


Ignore:
Timestamp:
05/21/2013 11:54:29 AM (4 years ago)
Author:
tom.prince
Comment:

Turn this into a pointer to the documentation in the repo.

Legend:

Unmodified
Added
Removed
Modified
  • Infrastructure/AdminScripts

    v10 v11  
    1 {{{
    2 #!html
    3 <div style="background: #ffff99; padding: 10px; border: 1px solid #ffcc88">Note: this page is still under construction and is out of date!</div>
    4 }}}
    5 
    6 = Fabric administration scripts =
    7 
    8 This page describes how to use the Fabric-based administration scripts to manage services running on twistedmatrix.com. The temporary location for the administration scripts is at: https://github.com/twisted-infra/braid
    9 
    10 == Getting the tools ==
    11 
    12 The first step is to install the needed libraries and tools. I'm using virtualenv and virtualenvwrapper in this guide, you are free to install everything on your stock python.
    13 {{{
    14 #!sh
    15 mkvirtualenv env_name
    16 workon env_name
    17 pip install fabric
    18 git clone git://github.com/GaretJax/twisted-setup-temp.git tmadmin  # May be svn or bzr as well, location is temporary
    19 cd tmadmin
    20 }}}
    21 
    22 You can now list all available tasks by calling:
    23 
    24 {{{
    25 #!sh
    26 fab --list
    27 }}}
    28 {{{
    29 Collection of utilities to automate the administration of Twisted's
    30 infrastructure. Use this utility to install, update and start/stop/restart
    31 services running on twistedmatrix.com.
    32 
    33 Available commands:
    34 
    35     dns.install
    36     dns.restart
    37     dns.start
    38     dns.stop
    39     dns.update
    40     ...
    41 }}}
    42 
    43 == Client configuration ==
    44 
    45 Before being able to run any task, we need to configure the tool to work with Twisted's infrastructure and your login user. All configuration directives are contained in the `config.py` file. The base installation ships with a sample configuration file in `config.py.sample`. Copy it over and start editing it:
    46 
    47 {{{
    48 #!sh
    49 cp config.py.sample config.py
    50 vim config.py
    51 }}}
    52 
    53 The two main directives to change are the `HOSTS` and `USER` ones. The first contains a list of hosts you want to administer, while the second one is, well, your ssh user to access the previously defined hosts. All found uppercase variables are lowercased and set as attributes of Fabric's `env` global variable. This means that any configuration directive recognized by Fabric can be defined in the `config.py` file as well.
    54 
    55 Note: your user has to be at least in the `service-admin` group on the servers you want to manage in order to be able to start/stop/restart and update services. If you want to be able to install new services or give permissions to other users to manage them, your user has to be able to gain root access as well.
    56 
    57 == Server configuration conventions ==
    58 
    59 === Directory structure ===
    60 
    61 Each service has its own directory under `/srv`. The structure of the service specific directory reflects the Filesystem Hierarchy Standard (e.g. pid files in `/srv/<service>/var/run/<service>.pid`, log files in `/srv/<service>/var/log/<service>.log`,...).
    62 
    63 === Users, groups and privileges ===
    64 
    65 Each service runs as its own system user and owns his root directory (i.e. `/srv/<service>`). Each service user has to be part of the `service` group. To operate on a service (start/stop, update,...) a given user has to be part of the `service-admin` group. Each user in this group has permissions to `sudo` to every user in the `service` group.
    66 
    67 As the service user is a system user, login for this user is disabled.
    68 
    69 === Init scripts ===
    70 
    71 For each service, a System V style init script is provided in `/srv/<service>/etc/init.d/<service>`. This script is owned by the service user and has mode `0755`. A symlink is provided in `/etc/init.d`, this allows the script to be managed by `init(8)` and allows updates to be applied to the service directory by a service administrator user.
    72 
    73 For convenience, a runner which simplifies the creation of new init scripts is available at `/usr/bin/twistd-service`. A minimal example of an init script for an hypothetical DNS service is shown below:
    74 {{{
    75 #!sh
    76 
    77 #!/usr/bin/twistd-service
    78 
    79 ### BEGIN INIT INFO
    80 # Provides:          dns
    81 # Required-Start:    $named $network $time
    82 # Required-Stop:     $named $network
    83 # Should-Start:
    84 # Should-Stop:
    85 # Default-Start:     2 3 4 5
    86 # Default-Stop:      0 1 6
    87 # Short-Description: Twisted DNS server
    88 # Description:       dns is a DNS server based on Twisted Names
    89 ### END INIT INFO
    90 
    91 SERVICE='dns'
    92 ARGS='dns'
    93 
    94 }}}
    95 
    96 This script is a complete System V compatible init script to run `twistd` from a specific virtualenv with the correct pid and logfile arguments under the correct user. It provides the `start`, `stop`, `restart`, `status` and `zap` commands. It can be run by the root user, the service specific user or any user in the `service-admin` group and it is started with the correct user.
    97 
    98 == Available tools ==
    99 
    100 === How to start/stop/restart services ===
    101 
    102 Each service has its own Fabric namespace. Actions are available as part of each namespace. For example, the `dns` service can be started, stopped,  and restarted as follows:
    103 
    104 {{{
    105 #!sh
    106 fab dns.start
    107 }}}
    108 {{{
    109 #!sh
    110 fab dns.stop
    111 }}}
    112 {{{
    113 #!sh
    114 fab dns.restart
    115 }}}
    116 
    117 === How to update existing services ===
    118 
    119 Similarly as done for managing the running states, an `update` task lives in each service namespace. It can be run as follows:
    120 
    121 {{{
    122 #!sh
    123 fab dns.update
    124 }}}
    125 
    126 Note that a restart is still needed after updating a service.
    127 
    128 === How to install new services ===
    129 
    130 A service which was just added to the fabfile can be installed by running its `install` task:
    131 
    132 {{{
    133 #!sh
    134 fab dns.install
    135 }}}
    136 
    137 Note, however, that while the previous actions did not require root privileges (being member of the `service-admin` group was sufficient), installing a new service requires to be able to `sudo` to `root`. This is needed to create the necessary users, install additional packages and create the base environment.
    138 
    139 === How to give service administration privileges to another user ===
    140 
    141 An existing user which does not currently have service administration tasks can be added to the `service-admin` group by executing the `make_service_admin` task and supplying a username:
    142 {{{
    143 #!sh
    144 fab make_service_admin:<username>
    145 }}}
    146 
    147 === How to setup a new server from scratch ===
    148 
    149 == Adding additional tools or modifying existing ones ==
     1The administration scripts are mostly documented at [https://github.com/twisted-infra/braid].