Changes between Version 10 and Version 11 of Infrastructure/AdminScripts


Ignore:
Timestamp:
05/21/2013 11:54:29 AM (15 months ago)
Author:
tom.prince
Comment:

Turn this into a pointer to the documentation in the repo.

Legend:

Unmodified
Added
Removed
Modified
  • Infrastructure/AdminScripts

    v10 v11  
    1 {{{ 
    2 #!html 
    3 <div style="background: #ffff99; padding: 10px; border: 1px solid #ffcc88">Note: this page is still under construction and is out of date!</div> 
    4 }}} 
    5  
    6 = Fabric administration scripts = 
    7  
    8 This page describes how to use the Fabric-based administration scripts to manage services running on twistedmatrix.com. The temporary location for the administration scripts is at: https://github.com/twisted-infra/braid 
    9  
    10 == Getting the tools == 
    11  
    12 The first step is to install the needed libraries and tools. I'm using virtualenv and virtualenvwrapper in this guide, you are free to install everything on your stock python. 
    13 {{{ 
    14 #!sh 
    15 mkvirtualenv env_name 
    16 workon env_name 
    17 pip install fabric 
    18 git clone git://github.com/GaretJax/twisted-setup-temp.git tmadmin  # May be svn or bzr as well, location is temporary 
    19 cd tmadmin 
    20 }}} 
    21  
    22 You can now list all available tasks by calling: 
    23  
    24 {{{ 
    25 #!sh 
    26 fab --list 
    27 }}} 
    28 {{{ 
    29 Collection of utilities to automate the administration of Twisted's 
    30 infrastructure. Use this utility to install, update and start/stop/restart 
    31 services running on twistedmatrix.com. 
    32  
    33 Available commands: 
    34  
    35     dns.install 
    36     dns.restart 
    37     dns.start 
    38     dns.stop 
    39     dns.update 
    40     ... 
    41 }}} 
    42  
    43 == Client configuration == 
    44  
    45 Before being able to run any task, we need to configure the tool to work with Twisted's infrastructure and your login user. All configuration directives are contained in the `config.py` file. The base installation ships with a sample configuration file in `config.py.sample`. Copy it over and start editing it: 
    46  
    47 {{{ 
    48 #!sh 
    49 cp config.py.sample config.py 
    50 vim config.py 
    51 }}} 
    52  
    53 The two main directives to change are the `HOSTS` and `USER` ones. The first contains a list of hosts you want to administer, while the second one is, well, your ssh user to access the previously defined hosts. All found uppercase variables are lowercased and set as attributes of Fabric's `env` global variable. This means that any configuration directive recognized by Fabric can be defined in the `config.py` file as well. 
    54  
    55 Note: your user has to be at least in the `service-admin` group on the servers you want to manage in order to be able to start/stop/restart and update services. If you want to be able to install new services or give permissions to other users to manage them, your user has to be able to gain root access as well. 
    56  
    57 == Server configuration conventions == 
    58  
    59 === Directory structure === 
    60  
    61 Each service has its own directory under `/srv`. The structure of the service specific directory reflects the Filesystem Hierarchy Standard (e.g. pid files in `/srv/<service>/var/run/<service>.pid`, log files in `/srv/<service>/var/log/<service>.log`,...). 
    62  
    63 === Users, groups and privileges === 
    64  
    65 Each service runs as its own system user and owns his root directory (i.e. `/srv/<service>`). Each service user has to be part of the `service` group. To operate on a service (start/stop, update,...) a given user has to be part of the `service-admin` group. Each user in this group has permissions to `sudo` to every user in the `service` group. 
    66  
    67 As the service user is a system user, login for this user is disabled. 
    68  
    69 === Init scripts === 
    70  
    71 For each service, a System V style init script is provided in `/srv/<service>/etc/init.d/<service>`. This script is owned by the service user and has mode `0755`. A symlink is provided in `/etc/init.d`, this allows the script to be managed by `init(8)` and allows updates to be applied to the service directory by a service administrator user. 
    72  
    73 For convenience, a runner which simplifies the creation of new init scripts is available at `/usr/bin/twistd-service`. A minimal example of an init script for an hypothetical DNS service is shown below: 
    74 {{{ 
    75 #!sh 
    76  
    77 #!/usr/bin/twistd-service 
    78  
    79 ### BEGIN INIT INFO 
    80 # Provides:          dns 
    81 # Required-Start:    $named $network $time 
    82 # Required-Stop:     $named $network 
    83 # Should-Start: 
    84 # Should-Stop: 
    85 # Default-Start:     2 3 4 5 
    86 # Default-Stop:      0 1 6 
    87 # Short-Description: Twisted DNS server 
    88 # Description:       dns is a DNS server based on Twisted Names 
    89 ### END INIT INFO 
    90  
    91 SERVICE='dns' 
    92 ARGS='dns' 
    93  
    94 }}} 
    95  
    96 This script is a complete System V compatible init script to run `twistd` from a specific virtualenv with the correct pid and logfile arguments under the correct user. It provides the `start`, `stop`, `restart`, `status` and `zap` commands. It can be run by the root user, the service specific user or any user in the `service-admin` group and it is started with the correct user. 
    97  
    98 == Available tools == 
    99  
    100 === How to start/stop/restart services === 
    101  
    102 Each service has its own Fabric namespace. Actions are available as part of each namespace. For example, the `dns` service can be started, stopped,  and restarted as follows: 
    103  
    104 {{{ 
    105 #!sh 
    106 fab dns.start 
    107 }}} 
    108 {{{ 
    109 #!sh 
    110 fab dns.stop 
    111 }}} 
    112 {{{ 
    113 #!sh 
    114 fab dns.restart 
    115 }}} 
    116  
    117 === How to update existing services === 
    118  
    119 Similarly as done for managing the running states, an `update` task lives in each service namespace. It can be run as follows: 
    120  
    121 {{{ 
    122 #!sh 
    123 fab dns.update 
    124 }}} 
    125  
    126 Note that a restart is still needed after updating a service. 
    127  
    128 === How to install new services === 
    129  
    130 A service which was just added to the fabfile can be installed by running its `install` task: 
    131  
    132 {{{ 
    133 #!sh 
    134 fab dns.install 
    135 }}} 
    136  
    137 Note, however, that while the previous actions did not require root privileges (being member of the `service-admin` group was sufficient), installing a new service requires to be able to `sudo` to `root`. This is needed to create the necessary users, install additional packages and create the base environment. 
    138  
    139 === How to give service administration privileges to another user === 
    140  
    141 An existing user which does not currently have service administration tasks can be added to the `service-admin` group by executing the `make_service_admin` task and supplying a username: 
    142 {{{ 
    143 #!sh 
    144 fab make_service_admin:<username> 
    145 }}} 
    146  
    147 === How to setup a new server from scratch === 
    148  
    149 == Adding additional tools or modifying existing ones == 
     1The administration scripts are mostly documented at [https://github.com/twisted-infra/braid].