InstallationBEgridClient0

From T2B Wiki
Jump to navigation Jump to search

Installation of a Quattor deployment server release 13.1

Aim of this procedure

This procedure describes how to install a Quattor deployment server (also called "BEgrid client") release 13.1. The main services/tools of such a server are : AII, DHCP, HTTPD, RUNCHECK and SINDES.

Hardware needed

4096MB RAM + 40GB hard drive + 4 cores If you want to use a VM under KVM, than choose virtio for NICs and HD. You need two NICs : one in the private, and one in the public network.

OS installation

This procedure is based on SL58. Avoid RH6-like distros (too much troubles with pysvn and Python2.6 and GNUTLS used instead of OpenSSL), unless you are looking for troubles !

During the installation process, choose the following options for the machine type : "Server", "Development tools". And un-select "GNOME".

Basic configuration tasks

Once the OS is installed, do the following :

  • choose a nice name, and create entries for the machine in the DNSs
  • adapt /etc/sysconfig/network :
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=<hostname_of_your_choice>
DNS1=193.190.247.140
DNS2=193.190.247.71
GATEWAY=193.190.247.65
  • configure the NICs by editing the files /etc/sysconfig/network-scripts/ifcfg-eth<x> (x=0,1)
  • disable Networkmanager :
chkconfig --level 2345 NetworkManager off
  • enable network service :
chkconfig --level 2345 network on
  • restart the network :
service network restart
  • change hostname in environment with hostname command
  • disable SELinux :
setenforce 0
vim /etc/sysconfig/selinux
--> SELINUX=disabled
  • configure ntp (add the line "server ntp.vub.ac.be" at the end of /etc/ntp.conf)
  • disable yum-autoupdate (remove yum.cron from /etc/cron.daily)
  • configure the firewall, but don't be too strict at the beginning : during the installation process, it is enough to allow only inputs from the private and public networks !

Packages installation

  • Repositories configuration : edit the following repo files in /etc/yum.repo.d/ :
# cat /etc/yum.repos.d/quattor.repo
[quattor-13.1]
name=quattor13.1
enabled=1
baseurl=http://yum.quattor.org/13.1/
gpgcheck=0
[quattor-components]
name=quattor-components
enabled=1
baseurl=http://quattor.web.lal.in2p3.fr/packages/ncm-components/
gpgcheck=0
[quattor-externals]
name=quattor-externals
enabled=1
baseurl=http://quattor.web.lal.in2p3.fr/packages/quattor/externals/
gpgcheck=0

# cat /etc/yum.repos.d/sl5_epel.repo 
[sl5-epel]
name=Scientific Linux 5x - EPEL
baseurl=http://quattor.web.lal.in2p3.fr/packages/os/sl5/epel/
enabled=1
gpgcheck=0

# cat /etc/yum.repos.d/sl5_addons.repo 
[sl5-addons]
name=Scientific Linux 5x - addons
baseurl=http://quattor.web.lal.in2p3.fr/packages/os/sl5/addons
enabled=1
gpgcheck=0

# cat /etc/yum.repos.d/java.repo 
[java]
name=java
enabled=1
baseurl=http://quattor.web.lal.in2p3.fr/packages/java/
gpgcheck=0

Also check that dag repo is disabled.
  • RPMs installation :
yum install aii-dhcp aii-ks aii-pxelinux aii-server cdb-sync dhcp dnsmasq httpd  mod_ssl ncm-lib-blockdevices neon-devel squid subversion subversion-devel tftp-server java

AII configuration

DHCP

  • Edit /etc/dhcpd.conf in order to have the following content :
# Use this to enble / disable dynamic dns updates globally.
ddns-update-style ad-hoc;


# write here your network name
shared-network iihe.ac.be {

  deny unknown-clients;
  not authoritative;

  # Write here your domain name
  option domain-name "iihe.ac.be";


  # Parameters for the installation via PXE using pxelinux
  filename                           "pxelinux.0";
  # Uncommnent this line if ISC DHCP ver. 2
  # option dhcp-class-identifier       "PXEClient";
  # Uncommnent this line if ISC DHCP ver. 3
  option vendor-class-identifier       "PXEClient";
  option vendor-encapsulated-options 01:04:00:00:00:00:ff;
    
  # Complete with (at least) the gateway + DNS.
  # Hosts entries will be inserted
  # automatically by AII in this section

  subnet 193.190.247.0 netmask 255.255.255.0 {
    option routers 193.190.247.65;
    option domain-name-servers 193.190.247.140;
  }
  
  subnet 193.190.247.96 netmask 255.255.255.224 {
    option routers 193.190.247.113;
    option domain-name-servers 193.190.198.10;
  }

  subnet 192.168.0.0 netmask 255.255.0.0 {
    option routers 192.168.10.100;
    option domain-name-servers 192.168.10.100;
  }

}
  • Service starting :
chkconfig --add dhcpd
chkconfig --level 345 dhcpd on

TFTP

  • Edit /etc/xinetd.d/tftp :
  server_args             = -s /osinstall/nbp
  disable                 = no
  • Restart the service :
service xinetd restart

OS base install

  • Create and feed the /osinstall sub-directories (OS base installation done during kickstart step) :
mkdir -p /osinstall/nbp
mkdir -p /osinstall/ks
ln -s /osinstall/ks /var/www/html/ks
cp -a /usr/lib/syslinux/pxelinux.0 /osinstall/nbp/
cp -a /usr/share/doc/aii-server-13.1.0/eg/localboot.cfg /osinstall/nbp/pxelinux.cfg/

mkdir /osinstall/nbp/sl630_x86_64
cd /osinstall/nbp/sl630_x86_64
wget http://linuxsoft.cern.ch/scientific/6.3/x86_64/os/images/pxeboot/initrd.img
wget http://linuxsoft.cern.ch/scientific/6.3/x86_64/os/images/pxeboot/vmlinuz

Acknowledgement script

  • Do the following :
cp -a /usr/sbin/aii-installack.cgi /var/www/cgi-bin/
chmod o+rx /var/www/cgi-bin/aii-installack.cgi


Aii-shellfe configuration

  • Create /etc/aii/aii-shellfe.conf with the following content (replace "dhcp55" by the real short name of your Quattor deployment server) :
cdburl = https://dhcp55.iihe.ac.be:444/profiles
profile_prefix = profile_
use_fqdn = 1
  • Apache must be a sudoers. Add the following lines at the end of /etc/sudoers (replace "dhcp55" by the real short name of your Quattor deployment server) :
apache  dhcp55.iihe.ac.be=(ALL) NOPASSWD: /usr/sbin/aii-shellfe
apache  dhcp55.wn.iihe.ac.be=(ALL) NOPASSWD: /usr/sbin/aii-shellfe
and comment the following line in /etc/sudoers :
Defaults    requiretty

Deployment scripts

The "official" way to operate Quattor deployments as well as the SCDB tools needed, and how to install and configure them, is described here is full details : https://trac.lal.in2p3.fr/Quattor/wiki/Download/SCDB#Installationofdeploymentscripts

However, at IIHE, we prefer to work in another way : when the sysadmin wants to deploy some changes he has just committed, he must run the command "runcheck" from within a shell in the Quattor deployment server. This command is in fact a Python script that does the following things :

  1. checkout of the Quattor templates into a tmp directory;
2. ant-build of the templates;
3. if the the build was successful, the xml profiles are copied to the webserver directory so that they are available for download by the client machines, and
4. the client machines are notified.

The installation and configuration of runcheck is described here :

  • Log in on the Quattor deployment server and download and untar the following tarball :
wget http://quattor.begrid.be/begrid/install/cb-v4-client.tar.gz
tar xvzf cb-v4-client.tar.gz
  • /opt/CB<x> (where <x> is the version number of the Centralized BEgrid Repository) is the usual place to put the software :
cp -a cp /opt/CBx
(Change the value of x according to actual version of the Centralized BEgrid Repository you want to use.)
  • Let's now give explanations by sub-directories :
    • /opt/CBx/keys : contains the BEgrid CA certificate and a valid user *.p12 file used to connect authenticate against the SVN Quattor repository
    • /opt/CBx/subversion: some subversion specific parameters; edit the servers file:
    • correct full path to key (.p12 file)
    • plain-text password for the key (it does not prompt for the password)
    • /opt/CBx/tmp: will contain the checkout and build files.
    • /opt/CBx/private: here you can put private files (such as passwords and certificate in the templates. passwd.tpl; pub_key.tpl)
    • svncheck does this by simple copy from this directory into cfg/clusters. So keep that structure.
    • remove the template cluster given as example, otherwise runcheck will try to build it later ...
    • /opt/CBx/private/<clutername-glite-version>/passwd.tpl :
      • This file contains the passwords that wil be used for your site.
      • You can pick any password you like.
      • (Unless certain nodes are not configured with Quattor, in that case they must match whitch the non Quattor nodes).
    • /opt/CBx/private/<clutername-glite-version>/local_users.tpl :
      • ???
      • Not needed for a CE or a WN.
    • /opt/CBx/private/<clutername-glite-version>/pub_key.tpl :
      • Contains the SSH key that will be used for remote SSH access to the nodes.
      • More info on generating a key can be found here:
    • /opt/CBx/private/<clutername-glite-version>/<your_machine_fqdn>.tpl :
      • This is the place where to put the *.tpl files containing the hostcert.pem and hostkey.pem of your machine.
      • Go to this page to know how to generate these private templates.
    • /opt/CBx/svncheck: this is the code written by Jean-François Roche :
    • in config.conf you can specify most needed parameters :
      • svn_repos: point it to the trunk of the centralised-begrid repository
      • cluster_regexp: a regexp to build only these clusters matching the regexp
      • also adapt parameters in the email section
  • Update of the pysvn library :
    • runcheck uses pysvn library to access the Quattor SVN;
    • the library is stored in /opt/CBx/svncheck/pysvn;
    • the already existing library is outdated (built for i686 architecture), it must be updated;
    • to build new pysvn :
cd /root/
wget http://pysvn.barrys-emacs.org/source_kits/pysvn-1.6.2.tar.gz
tar xvzf pysvn-1.6.2.tar.gz
cd pysvn-1.6.2
 and follow the instructions in the INSTALL.html file.
    • once the build is finished, replace the old library by the new one :
mv /opt/CBx/svncheck/pysvn /opt/CBx/svncheck/pysvn_old
mkdir /opt/CBx/svncheck/pysvn
cp -a /root/pysvn-1.6.2/Source/pysvn/_* /opt/CBx/svncheck/pysvn/

HTTPD

  • Basic configuration of the service :
chkconfig --add httpd
chkconfig --level 345 httpd on
service httpd start

mkdir -p /var/www/https/profiles
  • Configuration of the reverse proxy with a cache :
    • Check that the modules mod_proxy and mod_cache are installed and loaded (see the httpd configuration).
    • Reverse proxy is the only one supported by Quattor: your profiles will point to the RPM repository at quattor.begrid.be, but in fact your local Quattor deployment server will get the RPMs, (in theory optionally) cache them, and provide them to node that is being installed.
    • Using a disk cache is preferred to lower the load on the CB and the network (and it should be faster).
    • Create the file /etc/httpd/conf.d/cb-cache.conf with the following content :

#
# Reverse Proxy  (Added for AII)
#
# Comment this line if modules are already loaded in your default httpd.conf
LoadModule proxy_module modules/mod_proxy.so

<IfModule mod_setenvif.c>
    BrowserMatch "rpm/.*" nokeepalive force-response-1.0
    BrowserMatch "Python-urllib/.*" nokeepalive force-response-1.0
</IfModule>


ProxyRequests Off
<Proxy *>
        Order deny,allow
        Allow from all
        SetEnv force-proxy-request-1.0 1
        SetEnv proxy-nokeepalive 1
</Proxy>

ProxyMaxForwards 15
ProxyReceiveBufferSize 0
ProxyTimeout 300

<Location /begrid/>
        ProxyPass http://quattor.begrid.be/begrid/
        ProxyPassReverse /
</Location>

<Location /packages/>
        ProxyPass http://quattor.web.lal.in2p3.fr/packages/
        ProxyPassReverse /
</Location>

<Location /13.1/>
        ProxyPass http://yum.quattor.org/13.1/
        ProxyPassReverse /
</Location>
#
# Disk Cache (Added for AII)
#
# Comment these lines if modules are already loaded in your default httpd.conf
LoadModule cache_module modules/mod_cache.so
LoadModule disk_cache_module modules/mod_disk_cache.so

## Directory to host the cache
CacheRoot /var/www/cache

## Max size of total cache in kb (obsoleted by Apache 2.2, use htcacheclean instead as explained below)
#CacheSize 15000000

CacheEnable disk /begrid
CacheEnable disk /packages
CacheEnable disk /13.1

## CacheDirLevels*CacheDirLength must be smaller than 20 !!
## don't set this higher than necessary
## following setting will create 64*64=4096 subdirectories
## for all possible hashes 64^22
CacheDirLevels 2
CacheDirLength 1

## in bytes (1GB, should be enough for openoffice)
CacheMaxFileSize 1000000000
CacheMinFileSize 1

## expire after 100 days
CacheDefaultExpire 8640000
CacheMaxExpire 10000000
    • Create the cache directory and restart Apache :
mkdir /var/www/cache;chown apache.apache /var/www/cache
/etc/init.d/httpd restart
    • Since Apache 2.2, the CacheSize directive is not used anymore. So to limit the size of the disk space allocated for caching, you will have to use htcacheclean. For that, create the following cron job in /etc/cron.hourly/htcacheclean-cron.sh :
#!/bin/sh
 
htcacheclean -v -n -p/var/www/cache -l15000000K

SINDES

You will find a presentation of SINDES on this page.

A few things you should now about about the current status of SINDES :

  • SINDES is not part of Quattor, it is not maintained any more. Some other solutions (like FreeIPA) are being investigated by developers...
  • Some SINDES RPMs are still available in BEgrid repositories. Though quite old now, the latest version available from BEgrid repos is still working with Quattor release 13.1 (the AII hook, aii_sindes, is still there).
  • Even if SINDES is not mandatory, it is the only way to secure deployment of machines using SSL.

Now, here is the procedure to install and configure SINDES, and make it working with Quattor 13.1 :

  • Installation of RPMs :
    • You first need to enable the old BEgrid repos, by creating the file /etc/yum.repos.d/cb-v5-sl5.repo :
[cb-v5]
name=CB server - client repo - SL5
baseurl=http://quattor.begrid.be/begrid/install/apt/RPMS.cb-v5_i386_sl5/
enabled = 1

[quattor]
name=Quattor repo - SL4
#baseurl=http://quattorsw.web.cern.ch/quattorsw/software/quattor/yum/1.3/i386/RPMS.quattor_sl4
baseurl=http://quattor.begrid.be/begrid/install/apt/RPMS.quattor_i386_sl4/
enabled = 1

[rpmforge]
name = Red Hat Enterprise  - RPMforge.net - dag
#baseurl = http://apt.sw.be/redhat/el5/en//dag
#mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge
#mirrorlist = ///etc/yum.repos.d/mirrors-rpmforge
baseurl = http://quattor.begrid.be/begrid/install/apt/RPMS.dag_i386_el5/
enabled = 1
protect = 0
    • Packages installation :
wget http://quattor.begrid.be/begrid/install/apt/RPMS.cb-v5_i386_sl5/cb-client-sindes-0.6.0-sl50.1.noarch.rpm
yum localinstall cb-client-sindes-0.6.0-sl50.1.noarch.rpm
 The result should be :
Installing:
 cb-client-sindes                                         noarch                             0.6.0-sl50.1                                  /cb-client-sindes-0.6.0-sl50.1.noarch                             7.0 k
Installing for dependencies:
 SINDES-Shell-bin                                         noarch                             0.5-34                                        cb-v5                                                             9.1 k
 SINDES-ca                                                noarch                             0.9.99-2.el5                                  cb-v5                                                              15 k
 perl-Config-IniFiles                                     noarch                             2.72-2.el5.2                                  sl5-epel                                                           49 k
 perl-Crypt-SSLeay                                        x86_64                             0.57-3.el5.rfx                                sl5-addons                                                         96 k
 perl-Date-Manip                                          noarch                             5.56-1.el5.rf                                 sl5-addons                                                        211 k
 perl-IO-stringy                                          noarch                             2.110-5.el5                                   sl5-epel                                                           70 k
 perl-IPC-Shareable                                       noarch                             0.60-3.el5                                    sl5-epel                                                           39 k
 perl-List-MoreUtils                                      x86_64                             0.33-5.el5                                    sl5-epel                                                           75 k
 perl-Log-Dispatch                                        noarch                             2.21-1.el5.rf                                 rpmforge                                                           81 k
 perl-Log-Dispatch-FileRotate                             noarch                             1.19-1.el5.rf                                 sl5-addons                                                         24 k
 perl-Log-Log4perl                                        noarch                             1.15-1.el5.rf                                 rpmforge                                                          385 k
 perl-MIME-Lite                                           noarch                             3.021-1.el5.rf                                rpmforge                                                           95 k
 perl-Mail-Sender                                         noarch                             0.8.13-2.el5.1                                sl5-epel                                                           53 k
 perl-Mail-Sendmail                                       noarch                             0.79-9.el5.1                                  sl5-epel                                                           27 k
 perl-MailTools                                           noarch                             2.02-1.el5.rf                                 rpmforge                                                          100 k
 perl-Params-Validate                                     x86_64                             0.91-1.el5.rf                                 sl5-addons                                                        105 k
 perl-SINDES-GetCertificate                               noarch                             0.9.99-1.el5                                  cb-v5                                                              19 k
 perl-SINDES-Shell                                        noarch                             0.5-30                                        cb-v5                                                              21 k
 perl-SINDES-common                                       noarch                             0.5-24                                        cb-v5                                                              47 k
 perl-Term-Shell                                          noarch                             0.02_cern-1                                   quattor                                                            32 k
 perl-TimeDate                                            noarch                             1:1.16-5.el5                                  sl-base                                                            32 k
 perl-XML-DOM                                             noarch                             1.44-2.el5.rf                                 rpmforge                                                          188 k
 perl-XML-RegExp                                          noarch                             0.03-2.el5                                    sl5-epel                                                          8.2 k
    • Disable the old BEgrid repos, by setting enabled=0 everywhere in /etc/yum.repos.d/cb-v5-sl5.repo.
    • We also aii_sindes.pm :
wget http://quattor.begrid.be/begrid/Central_BEGrid_Repository/i386_quattor_sl4/aii_sindes-0.2.4-1.noarch.rpm
yum --nogpgcheck localinstall aii_sindes-0.2.4-1.noarch.rpm
    • And since aii_sindes.pem is expected to be in a directory AII :
mkdir /usr/lib/perl/AII
cp -a /usr/lib/perl/NCM/Component/aii_sindes.pm /usr/lib/perl/AII/
 and replace this line in /usr/lib/perl/AII/aii_sindes :
package NCM::Component::aii_sindes;
 by :
package AII::aii_sindes;
    • To configure SINDES, follow the instructions on this page.
    • Adapt AII configuration to SINDES, by adding the following lines to /etc/aii/aii-shellfe.conf :
cert_file = /etc/sindes/certs/apache.crt
key_file = /etc/sindes/keys/apache.key
ca_file = /etc/sindes/certs/ca.crt
    • Important requirement on the client side : openssl should be upgraded to at least the same version as on the deployment server, that is 0.9.8e-26. And since SINDES should be working before the SPMA step, so that the machine is able to download its XML profile, the following things must be done :
    • Make sure that you have at least version 0.9.8e-26 of openssl in client machines configuration.
    • Add openssl package in the list AII_OSINSTALL_BASE_PACKAGES (see the quattor/aii/ks/config template).


Template:TracNotice