CMSSWDeployment OldPage

From T2B Wiki
Jump to navigation Jump to search

CMSSW Deployment for the Brussels team

Useful links:

Registrations:

  • Subscribe to the Software Release Announcements
    • This list will tell you what to install when it is released /which release will be deprecated at which moment
  • Subscribe to the Software Distribution Tools Discussions
  • Subscribe to Facilities Operations
  • Subscribe to General Data Operations Discussions
  • Subscribe to Grid Announcements
  • Subscribe to Software Development Tools
  • To do the installation, make sure you have the lcgadmin role in CMS VO

Sites we are responsible for at the moment:

  • List of site we are responsible for at the moment
  • Old list of sites we are responsible for, this can be considered as obsolete (it is kept because for some sites there is a comment if a tag needs to be set by hand, and this is still true for some sites! E.g. French T1 coupled with T2!)
  • You can find the whole list in the file CompleteCElist in the directory: /user/cmssw/SWdeployment. Copy it in a new file before to make any change.

Special requests from sites:

  • Requests of releases from sites to be kept. IMPORTANT: the installation of a "PRE-RELEASE" has to be officially required during the DORP meeting.

Place to work from:

  • Login as cmssw on m1.iihe.ac.be (ssh -2X cmssw@m1.iihe.ac.be)
  • Copy your userkey.pem and usercert.pem naar .globus_user, where "user" is a name identifying yourself ;-)
  • create a proxy:
    • In the pwd (=/user/cmssw) directory there is a script CreateProxy.sh to create a proxy
    • Specify as an option the name you used for "user" in .globus_user, e.g. for .globus_NicePerson
source CreateProxy.sh NicePerson 
  • Store this proxy somewhere so that several proxies of different users can exist next to eachother

Removal of CMSSW releases

  • Removing software tags:
    • First check on the SAM page if the CMSSW release you want to remove is installed at the CE. If not, you don't have to submit a job ;-)
    • Check if the site is publishing the software tags.
    • A polite way to remove software is to remove first the software tags, do this e.g. one day before you actually remove the software, so running userjobs can finish properly.
    • Due to a new SAM test, removing a tag a day before you physically remove the software from a site can lead to discrepancy. To avoid this we should do
Set a tag "VO-cms-CMSSW_X_Y_Z_remove_scheduled"
Remove tag "VO-cms-CMSSW_X_Y_Z"
    • See for the lcg-tags commands described in the section Managing the Software Tags on how to do this.
  • Removing a CMSSW release:
    • To remove a software release the script with the name install_on_lcg.pl in the directory SWinstall is used. This script needs 3 arguments:
./install_on_lcg.pl RMXYZ siteCEname CMSSW_X_Y_Z
    • This script will make a jdl file to be send on the grid which will take care of the installation/removal at a remote site (CE).
    • Execute the script to remove software release XYZ at CE siteCEname, RMXYZ is string which will be used to create the names of the jdl and the sh, RM stands for Remove, XYZ stands for software release CMSSW_X_Y_Z.
    • The first thing you should do is check if the requirements in the jdl are fulfilled by doing:
glite-wms-job-list-match -a -c glite-wms2.conf RMXYZ_siteCEname.jdl
    • The output of this command lists all queues for the siteCEname which satisfy the requirements in the file RMXYZ_siteCEname.jdl
    • if the output of the previous is OK, you can submit the job:
glite-wms-job-submit -a -c glite-wms2.conf -o id_siteCE RMXYZ_siteCEname.jdl
    • The -o id_siteCE stores the jobId in a small file with name id_siteCE
    • Check the submission of the job with
glite-wms-job-status -i id_siteCE
    • Timeline for this kind of job is around 30 minutes, but there is a wide spread. In general you can state that if the job did not succeed after 6 hours that something probably went wrong.
    • If job is done, retrieve output
glite-wms-job-output -i id_siteCE --dir $PWD
    • where the last option makes sure that the output is stored in the current directory. If you forget to specify the --dir then the output is written to /tmp.
  • After removal check in the logfile of the job if the "CMSSW rpm list after install" agrees with what you see with SCRAM.
    • If there are rpm remnants of releases which are removed from the SCRAM list, but not from the rpm list, you should send a new removal job with a longer timeout for the cmsCleanSWArea script called by install_on_lcg.sh

Installation of CMSSW releases:

  • Deployment of a release at 1 site:
    • In XCMSI_new directory there is a script called cmsg.pl
    • Execute it with
./cmsg.pl -t apt -a slc4_ia32_gcc345 -i CMSSW_2_1_4 -s siteCEname
    • In the previous command -t is to specify the type, now (almost) everything is "apt". -a is to specify the "scram architecture", in general this is slc4. -i is to specify the software release to be installed and -s is to specify the siteCEname
    • The script prints a lot of output on the screen. Somewhere in the last lines you can see the jobId (GUID). Save this id in an idfile for each specific software release (in this case CMSSW_2_0_5).
echo jobId >> ids_205
    • In this case "jobId" is the Id given by GUID as output of the script and ids_205 is the idfile for software release CMSSW_2_0_5
    • Check the job in the same way as you do with a removal job
    • After retrieval of the output check the outputfile to see if everything worked
    • The software tag is automatically created (you can check if it is really there with the commands described in Managing the Software tags).
    • Every time you install something check if the SAM is ok and the new software release is seen
    • If the installation doesn't work and you get the error:
SubmitJDL: WARNING! String  does not look like a valid GUID, submission failed!
Use of uninitialized value in hash element at ./cmsg.pl line 414.
  you can submit the jdl file by hand using the following command:
glite-wms-job-submit -a -c glite-wms2.conf -o id_ISsite tmp/grid_install_sw_apt_something.jdl
  and "Good Luck"!!!! ;)
  • Deployment for a list of sites:
    • There is a shell script installscript.sh in the directory scripts. This script takes as arguments sitelist and the software version you want to be installed
./installscript.sh sitelist CMSSW_X_Y_Z
    • It will both write the output on the screen as to a log file InstalLogFile_CMSSW_X_Y_Z.txt
    • attention: The script will get stuck since the installation script tries to start a monitoring application. This should be removed. After a job is submitted and the installation script gets stuck, press CONTROL+C.
    • Check the jobs, retrieve the output (note that glite output 'Done with exit code 0' doesn't mean that all went fine, you have to check the SAM pages!).

Managing the Software Tags:

  • In the XCMSI/scripts directory there is a script called lcg-tags
  • To check the software tags on a certain CE, use
lcg-tags --vo cms --ce siteCEname --list 
  • In the same way you can do
lcg-tags --vo cms --ce siteCEname --remove --tags tagnames
  • and
lcg-tags --vo cms --ce siteCEname --add --tags tagnames

Rebuilding the RPM database

  • In some cases it is needed the rebuild the RPM database at a site. E.g. when you've submitted two installation/removal jobs to a site at the same moment. Since those jobs will access the db at the same time the interference will mess up the database.
  • To recover the database one has to send a job using the install_on_lcg.pl script you find under the /user/cmssw/RPM directory. The procedure is completely similar with the one for sending a removal job. The difference with removal lies in the script itself. In the install_on_lcg.pl script in the RPM directory you will see two lines uncommented while they are commented in the script under SWinstall.
 rm -fR $CMS_PATH/$SCRAM_ARCH/var/lib/rpm/__db.*
 rpmdb -vv --rebuilddb --dbpath $CMS_PATH/$SCRAM_ARCH/var/lib/rpm
 
  • Those lines will make sure the db gets rebuilt. You'll also see that the line
  RemoveRelease $swrelname
  
is commented so that you can give a nonsense argument as third argument when executing the script.

HOW TO remove/install software using a more automated tool

  • In /user/cmssw/SWdeployment there is a script JORIS.pl which takes care of creation/submission/follow-up of installation and removal jobs at T1/T2 sites. JORIS stands for Job Organizer for Remote Installation of Software. You can use this script to easily install or remove software at a list of sites. This script isn't far from mature so comments and improvements are more than welcome
  • To remove software release CMSSW_A_B_C one should perform the following steps
    • 1. Create the .jdl and .sh file you want to submit to the grid
   ./JORIS.pl --remove CMSSW_A_B_C --create mySiteCElist
   
    • The input you have to give is a file containing a (line break separated) list with the names of the CE's you want to have CMSSW_A_B_C removed
    • The script will create a directory RM_CMSSW_A_B_C containing the scripts and a file siteCElist which is a copy of mySiteCElist
    • 2. Replace the current software tag VO-cms-CMSSW_A_B_C by VO-cms-CMSSW_A_B_C_remove_scheduled
   ./JORIS.pl --remove CMSSW_A_B_C --tag rp
   
    • The script will replace (rp) the software tags at the sites in siteCElist in RM_CMSSW_A_B_C
    • 3. Submit the freshly created scripts
   ./JORIS.pl --remove CMSSW_A_B_C --submit
   
    • The script will submit all the *.jdl files in the RM_CMSSW_A_B_C directory
    • It will write the Grid Job Id's in *.id files
    • Remark, step 1, 2 and 3 can be combined to create, submit and change the tags at the same time but keep in mind that it is more user friendly to first remove the tag before actually removing the software.
    • 4. Check the status of the jobs
   ./JORIS.pl --remove CMSSW_A_B_C --status
   
    • The script will look at the status of each *.id file in RM_CMSSW_A_B_C
    • 5. Retrieve the output of the jobs
   ./JORIS.pl --remove CMSSW_A_B_C --output
   
    • The script will retrieve the output of each *.id file in a new directory RM_CMSSW_A_B_C/output
    • 6. Remove the software tag VO-cms-CMSSW_A_B_C_remove_scheduled
   ./JORIS.pl --remove CMSSW_A_B_C --tag rm
   
    • The script will remove (rm) the software tags at the sites in siteCElist in RM_CMSSW_A_B_C
    • Exactly this option can also be used to remove the tags VO-cms-CMSSW_A_B_C without replacing it
  • To install software one should perform the following steps
    • 1. Create and submit the .jdl and .sh file you want to submit to the grid
   ./JORIS.pl --install CMSSW_A_B_C --create mySiteCElist --submit
   
    • Caution; in opposite to the case of removal one is obliged to give both the --create and --submit option at the same time since the script behind this takes care of both at the same time
    • Cauton 2; the adding of the software tag is done automaticely unless you have a site where the sw area is shared but the software has to be tagged seperately
    • The input you have to give is a file containing a (line break separated) list with the names of the CE's you want to have CMSSW_A_B_C removed
    • The script will create a directory IS_CMSSW_A_B_C containing a file siteCElist which is a copy of mySiteCElist and the jobid's *.ids
    • 2. Check the status of the jobs
   ./JORIS.pl --install CMSSW_A_B_C --status
   
    • The script will look at the status of each *.id file in IS_CMSSW_A_B_C
    • 5. Retrieve the output of the jobs
   ./JORIS.pl --install CMSSW_A_B_C --output
   
    • The script will retrieve the output of each *.id file in a new directory IS_CMSSW_A_B_C/output
    • Wishlist (add one if you think of a nice feature):
    • also use this script to send an RPM rebuild job (not too difficult, should have an extra option like --remove/--install/--rpmrebuild which is pointing to a slightly modified install_on_lcg script)
    • when a job get submitted, the sitename is added to a list (is just a copy of sitelist you give as an argument). Once you try to submit another job you get an error. The difficult part here is to have the site disappearing from the list if the output status is done.

HOW TO install Generator Gridpack

  • In /user/cmssw/MadGraphLCG there is a script MADJORIS.pl (JORIS.pl adapted to gridpack installation) which takes care of creation/submission/follow-up of gridpacks installation jobs at T2 sites.
  • One can run these jobs in parallel with each other as well as in parallel with the CMSSW installation/removal jobs.
  • To install generator gridpacks one should perform the following steps
    • 1. Open the install_on_lcg.pl file and change the following settings
   generator="madgraph"
   version="4.2.11-cms2"
   categories="QCD benriched Vjets ttjets"
   energy=10TeV  
   
    • 2. Create the .jdl and .sh file you want to submit to the grid
   ./MADJORIS.pl --install Gridpackname --create mySiteCElist
   
    • 3. Submit the freshly created scripts
   ./MADJORIS.pl --install Gridpackname --submit
   
    • The script will submit all the *.jdl files in the MG_Gridpackname directory
    • 4. Check the status of the jobs
   ./MADJORIS.pl --install  Gridpackname --status
   
    • The script will look at the status of each *.id file in MG_Gridpackname directory
    • 5. Retrieve the output of the jobs
   ./MADJORIS.pl --install Gridpackname --output
   
    • The script will retrieve the output of each *.id file in a new directory MG_Gridpackname/output

CODE ON CVS

  • Whenever you make changes to scripts, make sure that the script is put on CVS!
 http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/UserCode/CMSSWdeployment/
 

Troubleshooting:

  • Problem at CE:
    • First check SAM tests (on SAM website, for link see top of this twiki)
  • Software tag looks similar to:
  VO-cms-CMSSW_2_0_8_install-failed-with-6400-on-2008/06/10_18:03:09
  
    • The installation failed
    • The code, in this case 6400, which you can also find in the .log file, is rather meaningless.
    • Retrieve the job output and try to find from the combination of the .log file and the .err file what went wrong.
  • The .err file tells (option 1):
  rpmdb: Lock table is out of available locks
  error: cannot open Packages index using db3 - Cannot allocate memory (12)
  
    • It indicates the RPM database is broken. To repair, see above.
    • This could also point to an nfs locking problem. link The behaviour that ones sees is a job that keeps on running (usually the site-admin can check what the job is doing) and is actually stuck on the apt-get update step. To solve this the site admin can do the following:
 Somewhere under /$VO_CMS_SW_DIR/slc4_ia32_gcc345/var/lib/apt/lists there is a file "lock". Remove it. 
 There might be other files named lock (or similar) in $VO_CMS_SW_DIR/slc4_ia32_gcc345/var/lib. Remove them if you find those.
 Maybe also a file similar to /opt/grid/lcg_experimental_sw/x86/cms/slc4_ia32_gcc345/var/lib/rpm/__db.00* should be removed
 
 after doing this it might work immediately but also an additional rebuild of the db could be needed.


  • The exit code of the job is not 0 and the .err file tells (option 2):
  memory alloc (4 bytes) returned NULL.
  E: Sub-process /opt/exp_soft/cms/slc4_ia32_gcc345/external/apt/0.5.15lorg3.2-CMS3/bin/rpm-wrapper returned an error code (1)
  
    • There is not enough memory to install the software release and the RPM database is broken.
    • Repair the RPM database
    • Remove more deprecated releases (or ask to site-admins which SLC4 based releases can be removed)
    • Try to install the software release again after you freed some space
  • The .err file tells:
 rpmdb: Locker is not valid
 rpmdb: Unknown locker ID: 33e 
 error: db4 error(22) from db->close: Invalid argument
 error: cannot open Packages index using db3 - Invalid argument (22)
 error: cannot open Packages database in /afs/in2p3.fr/grid/toolkit/cms2/afs/in2p3.fr/grid/toolkit/cms2/slc4_ia32_gcc345/var/lib/rpm
 E: could not open RPM database
 
or
 rpmdb: Program version 4.3 doesn't match environment version
 error: db4 error(-30974) from dbenv->open: DB_VERSION_MISMATCH: Database environment version mismatch
 error: cannot open Packages index using db3 -  (-30974)
 error: cannot open Packages database in /afs/in2p3.fr/grid/toolkit/cms2/afs/in2p3.fr/grid/toolkit/cms2/slc4_ia32_gcc345/var/lib/rpm
 E: could not open RPM database
 
remove the index files and rebuild the RPM database.
  • Error with the ...look.up file OR users not able to get the release compiling due to permissions:
    • This happens sometimes when the umask settings at the site are too restrictive
    • In the remove script (install_on_lcg.pl) you see a line commented:
  #echo "=== changing the permissions ==="
  #find \$VO_CMS_SW_DIR/slc4_ia32_gcc345/ -not -perm +go+r -print -exec chmod go+r {} \\; 
  
    • Uncomment these lines and comment
  RemoveRelease $swrelname
  
    • Send the job as described in the section Removal of Software Releases
    • To check if the job was successful in changing the permissions, check the output of the job. You should see something like
  === changing the permissions ===
  umask -S
  u=rwx,g=r,o=r
  
  • No compatible resources:
    • If the site is NOT ok in the SAM tests (check also when was the latest check!), the site is in trouble, kill the job, try submission again later
    • If the site is ok in the SAM tests and it is an installation job, the wrong memory value is published for this site. Open the file jdl.tmpl in XCMSI_new and change the requirement for the jdl.tmpl to a lower value. This is needed for Vienna, Ciemat, Lal, ... You can check the memory value by issuing the command:
   lcg-info --list-ce --vo cms --query "CE=*ciemat*" --attrs Memory,MaxWCTime
   
  • Problem with certificate:
    • When you use one of the lcg-tags commands and, you see the error
  error: cannot download /etc/sysconfig/edg from CE.
  lcg-tags: error:
  error: globus_ftp_client: the server responded with an error
  535 535-FTPD GSSAPI error: GSS Major Status: Authentication Failed
  535-FTPD GSSAPI error: GSS Minor Status Error Chain:
  535-FTPD GSSAPI error:
  535-FTPD GSSAPI error: accept_sec_context.c:170: gss_accept_sec_context: SSLv3 handshake problems
  535-FTPD GSSAPI error: globus_i_gsi_gss_utils.c:881: globus_i_gsi_gss_handshake: Unable to verify remote side's credentials
  535-FTPD GSSAPI error: globus_i_gsi_gss_utils.c:854: globus_i_gsi_gss_handshake: SSLv3 handshake problems: Couldn't do ssl handshake
  535-FTPD GSSAPI error: OpenSSL Error: s3_srvr.c:1816: in library: SSL routines, function SSL3_GET_CLIENT_CERTIFICATE: no certificate returned
  535-FTPD GSSAPI error: globus_gsi_callback.c:351: globus_i_gsi_callback_handshake_callback: Could not verify credential
  535-FTPD GSSAPI error: globus_gsi_callback.c:477: globus_i_gsi_callback_cred_verify: Could not verify credential
  535-FTPD GSSAPI error: globus_gsi_callback.c:769: globus_i_gsi_callback_check_revoked: Invalid CRL: The available CRL has expired
  535 FTPD GSSAPI error: accepting context
  
    • First check if your proxy has the correct role and if this role is correctly seen by the involved site.
    • If this is all ok, then maybe the site does not have the uptodate cert/CRL for Belgium CA. Ask the site to check this and update the file with the most recent file if necessary.
  • Software tag is not published:
    • Check the log file of the installation.
    • If you read:
  -ce ingrid.cism.ucl.ac.be --add -tags "VO-cms-slc4_ia32_gcc345" returned error 512
  grid_install_sw_apt_ingrid.cism.ucl.ac.be_CMSSW_2_0_8.pl: WARNING! 
  Command  /home/condor/local/execute/dir_23308/https_3a_2f_2fgrid-lb0.desy.de_3a9000_2fbDsR-fI45JQf3yeFRE91nA/scripts/lcg-tags 
  
    • Try to set the software tag by hand, see the section "Managing the Software Tags". If this doesn't work because of
  lcg-tags: error: lock file already present.
  lcg-tags: error: cannot create remote lockfile.
  
 remove the lock file
  edg-gridftp-rm gsiftp://ingrid.cism.ucl.ac.be:2811/opt/edg/var/info/cms/lock
  
 and try again.
 To remove the lock file you can use
 
     uberftp citeCEname
     cd /opt/edg/var/info/cms
     rm lock
  
 note that the location of the lock file is CE-dependent.
    • If the software tag is there when you check with the lcg-tags command but not on the SAM page, then probably something is wrong with the transfer. Maybe a bdii process needs to be restarted, ask the siteadmins to check what's wrong.
  • Installation fails and you read the following lines in the .err file:
 E: Could not get lock 
 /opt/exp_soft/cms/slc4_ia32_gcc345/var/lib/cache/slc4_ia32_gcc345/lock - 
 open (11 Resource temporarily unavailable)
 E: Unable to lock the download directory
 
If the site runs NFS for the SW area, then some nfslock daemons is not running, either on the server or on the WN.Site admins need to be involved to solve this.
  • If in the output of the job you see something like: "E: Dynamic MMap ran out of room"
    • This happens because CMS reached the limit of the cache size of apt-get. The recipe to fix it is described here.
    • There is a template fixapt_on_lcg.pl that should generate proper jobs. It works like the script for the removal procedure (install_on_lcg.pl) and submit as usual.
    • Check in the log file that the fix worked, you should see something like this: APT::Cache-Limit 33554432;
  • In the output of the installation job you see:
 /opt/exp_soft/cms/bootstrap-slc4_ia32_gcc345.sh: Permission denied
 
or
 cp: cannot create regular file `/nfs/exp_software/cms/slc4_ia32_gcc345/lcg/SCRAMV1/scramdb/project.lookup-BAK': Permission denied
 ./RM208_ce2.ppgrid1.rhul.ac.uk_install_on_lcg.sh: line 19: 
  /nfs/exp_software/cms/slc4_ia32_gcc345/lcg/SCRAMV1/scramd /project.lookup-NEW: No such file or directory 
 
    • This means that the software manager is mapped to a pool of accounts (which is the default (probably after some updates from the side of the site)).
    • Ask the site-admin to make sure that all lcgadmins get mapped to the same user and that the files in $VO_CMS_SW_DIR have the correct permissions.
  • If you see something like this in the error message:
 E: You don't have enough free space in /opt/exp_soft/CMS/slc4_ia32_gcc345/var/lib/cache/slc4_ia32_gcc345/
 
Then there is the possibility that either there is really not enough space or the job is running on the wrong WN (probably after a reconfiguration of the siteadmins). Let the siteadmins investigate the problem.
  • If the Installation succeeded and the tag is present in the tag list but it is not in the scram list and you see the following lines in the log file:
   cms+cmssw+CMSSW_3_2_8 is already the newest version.
   0 upgraded, 0 newly installed, 0 removed and 3 not upgraded. 
 
this means that the release is not properly registered in SCRAM. Christoph can send a job to fix the SCRAM DB.
  • If you have the following problem during the installation of MadGraph! gridpacks
 ==== Will start Madgraph installation ====
 Started at Tue Mar 31 10:25:22 CEST 2009
 INFO doing ttjets
 INFO Doing ./cmsDownloadME.py --gen madgraph --cat ttjets --energy 10TeV --ver 4.2.11-cms3
 Considering: /opt/exp_soft/cms/slc4_ia32_gcc345/generatorData/madgraph/10TeV/ttjets/4.2.11-cms3/ttbar_5flavours_xqcut40_10TeV_pack_32b.tar.gz
  Fetching file (new)....
 Traceback (most recent call last):
 File "./cmsDownloadME.py", line 173, in ?
  os.makedirs(dir)
 File "/usr/lib64/python2.3/os.py", line 154, in makedirs
  mkdir(name, mode)
 OSError: [Errno 13] Permission denied: '/opt/exp_soft/cms/slc4_ia32_gcc345/generatorData/madgraph/10TeV/ttjets/4.2.11-cms3'
 ERROR cmsDownloadME.py --gen madgraph --cat ttjets --energy 10TeV --ver 4.2.11-cms3
 
there is a problem with the permissions of the ttjets directory or one of the subdirectories. Contact the siteadmins to solve this issue.
  • Problem with index file during removal of RPM which was still there after the CMSSW release was removed. If you see output similar to the following in the '.out' of a removal job
 rpmdb: /opt/exp_soft/cms/slc4_ia32_gcc345/var/lib/rpm/Packages: page 1: unpinned page returned
 rpmdb: PANIC: Invalid argument
 error: db4 error(-30977) from dbcursor->c_get: DB_RUNRECOVERY: Fatal error, run database recovery
 
You can solve this by commenting the lines in the 'install_on_lcg' script:
 echo "==== remove index files ===="
 rm -fR $CMS_PATH/$SCRAM_ARCH/var/lib/rpm/__db.*
 

Tips and Tricks

  • Check if a site is available: lcg-infosites --vo cms ce | grep "gridce.iihe.ac.be"
  • Use an already existing valid proxy of one of your colleagues e.g. user2:
    • First list the proxy which is currently be used (maybe there is no existing proxy!):
  echo $X509_USER_PROXY
  
    • If you see something like:
  /user/cmssw/k5-ca-proxy_user1.pem
  
    • There is indeed an existing proxy, namely from user1.
    • To change it to user2:
  export X509_USER_PROXY=/user/cmssw/k5-ca-proxy_user2.pem
  



Obsolete scripts

    • How to use software tags using removetags.sh in the directory /user/cmssw/scripts.
    • One can choose to give 2 or 3 arguments.
    • The first argument is a file containing the sites you want to remove the tag from. In the same directory there is sitelist_all listing all the sites we maintain.
    • The second argument is the tag you want to be removed
    • The third argument (optional) is the tag you want to add
    • Removal for a list of sites:
    • To ease the removal step if you have to remove releases for a list of sites, there is a shell script in the directory scripts. This shell script will also check if the requirements in the file RMXYZ_siteCEname.jdl are fulfilled and if the job is ready to be submitted.
    • For now the script does not submit these jobs but will list the commands to do so
    • Installation at a list of sites:
    • There is a shell script installscript.sh in the directory scripts. This script takes as arguments sitelist and the software version you want to be installed
./installscript.sh sitelist CMSSW_X_Y_Z
    • It will both write the output on the screen as to a log file InstalLogFile_CMSSW_X_Y_Z.txt
    • attention: The script will get stuck since the installation script tries to start a monitoring application. This should be removed. After a job is submitted and the installation script gets stuck, press CONTROL+C.
    • Check the jobs, retrieve the output (note that Done with exit code 0 doesn't mean that all went fine, you have to check the SAM pages!).



Template:TracNotice