Forgot your password?
Document Actions
Installing
Installing gLite from scratch was not without problems. This document describes the basic steps that were determined by trial and error.
Installing rpms
You can update the rpms using:apt-get update
apt-get dist-upgrade
Configuring nodes
grid4
Grid4 is gliteCE and BDII_site, configuration requires the following, but see note about edg_mkgridmapfile below:
- /opt/glite/yaim/scripts/configure_node site-info.def gliteCE BDII_site
In /etc/cron.d/check-tomcat, add a log file to save an hourly e-mail:
- 10 * * * * root /etc/rc.d/init.d/tomcat5 start >> /var/log/cron-check-tomcat 2>&1
grid3
- Grid3 is currently our SE (Storage element node). Configuration is carried out as follows:
/opt/glite/yaim/scripts/configure_node site-info.def SE_classic
In /etc/cron.d/check-tomcat, add a log file to save an hourly e-mail:
- 10 * * * * root /etc/rc.d/init.d/tomcat5 start >> /var/log/cron-check-tomcat 2>&1
fire
Fire is the worker node host, running torque server.wget http://grid-deployment.web.cern.ch/grid-deployment/download/relocatable/gliteUI_WN-3.0.4.tar.gz
tar xvzfgliteUI_WN-3.0.4.tar.gzwget http://glitesoft.cern.ch/EGEE/gLite/APT/R3.0/rhel30/RPMS.Release3.0/glite-ce-blahp-1.5.10-1.i386.rpm
rpm -ihv glite-ce-blahp-1.5.10-1.i386.rpm/opt/glite/bin/BLParserPBS -p 33332 -s /opt/torque(should run as a service from init.d)
export PATH="/home/local/glite-WN/python/bin:$PATH"
/home/local/glite-WN/glite/yaim/scripts/configure_node site-info.def TAR_WN
Create the pool accounts:
/home/local/glite-WN/glite/yaim/scripts/run_function site-info.def config_usersNeed to also add pool accounts to pbs
Something like this:
for user in $(grep negtb /etc/passwd | awk -F ':' '{print $1}'); \
do qmkuser $user; qmkmember -a egee $user; done
- In /opt/edg/etc/profile.d/edg-wl-ui-gui-env.[c]sh,
/opt/edg/var/etc/profile.d/edg-wl-ui-gui-env.[c]sh and
/opt/glite/etc/profile.d/glite-wmsui-gui-env.[c]sh, modify default value of
JAVA_INSTALL_PATH to point at your Java.
ln -s /home/local/glite-WN/etc/profile.d/grid_env.sh /home/local/glite-WN/etc/profile.d/grid_env.csh /etc/profile.d
In addition needed to create a custom script/function for configuring torque server on fire. This can currently be executed thus:
/home/local/glite-WN/glite/yaim/scripts/configure_node site-info.def FIRE
or:
/home/local/glite-WN/glite/yaim/scripts/run_function site-info.def fire_config_existing_torque
But needs to be extracted as a standalone script for later use. This is because the normal config_torque_server function relies on a very old torque version.
After setting up the queues using this command, you can check the status using the following ldap command:
ldapsearch -x -H ldap://grid4.bccs.uib.no:2170 -b mds-vo-name=UiB,o=grid
This status should eventually be reflected on the onitoring site http://goc.grid.sinica.edu.tw/gstat/UiB/ but it does take some time (1-2 hours).
and search for GlueCEStateStatus
You need to regularly update the CAs, read this http://grid-deployment.web.cern.ch/grid-deployment/lcg2CAlist.html
Using yaim you can carry out the following commands:
# yum install lcg-CA
Make sure that you have the following in your /etc/yum.conf:
[CA]gpgcheck=0 is important
name=CAs
debuglevel=2
gpgcheck=0
baseurl=http://linuxsoft.cern.ch/LCG-CAs/current
The shutting down a cluster section here "http://egee.grid.cyfronet.pl/grid-operations/procedures/" also proved useful.
UI
The UI (user client) is installed for BCCS workstations in /net/pl/glite.
After any changes to the site-info.def file, run the configure script:
/net/pl/glite/glite/yaim/scripts/configure_node site-info.def TAR_UI
edg_mkgridmapfile
With installation of a gliteCE node, edg_mkgridmapfile should not be running. gliteCE uses VOMS grid-mapfile format and edg_mkgridmapfile generates a gridmap in the wrong format. Check that there is no entry for edg_mkgridmapfile in /etc/cron.d. If you install 2 nodetypes at once then the yaim scripts generate an edg gridmap file by mistake.
The problem is due to a bad egrep string in config_mkgridmap. The easiest solution is to comment out the whole section of /opt/glite/yaim/functions/config_mkgridmap starting at
if ( ! echo "${NODE_TYPE_LIST}" | egrep -q '^ *(gliteCE|FTS|LFC) *$' ); thenand then re-run configure-node.Integrating local VOMS
In order to grant local users access, we have set up our own VOMS server for bccs and NorGrid on hydra.ii.uib.no. In order to allow access to these users, we have added this to our list of supported VOMS servers. In site-info.def you should have:
VOS="bccs alice atlas biomed cms dteam lhcb ops"
and then:
VO_BCCS_USERS=""In order for the grid node to access the VOMS server and obtain user credentials we have to go into the VOMS server as administrator and grant 'list-users' permission for machines with a valid certificate.
VO_BCCS_VOMS_SERVERS="vomss://hydra.ii.uib.no:8443/voms/bccs"
VO_BCCS_SGM=""
VO_BCCS_QUEUES="bccs"
VO_BCCS_SW_DIR=$VO_SW_DIR/bccs
VO_BCCS_DEFAULT_SE=$CLASSIC_HOST
VO_BCCS_STORAGE_DIR=$CLASSIC_STORAGE_DIR/bccs
VO_BCCS_VOMS_EXTRA_MAPS=""
VO_BCCS_VOMSES="bccs hydra.ii.uib.no 15001 /O=Grid/O=NorduGrid/CN=host/hydra.ii.uib.no bccs"
VO_BCCS_VOMS_POOL_PATH=""
iptables and open ports
For votag SFT test, we get an error
error: a system call failed (No route to host)
The offending command was:
lcg-ManageVOTag -vo ops -host grid4.bccs.uib.no --list
This error is rectified by adding port 2811 to the list of allowed hosts on grid4. This was determined by using strace.
SAM tests
Not all SAM tests completed successfully initially. Here is a summary of what had to be done to get through the tests:
votag - See above
rgmasc - failed with
No CA files found matching "/etc/grid-security/certificates/*.0
Solved by reading bug report 15612, http://savannah.cern.ch/bugs/?func=detailitem&item_id=15612 and applying the workaround mentioned at the bottom.