Tutorial: Interrogating the GRID

• The LCG Grid Operations Centre This provides top-level information about the GRID nodes, including their configuration, contacts, status and scheduled downtimes. For example to check on the status of any GRID site, and particularly RAL-LCG2, query the GOCDB2 (Grid Operations Centre Database version 2)

  @ https://goc.grid-support.ac.uk/gridsite/db/

  For more information see Site Status

• Information Services. The Information Service provides information about the LCG-2 Grid resources and their status. It is used to locate both Computing Elements on which to run jobs and Storage Elements that hold replicas of date files and the catalogues. It is also used to monitor the performance of the grid and to provide accounting information about resources consumed by the VOs. Its data conforms to GLUE Schema.

• Monitoring and Discovery Service (MDS). It uses OpenLDAP (Open source Lightweight Directory Access Protocol). which is a specialized database optimized for reading, browsing and searching information. Anonymous read access is allowed so that all users can query. The LDAP model is based on a Directory Information Tree (DIT). Each node in the tree has a unique Distinguished Name (DN) that consists of a series of attributes each composed of a type and one or more values. Each node is an instance of an object class, which are collections of mandatory and optional attribute names and value types.

  Information is collected hierarchically. At each site individual computing and storage elements collect information into a Grid Resource Information Server (GRIS). At each site one element acts as is the Site Grid Index Information Server (GIIS) or site Berkeley DB Information Index (BDII) and collects information from the individual GRISes and propagates them to a top-level database which is also a BDII.

  Interrogation of this information can likewise be heirachical. Normally it is sufficient just to interrogate the top level BDII. However more detailed, and due to latencies in the system, more up to date, information can be obtained by querying directly the site GIISes (or site BDIIs) or even the local GRISes.

  Commands such as lcg-infosite, lcg-info and ldapsearch use MDS for example:-

  • lcg-infosite

      lcg-infosites --vo minos.vo.gridpp.ac.uk all
    

    will list all the CEs and SEs MINOS can use.

    For more information see Computing and Storage Resources MINOS can use (lcg-infosites)

  • lcg-info

      lcg-info --list-ce  --query 'CE=heplnx206.pp.rl.ac.uk*' --attrs TotalJobs,WaitingJobs,RunningJobs
    

    lists, for the RAL Tier 2 CE (heplnx206) the total, waiting and running jobs for every VO allowed to run there..

    For more information see Computing and Storage Attribute Queries (lcg-info)

  • ldapsearch

    ldapsearch -x -h site-bdii.gridpp.rl.ac.uk -p 2170 -b 'Mds-Vo-name=RAL-LCG2,o=Grid' '(GlueSALocalID=minos)' \
                GlueSARoot GlueSAStateAvailableSpace GlueSAStateUsedSpace 
    

    returns the available and used storage space on the RAL Tier 1 SE.

    For more information see Raw LDAP Queries (ldapsearch)

• Relational Grid Monitoring Architecture (R-GMA) is an implementation of the Grid Monitoring Architecture (GMA) In R-GMA, the information is presented as being in a global distributed relational database. This model is more powerful than the LDAP-based, since relational databases support more advanced query operations than LDAP does, and also support schema modifications in an easier way. The architecture consists of three major components:-
  • Producers: that register themselves with the Registry and supply information. They can either be Primary or Secondary. Typically Primary sources publish information as it becomes available while Secondary sources collect information from Primaries and record it in a database so as to be able to provide histories.
  • Consumers: that contact the Registry and hence locate Producers to which they submit queries
  • Registry: holds the schema which enables Consumers to locate Producers.
  To a user, the system appears to be a database populated with virtual tables supporting a restricted SQL. The user makes a query and behind the interface a Consumer locates the information from the Registry and then retrieves it from a Producer. SQL runs deeper than just the user API: Producers announce what they have to publish via an SQL CREATE TABLE statement and publish with an SQL INSERT and Consumers use an SQL SELECT to collect the information they need.

  R-GMA can be accessed using the command rgma.

  R-GMA will eventually replace MDS, but currently both systems are in use.

  For more information see R-GMA Queries (rgma)