Last modified: Fri Nov 13 06:21:14 GMT 2009
Nick West
Return to home page

Tutorial: Preparing and managing proxies

In the tutorial GRID Overview we introduced 3 key activities: Authorisation, Job Submission and Data retrieval and storage. In this tutorial we will concentrate on Authorisation and is divided into the following sections:-

Introduction
Short Term Proxies
Long Term Proxies
Changing Servers and Roles
Cron job authentication

Introduction

As it explains in the Glossary a proxy accompanies your job and identifies it as belonging to you so that the VO manager can provide authorisation to use the resources that you are entitled to use. You can create two sorts of proxies: short term ones that reside on your UI and last a maximum of 24 hours and long term ones that reside on the Resource Broker and can last up to a week.

The key points about proxies are:-

In order to submit a job from a UI you must have a valid short term proxy there. Initially the jobs you submit take a copy of that proxy with them.
In order to successfully run on the GRID, your job must have a valid proxy until it completes. Here there are two possibilities.
- If you are confident that it end before it's original proxy expires then you need do nothing more. Note though that you cannot extend the lifetime of submitted jobs by refreshing your local short term proxy; it has no effect on the copy associated with the job.
- In all other cases you need to create a long term before you submit the job. Then the Resource Broker allows the job's short term proxy to be refreshed from the long term one as required. In this case you can extend the lifetime of submitted jobs by refreshing your long term proxy; all that is necessary is that there is a valid one when the job's proxy needs to be refreshed.

So, and in summary, begin by creating a short term proxy. Then, unless you are doing short tests that will end before it expires, make sure you have a long term proxy up and running before you submit jobs and if necessary refresh until they all complete.

Short Term Proxies

To create a short-lived (12 hour default, 24 hour maximum) proxy

Login to our User Interface
Type:-
```
  voms-proxy-init  -voms minos.vo.gridpp.ac.uk
```
This contacts the VOMS server and retrieves an Attributes Certificate that is inserted into proxy certificates. If no -voms option is supplied it will create an old style GRID certificate without any VOMS extensions, which won't be any use, so don't forget it.
- By default the proxy will only last 12 hours. You can extend it to a maximum of 24 hours by using:-
```
  -valid 24:0
```
  - do NOT use -hour 24 which extends the basic GRID certificate but not the VOMS extension that it carries as, after the 12 hour limit is passed, you still have a certificate but it won't be recognised by any VOMS aware service!
- If you get the error:-
```
  VOMS Server for minos.vo.gridpp.ac.uk not known!
```
  then the UI is not properly configured for MINOS.
  In this case create the file:-
```
  ~/.glite/vomses
```
  containing:-
```
"minos.vo.gridpp.ac.uk" "voms.gridpp.ac.uk" "15016" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk/Email=ops@tier2.hep.manchester.ac.uk" "minos.vo.gridpp.ac.uk"
```
  and type:-
```
  voms-proxy-init  -voms minos.vo.gridpp.ac.uk  --confile ~/.glite/vomses
```
- To check on an existing proxy:-
```
  voms-proxy-info -all
```
  The -all option includes the VO extension information.
- To destroy one before it naturally expires:-
```
  voms-proxy-destroy
```

Long Term Proxies

To create and store a long-term proxy certificate on a proxy server, allowing proxies to be renewed and extending their effective lifetime beyond 12 hours.

 myproxy-init -d -n
  where
   -d    Use the subject of the certificate as the default username.
   -n    Don't prompt for a credential  passphrase, so that the WMS can perform the renewals automatically.
also
   -s    <proxy-sever hostname>

 Currently at Oxford on pplxint1 must first do: 

      setenv MYPROXY_SERVER lcgrbp01.gridpp.rl.ac.uk

By default, the long-term proxy lasts for one week and the proxy certificates created from it last 12 hours. These lifetimes can be changed using the -c and the -t option, respectively. If the hostname of the Proxy Server is wrong (environment MYPROXY_SERVER or -s option), or the service is unavailable, the output will end e.g.:-

Your proxy is valid until: Wed Sep 17 12:10:22 2003 
Unable to connect to adc0014.cern.ch:7512

or will fail entirely with

Unspecified myproxy-server. Please set the MYPROXY_SERVER environment variable

Our server is

lcgrbp01.gridpp.rl.ac.uk

To get information about a long-term proxy:-

  myproxy-info -d

To destroy a long-term proxy before it expires:-

  myproxy-destroy -d

This information comes from @ LCG-2 User Guide/4.4 Proxy Certificates

Changing Servers and Roles

Unless you know that you will need to change your default VOMS server or role within the VO, you can skip the rest of this tutorial.

By default voms-proxy-init will use the standard VOMS server defined for the experiment at the UI . You can select an alternative using the --voms option, for example

  voms-proxy-init --voms minos.vo.gridpp.ac.uk

Here the "minos.vo.gridpp.ac.uk" is just an alias that is used to look up the server details in a configuration file. This file can specified in several alternative ways:-

On the command line using --confile
For example

  voms-proxy-init --voms minos.vo.gridpp.ac.uk --confile ~/.glite/vomses

Using the environmental variable VOMS_USERCONF
For example
```
  setenv VOMS_USERCONF ~/.glite
```
Default
By default, the system checks the directory:-
```
  /opt/glite/etc/vomses
  ~/.glite
```

--confile and VOMS_USERCONF can either specify a directory holding configuration files or an individual configuration file. Each row of a configuration file contains:-

nick - the VO alias or nickname
host - the server host name
port - the server port
subject - the subject of the server's certificate
vo - the name of the VO that owns the server

When using the --voms option it is possible to specify a command to be sent to the server using the syntax:-

   --voms  vo-alias[:vo-command]

The default command is :all, and it gets all group membership information. However it is also possible to send other commands and in particular to select a role.

For example:-

  voms-proxy-init --voms minos.vo.gridpp.ac.uk:/Role=lcgadmin

will select the lcgadmin role.

Tutorial: Submitting jobs

In the tutorial GRID Overview we introduced 3 key activities: Authorisation, Job Submission and Data retrieval and storage. In this tutorial we will concentrate on Job Submission.

Before proceeding, make sure you have prepared a short term GRID proxy

Using glite-wms-job-* commands

We will start by running a "hello world" job using the new (as of Dec 2007) GRID gLite/WMS middleware. That's short for gLite Workload Management System. A key component of this system is the WMProxy which is responsible for accepting incoming requests from the User Interface (e.g. job submission, job removal). Before you can submit a job, you need to create a GRID proxy. Then you must it to the WMProxy. This process is called delegation and there are two alternatives way to do it:-

Explicit Delegation
Delegate once and then, as separate steps, perform operations, such as job submission, passing in the delegation identifier using the -d option.
Automatic Delegation
Delegate as part of each operation. This is called auto-delegation and is signalled using the -a option.

As delegation is quite time consuming the first method is more efficient if performing multiple operations, however as we will only be running one or two so we will use the simpler, but slower, method of delegating automatically, as part of the job submission command. To run your first job, follow the following steps.

Make sure that your UI is properly configured for gLite/WMS middleware.
Next make sure you have Set up a Short Term Proxt
In some scratch create a file called hello.jdl with this content:-
```
Executable = "/bin/echo";
Arguments = "Hello World";
StdOutput = "hw.out";
StdError = "hw.err";
OutputSandbox = {"hw.out", "hw.err"};
VirtualOrganisation = "minos.vo.gridpp.ac.uk";
```
For each job you need a "JDL file" (JDL is the Job Description Language) and in this case that's all you need.
As you might expect this just echoes the string "Hello World" to stdout. The stdout and stderr streams are redirected to files called hw.out and hw.err, and at the end of the job those files will be returned in the "output sandbox", a list of files which can be retrieved once the job is finished.

Check that everything is working with

  glite-wms-job-list-match -a hello.jdl

The -a is the option to request auto delegation; each time you use it you are delegating your proxy for a one shot use by the WMProxy.

If all is O.K., you should see something like this:-

Connecting to the service https://lcgwms01.gridpp.rl.ac.uk:7443/glite_wms_wmproxy_server

==========================================================================

                     COMPUTING ELEMENT IDs LIST
 The following CE(s) matching your job requirements have been found:

        *CEId*
...
 - lcgce02.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS
 - t2ce03.physics.ox.ac.uk:2119/jobmanager-lcgpbs-minos
 - t2ce03.physics.ox.ac.uk:2119/jobmanager-lcgpbs-short
 - t2ce02.physics.ox.ac.uk:2119/jobmanager-lcgpbs-minos
 - t2ce02.physics.ox.ac.uk:2119/jobmanager-lcgpbs-short

==========================================================================

Then submit the job:
```
  glite-wms-job-submit -a -o /tmp/hello.jid hello.jdl
```
here hello.jid is a temporary file containing the job ID - if you submit further jobs using the same file the IDs will be appended.
You can check on the status of the job with
```
  glite-wms-job-status -i /tmp/hello.jid
```
In particular look for the Current Status: and Status Reason: lines e.g.:-
```
  Current Status:     Scheduled
  Status Reason:      Job successfully submitted to Globus
```
Note that you don't need -a this time; you are only requesting services of the WMProxy, not of the wider GRID.
Due to various overheads in the system even a short job like this will take a few minutes, and potentially a lot longer if there are no free resources and the job has to be queued, but if all goes well the status should eventually say "Done (Success)". At that point you can retrieve the job output (the stdout and stderr files in this case) with
```
  glite-wms-job-output -i /tmp/hello.jid
```
The output is stored in a directory which is configured locally, often in /tmp, or you can supply your own with the --dir option. In any case the above command will print the location. For eample I see:-
```
                        JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://lcgwms01.gridpp.rl.ac.uk:9000/fFNhqhtvZ0Tv9tZ25sTQJg
have been successfully retrieved and stored in the directory:
/tmp/glite/glite-ui/west_fFNhqhtvZ0Tv9tZ25sTQJg
```
You can then examine the files to check that the output is correct.
The job above just executes a pre-installed command (/bin/echo), but in most cases you will want to ship a script and possibly other files with the job. These are sent in the so-called input sandbox. As a simple example, create a file called hello2.txt containing the string "Hello World". Then create a file hello2.sh:
```
#!/bin/bash
cat $1
```
Finally, create a new JDL file hello2.jdl:
```
Executable = "hello2.sh";
Arguments = "hello2.txt";
InputSandbox = {"hello2.sh", "hello2.txt"};
StdOutput = "hw.out";
StdError = "hw.err";
OutputSandbox = {"hw.out", "hw.err"};
VirtualOrganisation = "minos.vo.gridpp.ac.uk";
```
and submit it as described above. If all goes well this should send the files hello2.txt and .sh with the job, set hello2.sh to be executable, and run the command "hello2.sh hello2.txt" which will again print the string "Hello World" to stdout.
One thing to note is that permissions on the sandbox files are not preserved. A file named in the Executable field in the JDL will have the x bit set, but any other files will have it cleared.
You should also be aware that sandboxes are for small files, up to a few kb - resource brokers will generally limit the maximum size. Larger files should be accessed via the data management system.
The examples above have jobs which take a very short time to run. However, for real jobs you need to take into account that batch queues have time limits. This can be managed by adding a Requirement to the JDL, which specifies constraints on the site and queue used to run the job. This can be quite complex and you should consult the User Guide for full details. However, as a simple example you can specify a minimum CPU time of an hour with a JDL line like:
```
  Requirements = other.GlueCEPolicyMaxCPUTime > 60;
```
Note that the time limits are in minutes, although the Estimated/WorstResponseTime values are in seconds.

This section has been drawn from @ Submitting Jobs to the Grid and @ Submission and monitoring of jobs via WMProxy using the command line interface

Using Ganga

To start learning about Ganga, read the Introduction and Running Ganga sections of GridTools: Ganga

Using globus-job-run

It is also possible to submit very short jobs directly to a CE using the globus-job-run command. For example:-

  globus-job-run lcgce02.gridpp.rl.ac.uk /bin/pwd

and

  globus-job-run lcgce02.gridpp.rl.ac.uk  /usr/bin/whoami

This command is extremely useful for checking things like the environment and setup up of a CE.

For more information, type

  globus-job-run -help

Cron job authentication

Using a myproxy server as a source of GRID proxies

Cron jobs, for example running GBS, present a difficulty: short term proxies are just that. If you create one it has gone within a day so the question is: How can one be created automatically by some unsupervised process?

If you look at the options available to voms-proxy-init you will find:-

  -pwstdin Allows passphrase from stdin

which could encourage you to do something like:-

  voms-proxy-ini -pwstdin < <my-GRID-certificate-password-file>

DO NOT DO THIS!! Even if you protect this file the risk is too high. Your GRID password decrypts the encrypted certificate. Without the password, if the certificate is stolen, it is useless but with that file a hacker has full control and the cost to you will be high!

Instead the answer is to lodge a long term proxy with the myproxy server and retrieve a short term one from there. Note that this is a separate proxy to the one you may have set up to keep GRID jobs running. In that case the command is:-

  myproxy-init -d -n

which creates a proxy with the GRID certificate's username and without a password. In this case the command is

  myproxy-init -a -l <user-name>
  where
    -a Allow proxies to be retrieved with just pass phrase
    -l <user-name>  the user name to be assigned (default: $LOGNAME)

Remember: to run myproxy-init you must have a short term proxy. If you run this it will first ask for your GRID certificate password but it will then ask for, and reconfirm, a password that will give you access to short term proxies created from this proxy.

With that preparation we can now turn to the cron job that has not got a short term proxy. It can use the command myproxy-get-delegation to get a GRID certificate, without a VOMS extension. The problem is that it needs the password you assigned to the proxy. This time, the solution is to store the password in a file and then the command is:-

  myproxy-get-delegation -l <user-name> --stdin_pass  < <my-myproxy-password-file>

Note that this is a far lower risk. If this file is stolen and read they can only obtain short-term proxies and if the theft is discovered your proxy can be deleted from the myproxy, and your VOMS membership can be suspended. That's not to say that the password file should not be highly protected and it's worth changing frequently (have a script that creates a new password file each time you store a new certificate in the server?).

Having run myproxy-get-delegation you have a certificate but without a VOMS extension and you can add this with:-

  voms-proxy-init  -voms minos.vo.gridpp.ac.uk --confile ~/.glite/vomses  -noregen

This adds a VOMS extension to the certificate. When you run that command you will probably see the warning:-

  Warning: your certificate and proxy will expire ...
  which is within the requested lifetime of the proxy

because the certificate will be a little older than the VOMS addition. It's harmless or if it annoys you could make the VOMS extension shorter using

  -valid 11:0

voms-proxy-init option to trim them both.

If you want to try this out then, having run myproxy-init to lodge your proxy type:-

  voms-proxy-destroy

to delete it and then, having run myproxy-get-delegation and voms-proxy-init see that it is there with:-

 voms-proxy-info -all

Remember long term proxies only last a week so you still have to renew it from time to time so that your cron job won't be starved of a source of proxies.

For further information see @ GridPP: Using MyProxy

$MOG_SCRIPTS/setup/grid_proxy_maintainer.py

A small Grid Script called grid_proxy_maintainer.py has been written to simplify this process. For example to set up the proxy, type:-

  $MOG_SCRIPTS/setup/grid_proxy_maintainer.py set

then, in any GRID cron script that needs a proxy add the line e.g.:-

  $MOG_SCRIPTS/setup/grid_proxy_maintainer.py get --voms  minos.vo.gridpp.ac.uk

For more detail just type:-

  $MOG_SCRIPTS/setup/grid_proxy_maintainer.py

Return to home page