next up previous contents
Next: Message Service Up: The MINOS Off-line Software Previous: Contents   Contents


Job Control

Last significant change: 2002/06/21

The job control package provides the interface to the data for reconstruction and analysis modules. The package defines the basic unit for reconstruction and analysis, the ``JobCModule'' and provides several standard modules which allow users to organize the various modules into reconstruction/analysis jobs.

JobCModules allow physicist programmers to plug their reconstruction and analysis code into the framework. A JobCModule consists of a header (.h) and an associated (.cxx) file. A module calls algorithms which interface with the data to perform reconstruction. Several JobCModules are provided by the framework which perform basic reconstruction and analysis of MINOS data. These can be organized into reconstruction and analysis jobs using ROOT macros. Instructions for configuring the loon executable using ROOT macros are given in Section 1.1.

In addition to the modules provided by the framework, users can provide their own JobCModules and plug them into loon. Instructions for how to do this are provided in Section 1.2

Running and Configuring loon

The loon executable is configured via a ROOT C++ macro. The basic command is:

$~~>$ loon -[options] macro.C -o out.root in1.root in2.root in3.root ...
where ``macro.C'' is the ROOT macro which configures the job, and ``out.root'' is the default name to use for data output via the output module. The list of input data files are listed as ``in1-in3.root''. The possible command line options are:

-b : Run in batch mode without graphics.
-n : Do not execute ROOT logon and logoff macros.
-q : Quit the job after completing the commands in the macro.
-h : Print usage help summary.
-H[module] : Print help for a specified module.
-d[url]: Specify database URL.
-u[user]: Specify database user.
-p[passwd]: Specify database password.
-t[time]: Set the time limit for the job (eg. -t'10 minutes').
-r[n]: Specify the maximum number of records to evaluate before quitting.
-o[out]: Set the default name to use for output file.

The most common options are ``-bq'' which specify batch running of loon. loon can also be run interactively (just leave off the ``-bq'' option). In this mode the user will be given essentially a ROOT prompt and can execute C++ statements, ROOT macros and commands.

Writing a JobControl Macro

Job control macros provide several functions. The macros are used to 1) organize the analysis and reconstruction modules into ``paths'' which define the sequence the modules are applied to the input data, 2) alter the configuration of jobs modules and framework services, and 3) trigger module commands. Several example job control macros are provided in the Demo package which is a good source to see working examples of macros and JobCModules.

Creating job paths

The basic function of the job control macro is to define the list of reconstruction and analysis steps to perform on the data. The following example macro defines a very simple job path:

  JobC j;
// if MC        "RerootToRawDataModule::Get "
                "DigitListModule::Get "
                "DigitListModule::Reco "
                "UserAnalysis::Ana "
                "CheezyDisplayModule::Ana "
The first step in the macro, JobC j, creates a new job. A new reconstruction/analysis path, named ``Demo'' is created within this job using the C++ command j.Path.Create. The first argument to this command in the name of the path (``Demo'' in this case) which is used to reference this path later in the macro. The second argument is a list of modules (EventDump eg.) and methods (Ana, eg.) to add to the path. The example, shows the typical ``event preparation'' steps (RerootToRawDataModule::Get does the data format conversion for Monte Carlo events, DigitListModule builds the candidate digit lists which hold the hit strip data), as well as some modules provided by the framework to print and display information about events. Finally, this macro runs all the data from the input files through the ``Demo'' path via the command, j.Path("Demo").Run() .

While the ``Create'' command is the most common, the ``Path'' module provides several other commands to configure job paths. The complete list can be viewed in source file JobCPathModule.h which defines the interface to the ``Path'' module; several commands are listed here:

j.Path("Demo") // return the path named ``Demo''
j.Path.Delete("Demo"); // delete the job path named ``Demo''
j.Path.Add("Demo","Module::Method"); // Append to a job path.
j.Path.Attach("A","B"); Link the end of path A to the beginning of path B
j.Path.Report(); // Print a summary of all active job paths.
j.Path.Reset(); // Delete all active paths

Running job paths

Job paths can be run in several ways:

j.Path("Demo").Run(); // Run until all input data has been processed
j.Path("Demo").Run(10); // Run until 10 records have been read in
j.Path("Demo").RunNin(10); // Same as Run(10)
j.Path("Demo").RunNpass(10); // Run until 10 records pass all filters set in the path. Filters are discussed in Section 1.1.1.
j.Path("Demo").RunNfail(10); // Run until 10 records have been failed by the path.

To get a summary of the run, use the Report() command:

This produces output like this:

Demo(10 in 1 out 9 filt.)
  1) +DigitListModule::Get      n=10    (    10/     0) t=(  179.53/    7.13)
  2) +DigitListModule::Reco     n=10    (    10/     0) t=(   56.71/    1.46)
  3) +UserAnalysis::Ana         n=10    (     1/     9) t=(    0.01/    0.00)
  4) +EventDump::Ana            n=1     (     1/     0) t=(    0.01/    0.00)
This report indicates that the path ``Demo'' has processed 10 records, 1 of which passed the analysis, 9 of which were filtered. The path is made of 4 nodes (a Get, a Reco, and two Ana's). The number of event processed by each node is listed. In this example, the node UserAnalysis::Ana saw 10 records, passed 1 of those and filtered 9. The last two columns list the total cpu and system time (in seconds) spent inside the node.


Each time a module performs some reconstruction or analysis on a data record it has the option to pass or fail the record depending on some criteria. For example, a muon track fitter, ``FitTrack::Reco'' might fail events if the goodness of fit is below some criteria. In normal running, data records process down the job paths until the record is failed, at which point the processing stops and a new record in read in.

The handling of filter settings can be configured in several ways. First, the user can opt to turn off all the filters in a path off. In this case processing will continue even for events which fail. All the filters in a path can be switched on and off using a command like:

j.Path("Demo").SetAllFilters(false); // Turn all filters off
j.Path("Demo").SetAllFilters(true); // Turn all filters on

Further, filtering at specific points in the analysis chain can be configured. For example,


Users can also reverse the meaning of the pass/fail decisions made by elements of the analysis chain. For example, if one wanted to output (ie. ``pass'') all events which failed the reconstruction path ``Demo'':

Alternately, the filters can be switched one-by-one, eg.:

Users can view the status of filters set in a given path with the ``Path.Report()'' command:

  1) +DigitListModule::Get      n=10    (    10/     0) t=(  179.53/    7.13)
  2)  DigitListModule::Reco     n=10    (    10/     0) t=(   56.71/    1.46)
  3) !UserAnalysis::Ana         n=10    (     1/     9) t=(    0.01/    0.00)
  4)  EventDump::Ana            n=1     (     9/     0) t=(    0.01/    0.00)
The markers to the left of each node label that the filter is on (``+''), on and reversed (``!''), or off (blank). In this example, the filter for the UserAnalysis::Ana node has been reversed. As you can see from the report, 9 events were failed by this node, and 9 events were processed by the next node in the path. In the case that a node makes no pass/fail decision all records are passed regardless of the on/off/reverse setting.

Job Configuration

There are several things that can be configured from a job control macro. These include the modules which perform the reconstruction as well as various services provided by the loon program.

Module configuration

Each job module carries with it a set of configuration parameters. To see the set of configuration parameters using the ``Report'' command:

loon [0] gSystem->Load("");
loon [1] JobC j;
loon [2] j.Path.Create("A","EventDump::Ana");
loon [3] j.Path("A").Mod("EventDump").Report();
EventDump configured with: Registry: `EventDump.config.default', 13 entries.
     [U|L] DumpAll = 0
     [U|L] DumpCandDigits = 0
     [U|L] DumpFragments = 1
     [U|L] DumpObjectTable = 0
     [U|L] DumpRawDaqSnarlHeader = 1
     [U|L] DumpRawData = 0
     [U|L] DumpRawHeader = 1
     [U|L] First = 0
     [U|L] Freq = 1
     [U|L] Last = 999999999
     [U|L] PrintSeparator = 1
     [U|L] RawBlockList = 
     [U|L] Wait = 0
Here the ``EventDump'' module has been placed into the path ``A''. The ``EventDump'' module is currently configured using a configuration names ``EventDump.config.default'', which has 13 entries with the values shown. Any of these parameters can be changed using a ``Set'' command:
loon [4] j.Path("A").Mod("EventDump").Set("Freq=10");
loon [5] j.Path("A").Mod("EventDump").Report();      
EventDump configured with: Registry: `EventDump.config.default', 13 entries.
     [U|L] DumpAll = 0
     [U|L] DumpCandDigits = 0
     [U|L] DumpFragments = 1
     [U|L] DumpObjectTable = 0
     [U|L] DumpRawDaqSnarlHeader = 1
     [U|L] DumpRawData = 0
     [U|L] DumpRawHeader = 1
     [U|L] First = 0
     [U|L] Freq = 10
     [U|L] Last = 999999999
     [U|L] PrintSeparator = 1
     [U|L] RawBlockList = 
     [U|L] Wait = 0
From the report you should be able to see that the frequency parameter ``Freq'' has been changed from 1 to 10.

Modules support a second interface (``Cmd'' instead of ``Set''). These commands vary from module to module. They should be documented in the help for the module. For example, j.Path("").Mod("Module").Help();.

Framework services

Several of the framework services can be configured. For example, message print thresholds can be altered using the ``Msg'' module. Here are some examples:

j.Msg.SetLevel("Plex","Error"); // Raise the threshold from emitting messages from the Plex package so that only error messages are printed.
j.Msg.SetLevel("MyPackage","Verbose"); // Lower the threshold for ``MyPackage'' so that all debugging messages are printed.
j.Msg.Attach("MyPkg::Error","mypackage.err"); // Attach the file ``mypackage.err'' to the ``MyPkg::Error'' stream.
j.Msg.Format("MyPkg::Debug","File,Line",off); // Remove the file name and line number tag from messages printed to ``MyPkg::Debug'' stream.
j.Msg.Stats(); // Print statistics about message stream use. Useful to see if any warnings or errors were produced during your run.

Job modules provided by the framework

This section reviews some basic modules provided by the framework and how to use them.

Input module

The most important module provided by the framework is the input module. One input module is created when you create a job:

loon [0] JobC j;
loon [1] j.Input.Report();
INPUT configured with: Registry: `INPUT.config', 9 entries.
     [U|L] DDSDataSource = Daq
     [U|L] DDSKeepUpMode = FileKeepUp
     [U|L] DDSMaxSyncDelay = 15
     [U|L] DDSOffLine = 0
     [U|L] DDSPort = 9090
     [U|L] DDSServer =
     [U|L] DDSTimeOut = 120
     [U|L] Format = input
     [U|L] Streams = DaqMonitor,DaqSnarl,LightInjection
As you can see from the report the input module has several configuration parameters. Most of these are for controlling the interface to the DataDispatcher (DDS), which is not the most common use of the input module. Generally, the input module talks to files on a local disk. The two parameters you need to set in that case are ``Format'' which is in general either ``input'' for detector data, ``reroot'' for Monte Carlo. If you want to take data from the DataDispatcher you would j.Input.Set("Format=dds"). The second important parameter is the setting of which data streams you would like to subscribe to. For example, if you only wanted to see ``DaqSnarl'' records you would j.Input.Set("Streams=DaqSnarl");

The input module has the following commands:

j.Input.Next(n) // Advance n records in the stream (default is 1).
j.Input.Prev(n) // Go back n records in the stream (default is 1).
j.Input.GoTo(run,event) // Go to specified run event number.
j.Input.List() // List all files atteched to the input module
j.Input.AddFile("filename") // Add a file to the list
j.Input.RemoveFile("filename") // Remove a file from the list
j.Input.NextFile(n) // Advance n files in the list (default is 1)
j.Input.PrevFile(n) // Go back n files in list (default is 1)
j.Input.GoToFile(n) // Go to the ith file in the list
j.Input.GoToFile("file") // Go to the file ``file'' in the list
j.Input.Select("stream","cut") // Set a selection cut on a stream

When a JobC object is created the Input module automatically loads files from the loon command line.

Other modules

Here I just list some of the other modules that exist in the framework:

How to Make a Job Module

This section will outline how to make a job module and link it into loon.

Automatic generation of source code

To plug into the framework, users must write a new class which will contain the analysis and reconstruction code. The JobControl package provides a simple program which will prompt the user with questions about the module they intend to write, and then automatically generate template code for the h and .cxx which define the user's JobCModule.

To use the generation program, type:
$>$ gen_module

The program will prompt the user for several fields:

Once complete, gen_module will write the .cxx and .h files for the class. These classes will require some tweaking before they compile. For example, the Config() method will need to be hooked up before it will work. Also, comments will need to be added (search for the tag ``FILL_IN''. Oh, and you'll have to fill in your analysis and reconstruction code...

Once the module compiles, you'll need a GNUmakefile if you already have one. For most packages this is simple:

# GNUmakefile for simple package
LIB := lib$(PACKAGE)
LIBCXXFILES := $(wildcard *.cxx)
include SoftRelTools/
include SoftRelTools/
A ``gmake'' should get your module compiled into the package library and the library installed in your test release.

The next step is to attach your module to the loon program. This step is done by updating a file which lives in the SRT test and/or base release which stores the map from job module names to the libraries that provide them. To update this file, use this command:
$>$ makemodulemap
That should be all there is to it. When you ask to add a module to a job path, $\tt\bf loon$ will consult this file ( $SRT_PRIVATE_CONTEXT/tmp/jobmodules.txt), load the required library, and build your module.

Job modules: A closer look

This section takes a closer look at what goes on inside JobCModules using the JobControl/JobCDemoModuleA as an example. The header file for JobCDemoModuleA is shown in Figure 1.1. A sketch of the implementation for this module is shown in Figure [*].

Figure 1.1: Header file for a dummy job module.
\begin{verbatim}000 ///////////////...

User modules must inherit from the base class JobCModule as shown in line 019 in the figure. The user module can optionally provide implementations for one or more of the following methods:


The various methods that look at data records are allowed to make pass/fail decisions on the event by returning values of type JobCResult which can hold several pieces of information about the result of your reconstruction or analysis. The possible results fall into a few categories: The following are a few simple examples for using JobCResult:

JobCResult Module1::Ana(const MomNavigator* mom) {
// Simplest case - no errors, no pass/fail decisions
  return JobCResult::kAOK;
// This would also work:
// JobCResult r;
// return r;


JobCResult Module2::Ana(const MomNavigator* mom) {
// Another simple case: A single pass/fail decision
  if (this->PassesCuts(mom)) return JobCResult::kPassed;
  // else
  return JobCResult::kFailed;


JobCResult Module3::Ana(const MomNavigator* mom) {
// A more complicated case: Apply selection criteria and flag error 
// conditions
  JobCResult r; // The result of this method's analysis of mom

  SomeDataObject* d = this->GetSomeDataObject(mom);

  // Do some error checking on the object d (for example)
  if (d->GetSomeNumber() > this->GetValidRange()) {
    r.SetError();  // Flag this condition as an error
    r.SetFailed(); // Fails selection criteria

  // Apply selection criteria
  if (d->PassesCuts()) {
  else {

  // Return final result, could contain both ``failed'' and ``error''
  return r; 


In general, when users talk to modules, they are just chaning the module's configuration. If you have implemented the Config() method, then you are all set; the framework will trigger the Config() method when the Set() command is issued from a job control macro or from the loon prompt and the module will reconfigure itself based on the new set of parameters.

However, in some cases, you may need more specialized than the Set() command. In this case, you will want to talk to the module using the Cmd() command. This interface allows users to issue a large variety of commands to the module. For example,

loon [0] JobC j;
loon [1] j.Path.Create("A","AModule::Ana");
loon [2] j.Path("A").Mod("AModule").Cmd("Draw timeHisto chargeHisto");
This command might cause histogram windows to pop-up containing time and charge information.

If you find that you need to be able to talk to your module with more flexibility than is provided via Set() you will need to implement the HandleCommand method. This method receives a parsed version of the string command typed by the user as the argument to the Cmd() method. This parsed version is called a JobCommand.

JobCommands are of the form /c1/c2/c3/... o1 o2 o3 ... where c1, c2, c3, ... specify the command path (through menus and sub-menus) and o1, o2, o3... are the options associated with the command. Module can navigate the command path and options using the methods JobCommand::PopCmd(), JobCommand::PopOpt(), JobCommand::PopIntOpt(), JobCommand::PopFloatOpt(). For example,

  UserModule::HandleCommand(JobCommand *cmd) 
    std::string c1 = cmd->PopCmd();
    if (c1 == "Draw") {
      std::string opt = cmd->PopOpt();
      if      (opt == "Top")  this->DrawTopView();
      else if (opt == "Side") this->DrawSideView();
      else { ... print warning... }
    if (c1 == "Dump") {
      std::string c2 = c2 = cmd->PopCmd();
      if (c2 == "Summary") {
        std::string opt = cmd->PopOpt();
        if (opt == "Muon") {...}
        else if (opt == "Electron") {...code...}
      else if (c2 == "Stats") {
handles the commands:
  j.Path("A").Mod("UserModule").Cmd("Draw Top);
  j.Path("A").Mod("UserModule").Cmd("Draw Side");
  j.Path("A").Mod("UserModule").Cmd("Dump/Summary Muon");
  j.Path("A").Mod("UserModule").Cmd("Dump/Summary Electron");


next up previous contents
Next: Message Service Up: The MINOS Off-line Software Previous: Contents   Contents
MINOS Software 2017-11-20