MINOS Batch Group

Script Output Information

This page presents a summary of information concerning the output methods of the production scripts and to a lesser extent job control in general. For information about Concatenated Files click here.

Types of Output Files

During standard production there are three main Output streams classes of output files: Candidate files, Standard Ntuple Files, and Short Standard Ntuple Files.

Candidate files contain all of the candidate reconstruction objects and are largest files produced.
Standard Ntuples have much of the information reduced into an ntuple form for physics analysis groups to perform later analysis. These files tend to be smaller than the Candidate files by about an order of magnitude.
Short Standard Ntuples contain the same information as the Standard ntuples, but the strip information has been removed. This can save close to another factor of five in space as well. These files are no longer produced by default (Sept. 2006).

Each file type can have multiple streams attached to it, candidate files will frequently also contain DaqSnarl or SimSnarl (MC) information as well. All data production files also have the Beam Monitoring Ntuples attached which stores information concerning beam data.


Configuring the Output Module

Defining an output module is roughly equivalent to defining an output file. After declaring the module it is necessary to define what it is that will be written to that file. By default an unconfigured Output Module will attempt to create a candidate file named out.root. Configuration for an Output module falls in three major steps:
1. Declaring the Output Streams
2. Setting the Streams to Output
3. Setting the Output filename

Here is a section of sample code:
jcm is a generic job control output module for example
jcm = jc.Path("Reco1").Mod("Output")

jcm.Cmd("DefineStream NtpSt NtpStRecord");
jcm.Cmd("DefineStream NtpBDLite NtpBDLiteRecord");
// These two lines define the Standard Ntuple and Beam Monitoring Ntuple for output
jcm.Set("Streams=NtpBDLite,NtpSt");
//This line actually Sets the output module to output the two ntuple streams to the file
jcm.Set("FileName=ntupleS.root");
//Finally the filename for output is set

Starting in R1-18 Simple functions that contain these instructioned are in the production script files to streamline setting the output options


How to produce snts files

At the September 2006 collaboration meeting it was decided to no longer write out the snts (short ntuple) files. For future reference the code is included here


// Ntuple abridged record
jc.Path.Create("NtpSRFilter",
"NtpStFilterModule::Reco "
"Output::Put "
);
JobCModuile jcm = jc.Path("NtpSRFilter").Mod("Output");
jcm.Cmd("DefineStream NtpSt NtpStRecord");
jcm.Set("Streams=NtpSt");
jcm.Set("FileName=ntupleSt.sub.root");
jc.Path.Attach("Reco1", "NtpSRFilter");

What are the standard output files?
The data scripts produce files named in the following format:
CandX.root, ntupleStX.root, ntupleStX.sub.root.
The X will be S for spill or beam MC scripts and A for All or Cosmic scripts. CandX.root is the standard Candidate file, ntupleStX.root the full Standard Ntuple (NtpSt) file, and ntupleStX.sub.root the trimmed short ntuple.

Near Detectors Files
Near Spill Data scripts produce just the three standard output files. After production they will be renamed to the standard naming scheme as follows:
CandS.root -> N(RUN)_(SUB_RUN).spill.cand.(release).root
ntupleStS.root -> N(RUN)_(SUB_RUN).spill.sntp.(release).root
ntupleStS.sub.root -> N(RUN)_(SUB_RUN).spill.snts.(release).root

Near Beam MC scripts produce just the three standard output files. After production they will be renamed to the standard naming scheme as follows:
CandS.root -> n(RUN)_(SUB_RUN)_L(beam).cand.(release).root
ntupleStS.root -> n(RUN)_(SUB_RUN)_L(beam).sntp.(release).root
ntupleStS.sub.root -> n(RUN)_(SUB_RUN)_L(beam).snts.(release).root

Near Cosmic Data scripts produce just the three standard output files. After production they will be renamed to the standard naming scheme as follows:
CandA.root -> N(RUN)_(SUB_RUN).cosmic.cand.(release).root
ntupleStA.root -> N(RUN)_(SUB_RUN).cosmic.sntp.(release).root
ntupleStA.sub.root -> N(RUN)_(SUB_RUN).cosmic.snts.(release).root

Far Detector Files
Far Spill Data scripts produce six output files, the three standard output files once for the full data set and then again for a blinded data set. After production they will be renamed to the standard naming scheme as follows:
* Closed (Full) Data:
CandSBlind.root -> F(RUN)_(SUB_RUN).spill.bcnd.(release).root
ntupleStSBlind.root -> F(RUN)_(SUB_RUN).spill.bntp.(release).root
ntupleStSBlind.sub.root -> F(RUN)_(SUB_RUN).spill.bnts.(release).root
* Open Data:
CandS.root -> F(RUN)_(SUB_RUN).spill.cand.(release).root
ntupleStS.root -> F(RUN)_(SUB_RUN).spill.sntp.(release).root
ntupleStS.sub.root -> F(RUN)_(SUB_RUN).spill.snts.(release).root

Far MC Beam Data scripts produce just the three standard output files. After production they will be renamed to the standard naming scheme as follows:
CandS.root -> f(RUN)_(SUB_RUN)_L(beam).cand.(release).root
ntupleStS.root -> f(RUN)_(SUB_RUN)_L(beam).sntp.(release).root
ntupleStS.sub.root -> f(RUN)_(SUB_RUN)_L(beam).snts.(release).root

Far All Data scripts produce just the three standard output files. After production they will be renamed to the standard naming scheme as follows:
CandA.root -> N(RUN)_(SUB_RUN).all.cand.(release).root
ntupleStA.root -> N(RUN)_(SUB_RUN).all.sntp.(release).root
ntupleStA.sub.root -> N(RUN)_(SUB_RUN).all.snts.(release).root

A more detailed explanation of the MC file naming scheme may be found here.

Concatenated Files
Starting in Cedar process an effort to concatenate the standard ntuple files is being employed. This will hopefully reduce the total number of files being moved around in dcache and the total number of files that people need to locate to access. More information will be reported as it becomes available.

The concatenation takes place by making use of the hadd utility in root and is currently under the direction of Art Kreymer and Howard Rubin.