[g13] cooking g13 pass2

Thu Aug 19 14:41:49 EDT 2010

The skimming and trip files looks like they're working properly.  I  
will try to clean up more space on the work disk, although I seem to  
remember data not lasting very long on the disk last time anyway.

It may be possible to reduce the file size of the skims if there are  
banks in them that we don't need.  The banks currently being written  
to the skims are:
DCPB, ECHB, ECPB, EVNT, HEAD, HBID, HEVT, MVRT, PART, RC26, SCPB,  
SCRC, STPB, TAGR, TBER, TBID, TBTR, TDPL, TGPB, VERT

Do we really want the TDPL bank to be in the skimmed output?  If we  
want it that's fine, but if not we can definitely save some space.   
I'll only drop it from skims if people make it absolutely clear that  
we don't need it.

Finally, it may be a good idea to add the trip and gflux files to the  
tar file, but it's not necessary.

Otherwise I think we're good to go.  However, there will be a power  
outage on Saturday, so maybe we may want to wait until Monday to  
start?  I'm not sure it makes a difference though.

  - Paul

On Aug 19, 2010, at 11:04 AM, Ken Livingston wrote:

Dear All,
This describes the output of pass2 cooking. Don't ask about the cooking
unless you've checked this mail 1st or visited
http://clasweb.jlab.org/rungroups/g13/wiki/index.php/Pass2_cooking

I believe we're ready to go now with the cooking. Please read below and
take this last chance to check the outputs from the cooking of
/mss/clas/g13/data/clas_054392.A04. The "expansion factor" ie. ratio of
new files on silo after cooking to raw data is 3.65.

I intend to start pass2 properly tomorrow. Please let me know if you
have any different opinion. I'll be submitting all the A01 files from
54010 55147 for starters, and then the remaining AXX files in that  
range.

After endless discussions on the topic I only removed the banks that
Franz suggested. The complete BANK list in alphabetical order is:

"CL01" "DC0 " "DCPB" "DSTC" "EC  " "EC1 " "ECHB" "ECPB" "ECPC" "ECPI"
"ECPO"
"ECT " "EPIC" "EVNT" "GP_X" "GP_Y" "HBID" "HBTR" "HDPL" "HEAD" "HEVT"
"MTRK"
"MVRT" "PART" "RC26" "RFT " "SC  " "SC1 " "SCPB" "SCRC" "SCT " "ST1 "
"STN1"
"STPB" "STRE" "TACO" "TAGE" "TAGI" "TAGR" "TBER" "TBID" "TBTR" "TCT "
"TDPL"
"TESC" "TGBI" "TGPB" "TRL1" "TRPB" "VERT"

The outputs are going to 2 places - the silo and to our clasg13 area on
the work disks.

Here are the outputs on the silo:
/mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B00
/mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B01
/mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B02
/mss/clas/g13/production/pass2/v0/PAW/nt_054392_04.hbook
/mss/clas/g13/production/pass2/v0/TAR/skims_054392_A04.tar

The content of the tar file
/mss/clas/g13/production/pass2/v0/TAR/skims_054392_A04.tar is:
ifarml6.kliv> tar tvf
/w/cache5202/clas/mss/clas/g13/production/pass2/v0/TAR/ 
skims_054392_A04.tar
drwxrwxr-x clasg13/clas      0 2010-08-19 07:55:54 1pos1neg1neut/
-rw-r--r-- clasg13/clas   7821 2010-08-19 07:55:54 1pos1neg1neut/ 
hists.root
-rw-rw-r-- clasg13/clas     43 2010-08-19 07:55:54
1pos1neg1neut/filteroutputs.txt
-rw-rw-r-- clasg13/clas 415688400 2010-08-19 07:55:54
1pos1neg1neut/1pos1neg1neut_054392.A04.B00
drwxrwxr-x clasg13/clas         0 2010-08-19 08:00:24 2pos1neg/
-rw-r--r-- clasg13/clas      7598 2010-08-19 08:00:24 2pos1neg/ 
hists.root
-rw-rw-r-- clasg13/clas        33 2010-08-19 08:00:24
2pos1neg/filteroutputs.txt
-rw-rw-r-- clasg13/clas 325746000 2010-08-19 08:00:24
2pos1neg/2pos1neg_054392.A04.B00
drwxrwxr-x clasg13/clas         0 2010-08-19 08:04:47 2pos2neg/
-rw-r--r-- clasg13/clas      7596 2010-08-19 08:04:46 2pos2neg/ 
hists.root
-rw-rw-r-- clasg13/clas        33 2010-08-19 08:04:47
2pos2neg/filteroutputs.txt
-rw-rw-r-- clasg13/clas  54190800 2010-08-19 08:04:46
2pos2neg/2pos2neg_054392.A04.B00
-rw-r--r-- clasg13/clas   4138214 2010-08-19 08:12:10
strange_k0fix_DST_054392.A04.root
-rw-r--r-- clasg13/clas   1226211 2010-08-19 08:11:09
strange_kplus_DST_054392.A04.root
-rw-r--r-- clasg13/clas 181280488 2010-08-19 08:13:38  
POL_054392_A04.root
-rw-rw-r-- clasg13/clas     11516 2010-08-19 07:48:38 run_054392_A04.log
-rw-r--r-- clasg13/clas  11534336 2010-08-19 07:48:38
anahist_054392_A04.hbook
-rw-rw-r-- clasg13/clas       419 2010-08-19 08:14:21
slavamon_054392_A04.dat
-rw-rw-r-- clasg13/clas     43639 2010-08-19 08:14:21
slavamon_054392_A04_pp.dat
-rw-r--r-- clasg13/clas      3913 2010-08-19 08:14:21
slavamon_054392_A04.root
-rw-rw-r-- clasg13/clas      5717 2010-08-19 08:16:00 job.err
-rw-rw-r-- clasg13/clas     64109 2010-08-19 08:16:00 job.out

... and on the work disk

/work/clas/clasg13/pass2/v0/AnaHistFiles/
11524608 anahist_054392_A04.hbook

/work/clas/clasg13/pass2/v0/dariamon
396800 dariamon_054392_A04.root

/work/clas/clasg13/pass2/v0/GFLUX
396800 gflux054392.a04.hbook.gz
  20992 gflux54392_eb.a04.dat
   2560 gflux54392_erg2.a04.dat
   4096 gflux54392_erg.a04.dat
   3072 gflux54392_tc.a04.dat

/work/clas/clasg13/pass2/v0/LOGS
  7168 run_054392_A04.err
12800 run_054392_A04.log
65536 run_054392_A04.out

/work/clas/clasg13/pass2/v0/skims
4193280 strange_k0fix_DST_054392.A04.root
1312768 strange_kplus_DST_054392.A04.root

/work/clas/clasg13/pass2/v0/skims/1pos1neg1neut
415415808 1pos1neg1neut_054392.A04.B00

/work/clas/clasg13/pass2/v0/skims/2pos1neg
325581824 2pos1neg_054392.A04.B00

/work/clas/clasg13/pass2/v0/2pos2neg
54221824 2pos2neg_054392.A04.B00

/work/clas/clasg13/pass2/v0/slavamon
  1536 slavamon_054392_A04.dat
  5120 slavamon_054392_A04.root

/work/clas/clasg13/pass2/v0/TRIP
3072 clas_054392.A04.trip

And, based on the cooking of /mss/clas/g13/data/clas_054392.A04
the total per cook job going onto the clasg13 work disk is ~813M.
Most of that comes from Paul's 2 bigger skims which total ~740M

There's 1.5T availble on /work/clas/clasg13. That's about enough for the
skimmed output of about 30 runs, which is about isn't much! Paul, do you
have a plan for these?

The "expansion factor" ie ratio of new files on silo after cooking to
raw data is 3.65, based on the cooking of /mss/clas/g13/data/ 
clas_054392.A04

Cheers,
Ken

-- 
=======================================================
Ken Livingston

Dept. of Physics&  Astronomy,        Tel: +44 141 330 6428
University of Glasgow,               Fax: +44 141 330 5889
Glasgow G12 8QQ.
Scotland. UK.
=======================================================

_______________________________________________
g13 mailing list
g13 at jlab.org
https://mailman.jlab.org/mailman/listinfo/g13