[g13] cooking g13 pass2

Ken Livingston k.livingston at physics.gla.ac.uk
Fri Aug 20 04:24:48 EDT 2010 

Hi Paul,
I intended the gflux and trip stuff to be in the tar file. It's there 
now. Thanks for pointing that out.

If you decide to remake the skims TDPL removed that's fine. It would 
allow a bit more on the work disk. Best to take silence as assent. If 
nobody demands that we keep it by the end of the weekend we should ditch it.

Cheers,
Ken


On 08/19/2010 07:41 PM, Paul Mattione wrote:
> The skimming and trip files looks like they're working properly.  I
> will try to clean up more space on the work disk, although I seem to
> remember data not lasting very long on the disk last time anyway.
>
> It may be possible to reduce the file size of the skims if there are
> banks in them that we don't need.  The banks currently being written
> to the skims are:
> DCPB, ECHB, ECPB, EVNT, HEAD, HBID, HEVT, MVRT, PART, RC26, SCPB,
> SCRC, STPB, TAGR, TBER, TBID, TBTR, TDPL, TGPB, VERT
>
> Do we really want the TDPL bank to be in the skimmed output?  If we
> want it that's fine, but if not we can definitely save some space.
> I'll only drop it from skims if people make it absolutely clear that
> we don't need it.
>
> Finally, it may be a good idea to add the trip and gflux files to the
> tar file, but it's not necessary.
>
> Otherwise I think we're good to go.  However, there will be a power
> outage on Saturday, so maybe we may want to wait until Monday to
> start?  I'm not sure it makes a difference though.
>
>    - Paul
>
>
> On Aug 19, 2010, at 11:04 AM, Ken Livingston wrote:
>
> Dear All,
> This describes the output of pass2 cooking. Don't ask about the cooking
> unless you've checked this mail 1st or visited
> http://clasweb.jlab.org/rungroups/g13/wiki/index.php/Pass2_cooking
>
> I believe we're ready to go now with the cooking. Please read below and
> take this last chance to check the outputs from the cooking of
> /mss/clas/g13/data/clas_054392.A04. The "expansion factor" ie. ratio of
> new files on silo after cooking to raw data is 3.65.
>
> I intend to start pass2 properly tomorrow. Please let me know if you
> have any different opinion. I'll be submitting all the A01 files from
> 54010 55147 for starters, and then the remaining AXX files in that
> range.
>
> After endless discussions on the topic I only removed the banks that
> Franz suggested. The complete BANK list in alphabetical order is:
>
> "CL01" "DC0 " "DCPB" "DSTC" "EC  " "EC1 " "ECHB" "ECPB" "ECPC" "ECPI"
> "ECPO"
> "ECT " "EPIC" "EVNT" "GP_X" "GP_Y" "HBID" "HBTR" "HDPL" "HEAD" "HEVT"
> "MTRK"
> "MVRT" "PART" "RC26" "RFT " "SC  " "SC1 " "SCPB" "SCRC" "SCT " "ST1"
> "STN1"
> "STPB" "STRE" "TACO" "TAGE" "TAGI" "TAGR" "TBER" "TBID" "TBTR" "TCT"
> "TDPL"
> "TESC" "TGBI" "TGPB" "TRL1" "TRPB" "VERT"
>
>
> The outputs are going to 2 places - the silo and to our clasg13 area on
> the work disks.
>
> Here are the outputs on the silo:
> /mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B00
> /mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B01
> /mss/clas/g13/production/pass2/v0/BOS/cooked_054392.A04.B02
> /mss/clas/g13/production/pass2/v0/PAW/nt_054392_04.hbook
> /mss/clas/g13/production/pass2/v0/TAR/skims_054392_A04.tar
>
> The content of the tar file
> /mss/clas/g13/production/pass2/v0/TAR/skims_054392_A04.tar is:
> ifarml6.kliv>  tar tvf
> /w/cache5202/clas/mss/clas/g13/production/pass2/v0/TAR/
> skims_054392_A04.tar
> drwxrwxr-x clasg13/clas      0 2010-08-19 07:55:54 1pos1neg1neut/
> -rw-r--r-- clasg13/clas   7821 2010-08-19 07:55:54 1pos1neg1neut/
> hists.root
> -rw-rw-r-- clasg13/clas     43 2010-08-19 07:55:54
> 1pos1neg1neut/filteroutputs.txt
> -rw-rw-r-- clasg13/clas 415688400 2010-08-19 07:55:54
> 1pos1neg1neut/1pos1neg1neut_054392.A04.B00
> drwxrwxr-x clasg13/clas         0 2010-08-19 08:00:24 2pos1neg/
> -rw-r--r-- clasg13/clas      7598 2010-08-19 08:00:24 2pos1neg/
> hists.root
> -rw-rw-r-- clasg13/clas        33 2010-08-19 08:00:24
> 2pos1neg/filteroutputs.txt
> -rw-rw-r-- clasg13/clas 325746000 2010-08-19 08:00:24
> 2pos1neg/2pos1neg_054392.A04.B00
> drwxrwxr-x clasg13/clas         0 2010-08-19 08:04:47 2pos2neg/
> -rw-r--r-- clasg13/clas      7596 2010-08-19 08:04:46 2pos2neg/
> hists.root
> -rw-rw-r-- clasg13/clas        33 2010-08-19 08:04:47
> 2pos2neg/filteroutputs.txt
> -rw-rw-r-- clasg13/clas  54190800 2010-08-19 08:04:46
> 2pos2neg/2pos2neg_054392.A04.B00
> -rw-r--r-- clasg13/clas   4138214 2010-08-19 08:12:10
> strange_k0fix_DST_054392.A04.root
> -rw-r--r-- clasg13/clas   1226211 2010-08-19 08:11:09
> strange_kplus_DST_054392.A04.root
> -rw-r--r-- clasg13/clas 181280488 2010-08-19 08:13:38
> POL_054392_A04.root
> -rw-rw-r-- clasg13/clas     11516 2010-08-19 07:48:38 run_054392_A04.log
> -rw-r--r-- clasg13/clas  11534336 2010-08-19 07:48:38
> anahist_054392_A04.hbook
> -rw-rw-r-- clasg13/clas       419 2010-08-19 08:14:21
> slavamon_054392_A04.dat
> -rw-rw-r-- clasg13/clas     43639 2010-08-19 08:14:21
> slavamon_054392_A04_pp.dat
> -rw-r--r-- clasg13/clas      3913 2010-08-19 08:14:21
> slavamon_054392_A04.root
> -rw-rw-r-- clasg13/clas      5717 2010-08-19 08:16:00 job.err
> -rw-rw-r-- clasg13/clas     64109 2010-08-19 08:16:00 job.out
>
>
> ... and on the work disk
>
> /work/clas/clasg13/pass2/v0/AnaHistFiles/
> 11524608 anahist_054392_A04.hbook
>
> /work/clas/clasg13/pass2/v0/dariamon
> 396800 dariamon_054392_A04.root
>
> /work/clas/clasg13/pass2/v0/GFLUX
> 396800 gflux054392.a04.hbook.gz
>    20992 gflux54392_eb.a04.dat
>     2560 gflux54392_erg2.a04.dat
>     4096 gflux54392_erg.a04.dat
>     3072 gflux54392_tc.a04.dat
>
> /work/clas/clasg13/pass2/v0/LOGS
>    7168 run_054392_A04.err
> 12800 run_054392_A04.log
> 65536 run_054392_A04.out
>
> /work/clas/clasg13/pass2/v0/skims
> 4193280 strange_k0fix_DST_054392.A04.root
> 1312768 strange_kplus_DST_054392.A04.root
>
> /work/clas/clasg13/pass2/v0/skims/1pos1neg1neut
> 415415808 1pos1neg1neut_054392.A04.B00
>
> /work/clas/clasg13/pass2/v0/skims/2pos1neg
> 325581824 2pos1neg_054392.A04.B00
>
> /work/clas/clasg13/pass2/v0/2pos2neg
> 54221824 2pos2neg_054392.A04.B00
>
> /work/clas/clasg13/pass2/v0/slavamon
>    1536 slavamon_054392_A04.dat
>    5120 slavamon_054392_A04.root
>
> /work/clas/clasg13/pass2/v0/TRIP
> 3072 clas_054392.A04.trip
>
>
> And, based on the cooking of /mss/clas/g13/data/clas_054392.A04
> the total per cook job going onto the clasg13 work disk is ~813M.
> Most of that comes from Paul's 2 bigger skims which total ~740M
>
> There's 1.5T availble on /work/clas/clasg13. That's about enough for the
> skimmed output of about 30 runs, which is about isn't much! Paul, do you
> have a plan for these?
>
> The "expansion factor" ie ratio of new files on silo after cooking to
> raw data is 3.65, based on the cooking of /mss/clas/g13/data/
> clas_054392.A04
>
>
> Cheers,
> Ken
>
>
>
>
>
>    


-- 
=======================================================
Ken Livingston

Dept. of Physics&  Astronomy,        Tel: +44 141 330 6428
University of Glasgow,               Fax: +44 141 330 5889
Glasgow G12 8QQ.
Scotland. UK.
=======================================================

More information about the g13 mailing list