[Super-g] g12 pass1 cooking time estimates

[Johann Goetz]

Hi g12,
So pass0 v7 has finished (~5% of the data). We are still working on a
bug in g12_monitor to get the monitoring histograms. Anyways, the rate
of this pass was 256 files per 24 hours on average (over 9 days).

Now, if we were to increase the maximum number of jobs on the farm (for
our clasg12 account specifically for pass1) from 250 to 400, and we
include the rate of 150 jobs per day measured at FSU, we get 550 jobs
per day. Thus, pass1 will now take 112 days to complete, or about 16
weeks. This puts us to finish pass1 late November 2009.

My assumption here is that, with the increase in farm speed we expect in
the next few months -- nodes and cache space, we should be able to
offset downtime due to software problems and/or getting (slightly) less
than the 550 jobs.

I have done a profiling of the reconstruction program a1c here (these
images are free for anyone to use however they see fit):

http://clasweb.jlab.org/rungroups/g12/wiki/index.php/G12_Cooking#a1c_profiling

It looks as though the most time is spent doing the DC pattern matching
with the prlink tracks. This has lead me to resurrect the DC "Heddle"
filter program I wrote about 6 months ago although I don't expect to get
more than a 5% time difference to do the reconstruction, but we'll see.

Also, I am looking into getting access to the Berkeley computing farm
which could add many more jobs per day to the throughput.

-Johann