[Halld-offline] Offline Software Meeting Minutes, August 20, 2015

[Mark Ito]

Friends,

Please find the minutes below and at 
https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_19,_2015#Minutes 
.

   -- Mark
    _____________________________________________________


    GlueX Offline Meeting, August 19, 2015, Minutes

Present:

  * *FIU*: Mahmoud Kamel
  * *FSU*: Brad Cannon, Aristeidis Tsaris
  * *JLab*: Alex Barnes, Amber Boehnlein, Mark Ito (chair), David
    Lawrence, Paul Mattione, Elton Smith, Nathan Sparks, Justin Stevens,
    Simon Taylor, Beni Zihlmann
  * *NU*: Sean Dobbs
  * *UConn*: Richard Jones


      Announcements

 1. The |build_scripts| repository moved to GitHub
    <https://mailman.jlab.org/pipermail/halld-offline/2015-August/002122.html>
    earlier this week. On a related policy note, we decided that if a
    repository is maintained by a single person, then we can skip the
    pull-request/merge-by-someone-else procedure and allow the
    maintainer to push directory to the repository.
 2. Kei Moriya posted links from last week's calibration meeting on
    Offline Monitoring (see below). Paul mentioned that the last launch
    went smoothly and Kei is feeding back suggestions for SWIF to Chris
    Larrieu and has written a GlueX-oriented SWIF wrapper, |hdswif|. The
    next launch will be next week.
      * Status report
        <https://halldweb.jlab.org/wiki/images/b/b1/2015-08-12-offline_monitoring.pdf>
      * hdswif <https://halldweb.jlab.org/wiki/images/3/3a/Hdswif.pdf>


      Review of minutes from August 5

We went over the minutes from last time 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_5,_2015#Minutes>. 
There was no significant discussion.


      Software for Correcting for Pedestal Drifts

Elton raised the issue of how to apply pedestal corrections that occur 
on a time scale smaller than a single run. There are several online and 
offline aspects to the problem and the method is not settled on either 
side. We discussed several aspects of the problem.

  * The pedestals have been observed to drift significantly on a scale
    of a few hours. We might need to change pedestal a values on a 10
    minute time scale.
  * The CCDB is currently set up to handle constants on a run-by-run
    basis. The conceptual design for event-dependent constants has been
    worked out, but nothing has been coded.
  * JANA already has the hooks for obtaining event-dependent constants
    from the CCDB when/if that functionality comes along.
  * There is a potential problem with constants that apply to event
    ranges since with multiple threads not every thread will cross a
    given event boundary at the same time. This could be solved by using
    the "event barriers" David recently introduced in JANA. This
    mechanism requires that all threads synchronize on a designated
    point in the input data stream, each thread not going past it until
    all threads have caught up. Event barriers can be triggered without
    dependence on special event types in the data stream. The
    development of this mechanism was originally motivated by the idea
    of using data from EPICS events in the data stream in the
    reconstruction.
  * There may be a possible bias in pedestals since read-out is only
    done for hits; there will always be a hit nearby in time when
    obtaining pedestals.
  * Depending on the online scheme, we may obtain pedestals on an
    event-by-event basis, or average them over a range of events. In the
    former case, calibration constants are not needed; the pedestals are
    embedded in the event stream.

We did not decide on a definite action plan. The character of the online 
solution is still evolving and that will largely dictate the offline 
approach.


      Data Challenge 3 update

Mark has produced about half of 5000 input files and written them to the 
tape library. The files are 10 GB and contain about 430 k events each. 
Each file takes about a day of CPU time to produce. These files have 
also been reconstructed in a practice run on the data. Processing is 
ongoing.


      Spring 2015 Commissioning Simulations

Mark reported that all of the 30,600 jobs 
<https://halldweb.jlab.org/wiki/index.php/Spring_2015_Commissioning_Simulations> 
are complete, both event generation and reconstruction. The "raw" data 
(mcsmear output, HDDM format), and the REST files have been archived to 
the tape library and are now available on the cache disk under 
/cache/mss/halld/detcom/02/smeared and rest respectively.

Sean mentioned that Elton was interested in simulations using the latest 
version of the simulation and reconstruction (more recent than what we 
used). He also cautioned that the new start counter monitoring package 
does not have correspondingly new constants for simulated data yet. Mark 
will look into tagging a release with the latest code save the start 
counter plugin.


      Geant4 Update

Richard reported on his detailed comparison of tracks between Geants 3 
and 4. The results from a sample of 2:

 1. track 1: perfect agreement for about 750 steps
 2. track 2: spectacular disagreement

He traced the discrepancy to a feature implemented in Geant 3 where the 
magnetic field was turned off on the calorimeters to speed up shower 
development. This, in fact, is where most of the simulation time is 
spent. He found a subtle bug having to do with how the magnetic field is 
inherited from mother volumes in Geant 4 which needs to be addressed.

He will look at the cost in Geant 4 of leaving the field on and report 
to the group. He was hopeful that the CPU time penalty would not be too 
bad and we can run with the field on the calorimeters. This mode would 
be important for charged particles punching through the calorimeter, of 
particular interest for the charge pion polarizability experiment.

He is also working to complete the class that allows each straw to 
declare its number.

Richard also needs to test the Geant4 multi-threaded tracking facility.

David will work on implementing sensitive detectors along the lines used 
for the CPP.


      Hall D Package Manager Update

Nathan gave us an update on the current state of the project for 
facilitating builds of GlueX software. See his slides 
<https://halldweb.jlab.org/talks/2015/hdpm-update.pdf> for details of 
his presentation. The major changes since his last report to the group 
are (from slide 10):

  * Name change to hdpm from HDBuildManager
  * hdpm command user-interface
  * Check dependencies within listed packages
  * Control build commands with "commands.txt"
  * Support for geant4 and amptools
  * Setup scripts for 64-bit Linux and Mac OS X

He urged us to give it a try. An extensive README 
<https://github.com/JeffersonLab/hdpm> is available on GitHub.


      Flash ADC Pulse Processing Emulation: who is doing what?

David described some of the current issues with emulation of the 
firmware algorithm used in the Flash ADCs.

 1. The current code does not reproduce the results of the firmware exactly.
 2. Parameters for the algorithm are different for different detectors.
    There is not mechanism at present to retrieve these parameters at
    reconstruction time.
 3. There are several people working on the problem, including Mike
    Staib, Mark Dalton and Kei.

To solve the second issue, in addition to access to the parameters 
themselves, one requires access to the translation tables to identify 
the relevant detector.

David will be calling a meeting to bring interested parties together, 
discuss the needed infrastructure, and try to avoid duplication of effort.


      Who is responsible for deleting Git branches?

The sim-recon Git repository is starting to accumulate a significant 
number of branches from code improvements pushed upstream. Most can be 
deleted, but we have not assigned responsibility for doing so within the 
group.

Mark proposed a policy where the developer that originally pushed the 
changes is responsible for deleting branches after merge. There was some 
discussion about whether having "person doing the merge does the delete" 
instead. That might be more efficient, but Mark argued that the person 
pushing the changes was in the best position to judge. In the end we 
decided to try Mark's proposal.


      Default for builds: with debug symbols or without?

Nathan presented results of studies 
<https://halldweb.jlab.org/talks/2015/debug_symbols.pdf> on performance 
of both the GCC and the Clang C++ compilers comparing performance with 
and without the production of debugging symbols. The symbols are not 
only important for running the debugger, but give line number 
information on crashes when binaries are run normally. Compilation is 
slowed significantly and the disk footprint is an order of magnitude 
bigger with the symbols.

The current default is to build with debugging symbols. Although Nathan 
proposed changing the default, the group endorsed David's original 
judgment that having symbols is worth the cost.

We also talked a bit about Clang. It seems to perform marginally better 
than GCC. One major advantage that people reported is that the error 
messages produced at compiler time are much easier to understand, no 
small advantage. Clang also seems to be available as a package in most 
common Linux distributions.


      Action Item Review

We did not get to this; the room was emptying quickly at this point. ;-)

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_August_19,_2015&oldid=69473"

  * This page was last modified on 21 August 2015, at 10:30.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20150821/f1bce980/attachment.html>