[Halld-offline] Offline Software Meeting Minutes, August 6, 2014

[Mark Ito]

Folks,

Please find the minutes below and at 
https://halldweb1.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_6,_2014#Minutes 
.

   -- Mark
______________________________________

GlueX Offline Meeting, August 6, 2014
Minutes

    Present:
      * CMU: Will Levine, Paul Mattione
      * FSU: Aristeidis Tsaris
      * JLab: Mark Ito (chair), David Lawrence, Simon Taylor, Beni Zihlmann
      * NU: Sean Dobbs

Review of Minutes from July 23

    We went over the [29]minutes.

Automatic Tests

      * Both b1pi and single-track were working two weeks ago.
      * Recent single-track failures were due to a scripting mistake Mark
        introduced. It is fixed now.
      * Recent b1pi failure is suspected to be due to CCDB table type
        definition changes. Discussion was deferred to later in the
        meeting.

Build Items

    Mark has done a tiny bit more work on the Version Management System,
    principally coming up with a new name for the project, now abbreviated
    VMS. ;-)

Complaints

    Will complained about a problem where a typo in the specification of a
    SQLite CCDB database file resulted in a "weird crash" with no explicit
    indication that the file could not be found. He will send a problem
    report out to the offline email list.

Make and SCons

    We discussed several topics.

With Sim-Recon

    Mark recounted the exercise of deleting all of the makefiles in the
    sim-recon tree and then putting them all back. Richard Jones had
    pointed out that simply removing makefiles renders some directories
    unbuildable.

    We thought that converting all directories to SCons should be pursued.
    David is best qualified to carry this out. One difficulty is AmpTools.
    He thought that others might be better at that conversion. Paul pointed
    out that if we split some packages into a separate analysis tree,
    AmpTools would likely make that migration. We agreed to go ahead with
    the program of global conversion, but to save AmpTools for last.

Install Target

    We endorsed the policy, suggested by Matt Shepherd, of having "install"
    as the standard target for both make and SCons that renders a
    user-usable build, in particular copying libraries and executables into
    publicly accessible directories.

HDDS

    David has checked-in a working set of SCons files for HDDS. One
    difficulty is that it inverts the directory hierarchy between bin/lib
    and platform identifier (with the latter on top) from the BMS practice
    bringing it into alignment with the sim-recon SBMS practice. However
    the SBMS system of sim-recon currently expects the old hierarchy. We
    will thus need to coordinate the switch of HDDS with modifications to
    the sim-recon SCons system.

Building Plugins

    We talked about the procedure for building plugins, private and public
    (those already in sim-recon tree).
      * Private. Use the mkplugin[?] script. A SConstruct will be generated
        for you. "scons install" will place the shared library in your
        "HALLD_MY" directory.
      * Public. Two cases.
          1. Built as part of the sim-recon complete build. Nothing to do.
             Use the version from the sim-recon build.
          2. Not built by complete build. Check out the complete sim-recon
             tree. Go to the plugin directory of interest and do a "scons
             -u install".

Data Challenge 3

    Mark has made some progress, but we are now a bit behind our nominal
    schedule of starting mid-August.

Tagger Simulation Update

    Paul brought us up-to-date.

    Richard handed the development branch over to Paul. Paul has made the
    changes to the analysis library to work with the new smeared tagger
    time, although for now he is cheating by taking the true tagger hit.
    This is useful for now to establish that the branch will work as
    before, when indeed this exact cheat was in, once the branch changes
    are put on the trunk.

    There is a problem right now with BCAL reconstruction on the branch.
    Although the number of BCAL hits is nominal, the number of
    reconstructed showers is way down at only 3% of nominal. It is not
    known if this problem is present on the trunk as well since the trunk
    is broken and cannot be tested (see the next section).

    Paul had introduced a new class: DRFTime[?] to store the
    beam-bucket-choice information[?]. Other important changes were made;
    he will report on them in the near future.

Broken Trunk

    There is a suspicion that changes to table definitions, introduced by
    Richard, in the CCDB have broken the trunk, crashing jobs, in
    particular the b1pi test. The changes are in the tagger constants,
    likely having to do with the geometry and energy binning information
    coming from the database.

    If we assume that this is the case, then the preferred method for
    changing the format of the tables would have been to simply define new
    tables and have the new code refer to the new tables. That way old code
    will continue to see the old tables it has always seen. The only
    problem with this method is aesthetic; it leads to names like adcgain2
    when adcgain is more succinct.

    Note that this method is not necessary if only one user is messing with
    the constants. The format changes and the code changes happen in sync.
    The problem comes if more than one set of code is accessing the same
    table name. We think that that is what is happening now; the trunk and
    your development branch have different expectations about what they
    will find. Another way of saying it is that the table definitions are
    global in the CCDB system.

    We could version control the database itself, or version control table
    format definitions, but that adds yet another layer of versioning to
    the already rich versioning structure already in the database design.

    Mark proposed rolling back the CCDB database to it's state as of 8 pm,
    Fri. Aug. 1. This will break the code on the development branch.
    Richard would then have to add new calibration types to the CCDB (with
    global scope) and change the code on the branch to use them.

    [added in press] The backup file has been identified. It is trivial to
    do the restore, though we will have to ask the Computer Center to do
    it.

    There is some time pressure in doing this since if people put in CCDB
    changes before the roll back, they will be lost. A day or two delay is
    probably acceptable.

    We need to talk to Richard before doing anything.

    [added in press] Richard objects to the roll-back solution. He will be
    at JLab on Friday and Mark plans to discuss this with him then. In the
    meantime, Paul, Sean, and Mark are looking at other options.

Offline Commissioning

    Sean called our attention to the [30]tasks outlined for the offline in
    the commissioning plan. We need to add content to this page. We
    discussed the issue of overlap of this task list with the efforts of
    the individual detector groups during commissioning, but for the
    mid-level and high-level tasks the offline has clear responsibility.

BCAL Time Reconstruction Update

    Will reported that the situation with this issue has not changed in
    quite a while. He will send some example events to Richard as we had
    discussed before. This issue needs to get more attention; it will
    remain on the agenda for future meetings and ideas are solicited from
    the Collaboration.

Action Items

     1. Send complaint about missing CCDB database files to email list.
        (Will)
     2. Add SConscripts to all sim-recon directories. (David)
     3. Fix sim-recon expectations of HDDS hierarchy in SBMS. (David)
     4. Report on analysis library changes related to introducing global
        event time smearing. (Paul)
     5. Resolve the broken-trunk problem.
     6. Understand the source of the BCAL cluster finding bug.
     7. Make some commissioning plans for the offline. (all)
     8. Upgrade priority of resolving the BCAL-late-timing-tail issue.
        (Mark)

References

   29. 
https://halldweb1.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_July_23,_2014#Minutes
   30. https://hdops.jlab.org/wiki/index.php/Offline_Analysis_Commissioning

-- 
Mark M. Ito, Jefferson Lab, marki at jlab.org, (757)269-5295