[Halld-offline] Offline Software Meeting Minutes, June 28, 2017

[Mark Ito]

Please find the minutes below and at 
https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_June_28,_2017#Minutes 
.

__________________________


    Minutes

Present:

  * *CMU*: Naomi Jarvis, Mike Staib
  * *FIU*: Mahmoud Kamel
  * *Glasgow*: Peter Pauli
  * *JLab*: Alexander Austregesilo, Thomas Britton, Brad Cannon, Hovanes
    Egiyan, Mark Ito (chair), David Lawrence, Dmitry Romanov, Elton
    Smith, Nathan Sparks, Simon Taylor, Beni Zihlmann
  * *Yerevan*: Hrach Marukyan

There is a recording of this meeting <https://bluejeans.com/s/ex7@/> on 
the BlueJeans site. Use your JLab credentials to access it.


      Announcements

 1. *Nightly builds include hdgeant4 and gluex_root_analysis.* See
    Mark's email
    <https://mailman.jlab.org/pipermail/halld-offline/2017-June/002826.html>.
 2. *HDPM 0.7.1 released.* See Nathan's email
    <https://mailman.jlab.org/pipermail/halld-offline/2017-June/002828.html>.
 3. *Status of HDvis, the GlueX 3-D Event Display.* Thomas and Dmitry
    demo'ed the latest version. Click here
    <https://halldweb.jlab.org/talks/2017/HDvis/js/event.html> to view
    an event. The underlying technology has changed from ROOT to
    three.js <https://threejs.org/> and it now runs in the browser. They
    will give a fuller report at the next offline meeting.
 4. *Status of REST production*. Alex summarized progress. See the
    2017-01 ver01 batch01 web page
    <https://halldweb.jlab.org/data_monitoring/recon/summary_swif_output_recon_2017-01_ver01_batch01.html>
    for details. Twenty three thousand jobs have run or 75% of the first
    half of the Spring 2017 run. The efficiency of use of the farm nodes
    has had some mysteries from time to time. We should have been able
    to go twice as fast.
 5. *New Sim-Recon release, version 2.15.0*. See Mark's email
    announcement
    <https://mailman.jlab.org/pipermail/halld-offline/2017-June/002835.html>.


      CCDB update

Dmitry gave us an update. See his slides for the details 
<https://docs.google.com/presentation/d/1I4HwOJvdjp5HgRyDrGdSxHKOUsw8QfE-mGETosQI16Q/edit?usp=sharing>. 
In summary, in response to reports of extremely slow start-up times for 
DANA-based jobs (~10 minutes), he has implemented a memory caching 
system for CCDB. The long delays were coming from multi-threaded jobs 
that access SQLite versions of the database. Each thread needs to access 
the database for its constants and in addition, some systems are 
accessing the database multiple times, each time retrieving the same 
data. He has measured an improvement of a factor of 35 in the 
CCDB-related start-up time for 36 threads in a fully loaded DANA program.

A new tagged release of CCDB, with the cache feature available, will 
appear soon.


      Progress on using the OSG

We skimmed Richard Jones's recent email 
<https://mailman.jlab.org/pipermail/halld-offline/2017-June/002827.html> 
on running GlueX software on the OSG. Folks are encouraged to try it 
out. The first step is to get a certificate for the Gluex VO from the OSG.

In this context David reported that as part of his preparations for the 
upcoming S&T review, Graham Heyes (and others) have noticed that our 
demand on the farm for simulation is almost an order of magnitude less 
than projections we have made in the past. Part of this is focus on 
asymmetries, which do not need large samples of Monte Carlo, another 
part is continued work in understanding detector response and modeling 
it faithfully in the simulation. There are already physics analyses 
underway that could benefit from high-quality simulated data; it is not 
as if there is no structural demand.


      HDPM support

Nathan gave an assessment on HDPM support. He will no longer be able to 
support the package going forward. Recall that it provides pre-built 
binaries for several distributions with a relatively simple command-line 
interface. Several collaborators are using it to support their 
development work. To create the binaries, Nathan has been using Docker 
containers to emulate the systems on the various target Linux 
distributions. There is documentation to maintain, occasional bugs that 
need fixing and 
computing-environment-changes/feature-requests/new-packages that need to 
be supported.

The support issue comes down to finding a volunteer to take over for 
Nathan. David pointed out that the support task is a good, finite-effort 
collaboration service task. He agreed to bring up the issue with the 
collaboration board.

-- 
Mark Ito, marki at jlab.org, (757)269-5295

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170629/94f210a0/attachment.html>