[Halld-offline] Containers Meeting Notes, March 30, 2018

[Mark Ito]

Container Launchers,

Please find the notes below and at

https://halldweb.jlab.org/wiki/index.php/GlueX_Containers_Meeting,_March_30,_2018

   -- Mark

________________


  GlueX Containers Meeting, March 30, 2018


      Notes

Present: Sean Dobbs, Mark Ito, Thomas Britton, Beni Zihlmann, Zisis 
Papandreau, David Lawrence, Richard Jones

  * Atlas and CMS and OSG et al. designed a grid to do raw data
    reconstruction.
      o MC was not considered a problem
      o The tiered system has as a component the raw data distribution
        system
      o software and databases were staged as "image files"
  * XROOTD
      o originally developed at SLAC
      o redirector: directory service, contains meta data (file locations)
      o on the local host open a file just like a local file except file
        name is replaced with a URL like root://<name of catalog><IP
        name of redirector><file name>, triggers an open over network
      o redirector refers opener to site, a data connection is opened on
        site, data streamed over connection
      o our bandwidth requirements should be OK
      o David: at NERSC, 64 threads on a single node, reconstruction, 4
        MB/s read rate
      o there is a caching system, if everything is mostly working
        cache, is not heavily used
  * UConn demo system
      o redirector service started
      o 50 servers, each serving a few files
      o 10 Gb pipe
  * Client side responsibilities
      o download and build XROOTD
      o set-up ld_preload[?]
      o open calls get intercepted and redirected to XROOTD system
  * Sean has also messed with setting up an XROOTD server
  * David mentioned the XROOTD idea to Amber Boehnlein, she seems
    supportive
      o Consensus that it would be best if infrastructure supported at
        JLab on the submit host
      o Could be used for both raw data and random trigger files
      o Richard and Thomas will work on staging random triggers on UConn
        demo system
  * David: Chip may allow Physics to use 5% of future LQCD KNL cluster
  * David: planning for summer campaign at NERSC, reconstruction
      o Sean: candidates for off-site resource campaigns:
          + ρ and ω yield measurements with 10 or 20 files per run
          + 2017 data: 2 periods, low intensity and high intensity
          + 2018 data: 4 periods

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Containers_Meeting,_March_30,_2018&oldid=86291"

  * This page was last modified on 7 April 2018, at 21:52.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20180407/1293f44d/attachment.html>