[Halld-online] Online Meeting minutes

[David Lawrence]

Hi All,

  Minutes of today’s Online meeting have been posted on the wiki and copied below.

https://halldweb.jlab.org/wiki/index.php/OWG_Meeting_10-Aug-2016 <https://halldweb.jlab.org/wiki/index.php/OWG_Meeting_10-Aug-2016>

Regards,
-David


Minutes[edit <https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_10-Aug-2016&action=edit&section=5>]

Attendees: David L. (chair), Simon T., Naomi, J., Eugene C., Sandy P., Chip W., Beni Z., Bryan M., Dave A., Vardan G., Mark I., Curtis M., Cody D.
Announcements[edit <https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_10-Aug-2016&action=edit&section=6>]

New RAID Storage node is also delivered and is in Dat Center
David will be meeting with Paul L. next week to discuss where the nodes will be installed in rack room relative to switch and allowing for future farm expansion
High Intensity Running
Wiki page is up and gathering information
Need a dedicated meeting to help generate answers to questions (or form plan to generate answers)
Need new numbers by end of month to share with management and help with our own planning
Main issues:
Reduce L1 trigger rate
Bandwidth limitation in VME
Network Bandwidth to CC[edit <https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_10-Aug-2016&action=edit&section=7>]

Chip sent out an e-mail with some details on how we could go about maximizing are bandwidth to permanent tape storage. He went over some highlights and gave a few other ideas.
We currently have two 10Gbit fiber links from Hall-D to CEBAF center that have been tested to give an aggregate of ~1.8GB/sec sustained rate
Other fibers were strung and are available, but would need to be outfitted on both ends
Hall-B has a 40GBit link. Our fibers could be used for this, but there is significant cost involved for the hardware needed on both ends.
May be cheaper to implement 4x10Gbit links with clever scripting to ensure all links are being utilized
Lustre file system is capable of consuming a couple of Gbyte/sec
To maximize data rate, need to setup 2 file transfers per link for a total of 4 simultaneous file transfers
New RAID disk should be capable of supplying 1.6Gbyte/sec. (Old ones probably not, though we may improve their capability if we reconfigured them with the right stripes)
May need to configure new RAID to have 4 stripes to make it easy for DAQ to efficiently write to disks at maximum rate. (Mount these as 4 separate file systems?)
Will need to limit how much is written to each disk so that only outer cylinders of disk are used which have higher bandwidth
Slightly less than 2GB/sec currently available into tape library
If we intend to write 1.6GByte/sec sustained, Chip would need 1 month lead time to procure additional tape drives
Chip suggested option: Record 1.6GByte/sec to Data Center of "pre-raw" data. After run, submit farm jobs to apply L3 filtering/reformatting and generate "raw" data which will then have 2 copies made for permanent storage
The "pre-raw" data will then be discarded and the tapes reused for the "raw" data.
DAQ[edit <https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_10-Aug-2016&action=edit&section=8>]

Sergey has been testing single crate data rates
Modified trigger rate, holdoff time, and occupancy (via threshold and noise)
Used 3 CDC crates with fADC125 modules
256kHz period trigger
200kHz, 80% livetime, 35MB/sec/crate (no hit occupancy, headers only)
110kHz random trigger
Required ~2.5μs holdoff time (suspect this is due to minimum holdoff needed for system)
Lowered thresholds to implement non-zero hit occupancy
Realistic readout limit is 50-60MB/s per fADC125 crate (theoretical with current firmware is 66MB/s)
Realistic readout limit is ~120MB/s per fADC250 crate (theoretical with current firmware is200MB/s)
Cody thinks he knows why and can squeeze close to a factor of 2 out of it
fADC250 readout limited to 200MB/s
Also requires 2.5μs holdoff
Asked Cody if it would be possible to drop Trigger Time words from all but 1st and last events in block. He thought is was possible but will need to look at it.
Dave A. has been testing 10Gbit ethernet boards for ROCs.
Able to achieve ~630MB/s
Does not know current cost of cards, but likely around $500 each (we would probably need 14)
Switches in hall have 1Gbit links with a 10Gbit uplink.
May have second 10Gbit uplink that is currently unused
Options: Replace switches with one(s) containing multiple 10Gbit ports or run fibers directly from ROCs upstairs to switch in Counting House
Naomi showed some results of a brief study of the single chip occupancy in the fADC125
Chip occupancy seems to be around 5% for 80nA PARA run with 50μm diamond (210nA run needs to be looked at again)
AOT[edit <https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_10-Aug-2016&action=edit&section=9>]

Beni noted that we need to ask all subsystem groups to minimize their readout windows so as to minimize data footprint and therefore, bandwidth requirement
CDC and FDC have been looked at and are already at minimum

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-online/attachments/20160810/61fdbffc/attachment.html>