<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi All,<div class=""><br class=""></div><div class="">Minutes from today’s online meeting are now posted:<div class=""><br class=""></div><div class=""><a href="https://halldweb.jlab.org/wiki/index.php/OWG_Meeting_11-Jan-2017" class="">https://halldweb.jlab.org/wiki/index.php/OWG_Meeting_11-Jan-2017</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><h1 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 23.876px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="Minutes">Minutes</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=5" title="Edit section: Minutes" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h1><div style="margin: 0.4em 0px 0.5em; line-height: 1.5em; font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class="">Attendees: David L. (chair). Sergey F., Dave A., Carl T., Vardan G., Bryan M., Simon T., Curtis M., Hovanes E., Graham H.</div><h2 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 19.05px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="Announcements">Announcements</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=6" title="Edit section: Announcements" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h2><ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url(data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=); font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class=""><li style="margin-bottom: 0.1em;" class="">First run meeting of the Spring 2017 run is tomorrow morning</li></ul><h2 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 19.05px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="DAQ_specs_for_upcoming_run">DAQ specs for upcoming run</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=7" title="Edit section: DAQ specs for upcoming run" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h2><ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url(data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=); font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class=""><li style="margin-bottom: 0.1em;" class="">Current expectation is that we should be all to run DAQ at 50kHz during Spring run<ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url("data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=");" class=""><li style="margin-bottom: 0.1em;" class="">This is 2.5 times larger than original spec. but previous experience drives this expectation</li><li style="margin-bottom: 0.1em;" class="">First limitation is rate we can write to disk. This is roughly 1GB/s</li></ul></li><li style="margin-bottom: 0.1em;" class="">David showed a preliminary plot indicating this rate may be achievable with 1GB/s but further investigation is needed since this appears somewhat inconsistent with what was extrapolated from Spring 2016 data</li></ul><h2 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 19.05px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="gluonraid3_issues">gluonraid3 issues</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=8" title="Edit section: gluonraid3 issues" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h2><ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url(data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=); font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class=""><li style="margin-bottom: 0.1em;" class="">Corrupted data observed on gluonraid3, partition 3</li><li style="margin-bottom: 0.1em;" class="">Fall run used partition3 and 4 for beam data<ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url("data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=");" class=""><li style="margin-bottom: 0.1em;" class="">Each partition used 10 disks configured using software RAID0 and formatted with XFS</li></ul></li><li style="margin-bottom: 0.1em;" class="">Most files currently on gluonraid3, partition 3 have different md5 checksum values compared to what is on tape<ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url("data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=");" class=""><li style="margin-bottom: 0.1em;" class="">Looking closer at one file revealed the file on tape started with a valid EVIO block header while the copy on gluonraid3 was corrupted.</li><li style="margin-bottom: 0.1em;" class="">Timestamp of file on gluonraid3 was close to the timestamp of the mss stub file (from Dec. 16) suggesting file has not been modified.</li></ul></li><li style="margin-bottom: 0.1em;" class="">Hovanes pointed out that the problem could be with the controller and not a disk</li><li style="margin-bottom: 0.1em;" class="">Current plan is to invest in 2 hardware RAID controller cards to replace the JBOD controllers<ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url("data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=");" class=""><li style="margin-bottom: 0.1em;" class="">Will try and get them delivered and installed by end of next week</li></ul></li><li style="margin-bottom: 0.1em;" class="">Plan B: Use gluonraid1 and gluonraid2 at 800MB/s</li><li style="margin-bottom: 0.1em;" class="">Plan C: Try higher level of software RAID</li><li style="margin-bottom: 0.1em;" class="">Plan D: Get ZFS configuration optimized</li><li style="margin-bottom: 0.1em;" class="">Sergey noted that ieven if we do have RAID level redundancy for data protection, if a disk goes out the system performance will be degraded while it repairs itself, likely making it unusable for high rate I/O</li><li style="margin-bottom: 0.1em;" class="">Dave A. asked about spare disks for gluonraid3</li></ul><h2 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 19.05px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="Run_Preparations">Run Preparations</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=9" title="Edit section: Run Preparations" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h2><ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url(data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=); font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class=""><li style="margin-bottom: 0.1em;" class="">FCAL10 issue has been resolved (bad f250 module)</li><li style="margin-bottom: 0.1em;" class="">CODA 3.0.7 may be available by end of next week. It could provide better performance due to:<ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url("data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=");" class=""><li style="margin-bottom: 0.1em;" class="">More efficient object allocation leading to less garbage collection</li><li style="margin-bottom: 0.1em;" class="">Multi-threaded event building</li></ul></li></ul><h2 style="background-image: none; background-color: rgb(255, 255, 255); font-weight: normal; margin: 0px 0px 0.6em; overflow: hidden; padding-top: 0.5em; padding-bottom: 0.17em; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: rgb(170, 170, 170); font-size: 19.05px; font-family: sans-serif; font-variant-ligatures: normal; orphans: 2; widows: 2; background-position: initial initial; background-repeat: initial initial;" class=""><span class="mw-headline" id="Future_Online_Meetings">Future Online Meetings</span><span class="mw-editsection" style="font-size: small; margin-left: 1em; vertical-align: baseline; line-height: 1em; display: inline-block;"><span class="mw-editsection-bracket">[</span><a href="https://halldweb.jlab.org/wiki/index.php?title=OWG_Meeting_11-Jan-2017&action=edit§ion=10" title="Edit section: Future Online Meetings" style="text-decoration: none; color: rgb(90, 54, 150); background: none;" class="">edit</a><span class="mw-editsection-bracket">]</span></span></h2><ul style="line-height: 1.5em; list-style-type: square; margin: 0.3em 0px 0px 1.6em; padding: 0px; list-style-image: url(data:image/gif;base64,R0lGODlhBQANAIAAAGOMnP///yH5BAEAAAEALAAAAAAFAA0AAAIJjI+pu+APo4SpADs=); font-family: sans-serif; font-size: 12.7px; font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class=""><li style="margin-bottom: 0.1em;" class="">Next meeting is in 2 weeks, but we will likely be having regular run meetings at that time and will start deferring the Online meetings until after the run.</li><li style="margin-bottom: 0.1em;" class="">David noted that the original intent of the Online meetings was to provide tight coordination between Controls, Trigger, DAQ, and all other online systems (monitoring, copy to tape, gluon computer admin, ...). At this point though these topics have naturally broken off into separate meetings. The exception being DAQ which is partially covered in the L1 meetings and partially in the Online meetings. The Online meetings are usually not attended by Sascha who is the primarily responsible person for a large part of the DAQ. Thus, we should think whether the format of this meeting is the most efficient.</li></ul><div class=""><br class=""></div></div><div class=""><br class=""></div><div class="">Regards,</div><div class="">-David</div></div></body></html>