<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi Ben, <o:p></o:p></p>
<p class="MsoNormal">Thanks for the feedback. Yes the plan is to make some warning message available in the RunControl GUI but not to stop the run. Then shift crew will be instructed to check this error message during each run and alert the GEM expert immediately.
<o:p></o:p></p>
<p class="MsoNormal">It will also be very helpful to have a log entry pass/fail to be emailed directly to GEM expert as you suggested<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I understand that it is a hardware issue that get fixed only after access to the hall but it is certainly not the only issue that causes the MPD/DAQ to hang<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards <o:p></o:p></p>
<p class="MsoNormal">Kondo<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Benjamin Raydo <braydo@jlab.org> <br>
<b>Sent:</b> Tuesday, December 21, 2021 9:16 PM<br>
<b>To:</b> Gnanvo, Kondo (kg6cq) <kg6cq@virginia.edu>; Bryan Moffit <moffit@jlab.org>; Alexandre Camsonne <camsonne@jlab.org><br>
<b>Cc:</b> sbs_gems@jlab.org<br>
<b>Subject:</b> Re: GEM MPD DAQ - APV hang up issue for January run<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Hi Kondo,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">It doesn't seem like a big deal to change the readout list to work that way (automatically disable the APVs that failed to initialize and report a warning). Somehow someone will have to be on top
of this to ensure long runs are not taken before noticing this issue (which would probably boil down to someone looking at RunControl GUI warnings and reporting this to GEM experts).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I would definitely have to defer to Bryan on implementing this and he's likely a lot more aware of possibilities on how this could be reported (maybe a GMN GEM status log entry for pass/fail that
has a GEM expert emailed to help keep them on top of the issue? Not sure how reasonable/feasible this would be).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I definitely understand nothing here is ideal short of fixing the APV init issues. If you think it would be helpful to summarize the type issues and ways they are recovered maybe we could try to
figure out a software or firmware fix (we have some of these issues showing up in EEL125 setup too so I was planning to look more at this in January anyway - I'm definitely aware of some cases where the MPD firmware is in a funny state that causes all APVs
to fail to initialize that I'm hoping to address for example).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Ben<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="1" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Gnanvo, Kondo (kg6cq) <</span><a href="mailto:kg6cq@virginia.edu">kg6cq@virginia.edu</a><span style="color:black">><br>
<b>Sent:</b> Tuesday, December 21, 2021 11:55 AM<br>
<b>To:</b> Bryan Moffit <</span><a href="mailto:moffit@jlab.org">moffit@jlab.org</a><span style="color:black">>; Alexandre Camsonne <</span><a href="mailto:camsonne@jlab.org">camsonne@jlab.org</a><span style="color:black">>; Benjamin Raydo <</span><a href="mailto:braydo@jlab.org">braydo@jlab.org</a><span style="color:black">><br>
<b>Cc:</b> </span><a href="mailto:sbs_gems@jlab.org">sbs_gems@jlab.org</a><span style="color:black"> <</span><a href="mailto:Sbs_gems@jlab.org">Sbs_gems@jlab.org</a><span style="color:black">><br>
<b>Subject:</b> [EXTERNAL] GEM MPD DAQ - APV hang up issue for January run</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:11.0pt">Dear Bryan, Ben.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">We discussed at the RC meeting yesterday whether MPD readout list could be modified to allow the DAQ to keep on running even if a single APV fail to configure properly at the beginning of a run.
</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Right now, we have had a lot of occurrence where one APV failure cause an MPD / fiber link error that a shift crew can not really debug or fix. A GEM or DAQ expert will ultimately be called and it would results
to this APV been disabled. This usually results in a lot of wasted beam time during the shift without even fixing the APV problem.
</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">So it will be more efficient if the APV25 configuration fail is ignored and allow the DAQ to continue with some warning that will allow the shift crew to alert the GEM expert who can take a look at the issue
when there is an opportunity. This way, there is no beam time lost. </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Maybe the DAQ configuration will be allow to crash only if a number of APV25 (that we decide on, maybe 3 or 5 …), or all APVs on a backplane or MPD all fail to configure which would indicate a more serious
problem. </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">I was wondering if it is possible to implement such scheme in the MPD readout list
</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Best regards </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Kondo</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal"><b><span style="font-size:11.0pt">From:</span></b><span style="font-size:11.0pt"> Sbs_gems <</span><a href="mailto:sbs_gems-bounces@jlab.org"><span style="font-size:11.0pt">sbs_gems-bounces@jlab.org</span></a><span style="font-size:11.0pt">>
<b>On Behalf Of </b>Holly Szumila-Vance<br>
<b>Sent:</b> Monday, December 20, 2021 4:12 PM<br>
<b>To:</b> </span><a href="mailto:sbs_gems@jlab.org"><span style="font-size:11.0pt">sbs_gems@jlab.org</span></a><span style="font-size:11.0pt"><br>
<b>Subject:</b> [Sbs_gems] No SBS GEM meeting on Wednesday this week</span><o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Dear all,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">There will be no SBS GEM meeting this Wednesday. Please look for updates by e-mail.
</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Enjoy the holidays, and see you in the New Year!</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">-Holly</span><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>