<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">We had a minor ssh issue. Just troublesome to sort, since I couldn't ssh in to look around.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Sounds like something is still quite wrong with lcls-llrfcpu02. <span style="font-size: 12pt;">What do we need to do to help troubleshoot it?</span></p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Wesley</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Sonya Hoobler <sonya@slac.stanford.edu><br>
<b>Sent:</b> Friday, September 21, 2018 11:54:31 AM<br>
<b>To:</b> Wesley Moore<br>
<b>Cc:</b> lerftest-ctrls@jlab.org; Curt Hovater; se_c@cox.net<br>
<b>Subject:</b> Re: [Lerftest-ctrls] RF CPU reboot & iocConsole problems</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi again,<br>
<br>
I see that lcls-llrfcpu02 still does not boot--tftp timeout.<br>
<br>
I'll go back to hands-off.<br>
<br>
Sonya<br>
<br>
<br>
<br>
On Fri, 21 Sep 2018, Sonya Hoobler wrote:<br>
<br>
> Hi Wesley,<br>
><br>
> I just logged in to look around and things seem improved.<br>
><br>
> Was something done to address the problems?<br>
><br>
> Thanks,<br>
> Sonya<br>
><br>
><br>
> On Thu, 20 Sep 2018, Sonya Hoobler wrote:<br>
><br>
>> Hi Wesley,<br>
>> <br>
>> Thank you for the update and for following up.<br>
>> <br>
>> I won't do anything until hearing back from you.<br>
>> <br>
>> Sonya<br>
>> <br>
>> <br>
>> <br>
>> On Thu, 20 Sep 2018, Wesley Moore wrote:<br>
>> <br>
>>> Sonya,<br>
>>> <br>
>>> Larry rebooted lcls-llrfcpu02. Said it powered back up, but isn't showing <br>
>>> any connectivity. Looks the same from my end.<br>
>>> <br>
>>> lclsfs - can't ssh, but pingable<br>
>>> lclsapp1 - seems fine<br>
>>> lclsapp2 - can't ssh, but pingable<br>
>>> lcls-llrfcpu01 - seems fine<br>
>>> lcls-llrfcpu02 - can't ssh, can't ping<br>
>>> Control room hosts (lclsl01-03): can't ssh, but pingable<br>
>>> <br>
>>> Let me follow up with the guy that setup the fileserver and see if we can <br>
>>> get that checked out first. We may need to reboot stuff after that's <br>
>>> sorted out.<br>
>>> <br>
>>> Wesley<br>
>>> <br>
>>> On 9/20/18, 9:39 AM, "Lerftest-ctrls on behalf of Wesley Moore" <br>
>>> <lerftest-ctrls-bounces@jlab.org on behalf of wmoore@jlab.org> wrote:<br>
>>><br>
>>> Looks like at least lcls-llrfcpu02 needs to be rebooted. Others seem <br>
>>> likely as well. The control room hosts aren't connecting either. Have <br>
>>> you heard anything from Larry?<br>
>>><br>
>>> Wesley<br>
>>><br>
>>> On 9/19/18, 7:20 PM, "Lerftest-ctrls on behalf of Sonya Hoobler" <br>
>>> <lerftest-ctrls-bounces@jlab.org on behalf of sonya@slac.stanford.edu> <br>
>>> wrote:<br>
>>><br>
>>> Hi Wesley, all,<br>
>>><br>
>>> I just tried a reboot of RF CPU lcls-llrfcpu02 and it never <br>
>>> successfully<br>
>>> re-booted up.<br>
>>><br>
>>> I can't view the boot-up sequence because iocConsole is also no <br>
>>> longer<br>
>>> working for either CPU:<br>
>>><br>
>>> [softegr@lclsapp1 iocCommon]$ iocConsole lcls-llrfcpu01<br>
>>> : ssh -x -t -l laci lclsapp2.acc.jlab.org bash -l -c " <br>
>>> pyiocscreen.py -t HIOC lcls-llrfcpu01 lclsts1 2001 "<br>
>>> Read from socket failed: Connection reset by peer<br>
>>> [softegr@lclsapp1 iocCommon]$ iocConsole lcls-llrfcpu02<br>
>>> : ssh -x -t -l laci lclsapp2.acc.jlab.org bash -l -c " <br>
>>> pyiocscreen.py -t HIOC lcls-llrfcpu02 lclsts1 2002 "<br>
>>> Read from socket failed: Connection reset by peer<br>
>>><br>
>>> I tried a remote reboot of the terminal server.<br>
>>><br>
>>> I also tried ipmitool (and EPICS ipmi) to remotely restart the CPU.<br>
>>><br>
>>> But still no signs of life.<br>
>>><br>
>>> Perhaps you could take a look at the network and/or locally? We may <br>
>>> need a<br>
>>> local power-cycle of the CPU and the terminal server. I'm cc'ing <br>
>>> Larry<br>
>>> Farrish, who may also be able to help with that.<br>
>>><br>
>>> This is not super urgent. When you have a chance during your normal<br>
>>> working hours, I'd appreciate any help.<br>
>>><br>
>>> Thanks,<br>
>>> Sonya<br>
>>><br>
>>> _______________________________________________<br>
>>> Mailing List: Lerftest-ctrls@jlab.org<br>
>>> <a href="https://mailman.jlab.org/mailman/listinfo/lerftest-ctrls">https://mailman.jlab.org/mailman/listinfo/lerftest-ctrls</a><br>
>>> Wiki: <a href="https://wiki.jlab.org/lerf/index.php/Network">https://wiki.jlab.org/lerf/index.php/Network</a><br>
>>> <br>
>>> <br>
>>><br>
>>> _______________________________________________<br>
>>> Mailing List: Lerftest-ctrls@jlab.org<br>
>>> <a href="https://mailman.jlab.org/mailman/listinfo/lerftest-ctrls">https://mailman.jlab.org/mailman/listinfo/lerftest-ctrls</a><br>
>>> Wiki: <a href="https://wiki.jlab.org/lerf/index.php/Network">https://wiki.jlab.org/lerf/index.php/Network</a><br>
>>> <br>
>>> <br>
></div>
</span></font></div>
</body>
</html>