[Ace] Network timeouts being reported from various applications -- feels ominous.

[Brad Cumbia]

Omar, what ioc’s were you having trouble with?

--
Brad Cumbia

Accelerator Network Engineer
Accelerator SysAdmin Group

Thomas Jefferson National Accelerator Facility
12000 Jefferson Avenue
Newport News, Virginia 23606
Phone (757)269-5839

From: Ace <ace-bounces at jlab.org> on behalf of Michele Joyce <erb at jlab.org>
Date: Tuesday, September 7, 2021 at 10:26 AM
To: Gary Croke <gcroke at jlab.org>, Omar Garza <garza at jlab.org>, Theo Larrieu <theo at jlab.org>, ace at jlab.org <ace at jlab.org>, controls_dept at jlab.org <controls_dept at jlab.org>
Subject: Re: [Ace] Network timeouts being reported from various applications -- feels ominous.
Is anything machine dependent?


From: Controls_dept <controls_dept-bounces at jlab.org> on behalf of Gary Croke <gcroke at jlab.org>
Date: Sunday, September 5, 2021 at 11:09 AM
To: Omar Garza <garza at jlab.org>, Theo Larrieu <theo at jlab.org>, ace at jlab.org <ace at jlab.org>, controls_dept at jlab.org <controls_dept at jlab.org>
Subject: Re: [Controls_dept] Network timeouts being reported from various applications -- feels ominous.
What's the suspicion?  Has there been a big uptick in network traffic recently, or have we gradually been approaching some limit?  Or just some failing hardware somewhere?

________________________________
From: Controls_dept <controls_dept-bounces at jlab.org> on behalf of Omar Garza <garza at jlab.org>
Sent: Saturday, September 4, 2021 7:58 PM
To: Theo Larrieu <theo at jlab.org>; ace at jlab.org <ace at jlab.org>; controls_dept at jlab.org <controls_dept at jlab.org>
Subject: Re: [Controls_dept] Network timeouts being reported from various applications -- feels ominous.

This nay also be related
In the past weeks, we have also performed hard reboots on iocs with  multiple tries failing to complete network connections  .
2-3 hard reboots to  succeed.
Omar

Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Android<https://aka.ms/AAb9ysg>
________________________________
From: Ace <ace-bounces at jlab.org> on behalf of Theo Larrieu <theo at jlab.org>
Sent: Friday, September 3, 2021 5:27:32 PM
To: ace at jlab.org <ace at jlab.org>; controls_dept at jlab.org <controls_dept at jlab.org>
Subject: [Ace] Network timeouts being reported from various applications -- feels ominous.




Observing that in the past 24 hours there have been several seemingly unrelated applications having trouble making connections to EPICS signals.



Cavity History

https://logbooks.jlab.org/entry/3900743



FOPT

https://logbooks.jlab.org/entry/3900317



myRestore

https://logbooks.jlab.org/entry/3900705



It might be wise to start looking for signs of distress in name server, softIOC hosts, network switches, etc.



-Theo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/ace/attachments/20210907/ca09b633/attachment-0001.html>