[Dsg-rich] RICH PVs disconnect after softIOC reboot.

[Nathan Baltzell]

Hi Tyler,

Need to look into why the cRio isn’t connecting to the softioc (attached). Do you see any errors or logging info related to EPICS on the cRio.  We definitely shouldn’t have to reboot the softioc every time these client-mode cRios are rebooted.  Is the problem specific to this cRio?  Does the EP cRio have the same behavior?

I confirmed everything is good on the softioc, and it’s accepting connections and writes to various N2/gas channels.

-Nathan




> On May 14, 2018, at 10:53 AM, Tyler Lemon <tlemon at jlab.org> wrote:
> 
> Hi Nathan,
> 
> We will not reboot the IOC anymore before talking to you and getting the okay to do so.
> 
> Since the IOC was working fine again with your help, I did the last reboot of the cRIO to finalize the PV typo correction in the cRIO's start-up application. After the cRIO started back up and its start-up program started running, the EPICS screen now seems to be frozen. I have verified that the LabVIEW is working correctly up to the point where it writes to the variable assigned to the PV.
> 
> At this point in the past, a reboot of the soft IOC would be done to try to fix the frozen EPICS. Is there something else that we could do besides rebooting the IOC that would fix the frozen EPICS screen?
> 
> Please let us know.
> 
> -Tyler
> 
> From: "Nathan Baltzell" <baltzell at jlab.org>
> To: "Tyler Lemon" <tlemon at jlab.org>
> Cc: "dsg-rich" <dsg-rich at jlab.org>
> Sent: Monday, May 14, 2018 10:43:09 AM
> Subject: Re: RICH PVs disconnect after softIOC reboot.
> 
> > After the cRIO reboot, the PVs were disconnected.
> 
> Due to the way this setup works, that’s hard to believe.  You can completely turn off the crio permanently without even affecting the softioc (except the numbers won’t update, and a heartbeat alarm will be generated, and then when you reboot the crio the numbers will start updating again).  You can test that if you want.
> 
> Are you sure you didn’t reboot the softioc before the PVs went disconnected?
> 
> Please let me know next time the PVs go disconnected, before doing anything with the softioc.
> 
> -Nathan
> 
> 
> 
> 
> 
> > On May 14, 2018, at 10:35 AM, Tyler Lemon <tlemon at jlab.org> wrote:
> > 
> > Hi Nathan,
> > 
> > There was a PV that had a typo on the cRIO client that was corrected, to set the corrected LabVIEW program as the start-up application, the cRIO needs to reboot. After the cRIO reboot, the PVs were disconnected. The past procedure we've used to fix this is to reboot the softIOC.
> > 
> > In the future, we will not reboot the softIOC. Is there something else we should do if after a cRIO reboot, the PVs are disconnected?
> > 
> > Best regards,
> > Tyler
> > 
> > From: "Nathan Baltzell" <baltzell at jlab.org>
> > To: "Tyler Lemon" <tlemon at jlab.org>
> > Cc: "dsg-rich" <dsg-rich at jlab.org>
> > Sent: Monday, May 14, 2018 10:22:32 AM
> > Subject: Re: RICH PVs disconnect after softIOC reboot.
> > 
> > Hi Tyler,
> > 
> > There was a bug in the database file after some cleanup after the run.  I fixed it and rebooted.  Sorry about that.  
> > 
> > But actually, there’s basically zero reason to ever reboot that ioc.  Certainly no changes on the crio ever require rebooting that ioc.  What were you trying to achieve by rebooting it?
> > 
> > -Nathan
> > 
> > 
> > > On May 14, 2018, at 10:10 AM, Tyler Lemon <tlemon at jlab.org> wrote:
> > > 
> > > Hi Nathan,
> > > 
> > > This morning I rebooted RICH's soft IOC after updating an incorrect PV name on the EPICS client of  RICHCRIO (IP: 129.57.160.213).
> > > 
> > > After the reboot, all but three RICH interlock PVs show as disconnected on the EPICS screen, even the ones for RICHCRIOS1EP (IP: 129.57.160.214) whose EPICS client was not touched in any way.
> > > 
> > > Rebooting the softIOC does not fix the error, nor does rebooting the cRIO.
> > > 
> > > Is there anything else that would allow all PVs to reconnect?
> > > 
> > > Please let us know.
> > > 
> > > Best regards,
> > > Tyler

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/dsg-hallb_rich/attachments/20180514/1207b1e5/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2018-05-14 at 11.17.31 AM.png
Type: image/png
Size: 34815 bytes
Desc: not available
URL: <https://mailman.jlab.org/pipermail/dsg-hallb_rich/attachments/20180514/1207b1e5/attachment-0002.png>