[dsg-hallb_rich] RICH-1 EP cRIO failed

[Valery Kubarovsky]

Tyler,
I think that we have to prepare the spare cRIO and swap it during the first opportunity.
Machine was down today for example more than 4 hours. We can get time for control access any time.
But for sure we will get time for cRIO swap on Tuesday morning.
Regards,
Valery


________________________________
From: Tyler Lemon <tlemon at jlab.org>
Sent: Sunday, December 12, 2021 13:02
To: Valery Kubarovsky <vpk at jlab.org>; dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>
Subject: Re: RICH-1 EP cRIO failed


Hi Valery,

I saw your latest logbook entry.

Just to confirm, we will wait until Tuesday to go into the hall to swap the cRIO?

Thanks,
Tyler

________________________________
From: Valery Kubarovsky <vpk at jlab.org>
Sent: Sunday, December 12, 2021 12:40 PM
To: Tyler Lemon; dsg-hallb_rich at jlab.org
Subject: Re: RICH-1 EP cRIO failed

I think that we have to be prepared to swap the cRIO.
When we will get the opportunity to have control access to the Hall I cannot say.
I will talk with RC and let you know.
Valery

________________________________
From: Tyler Lemon <tlemon at jlab.org>
Sent: Sunday, December 12, 2021 12:36
To: Valery Kubarovsky <vpk at jlab.org>; dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>
Subject: Re: RICH-1 EP cRIO failed

Okay.

Does that mean I should get ready to go in to Hall B to swap the cRIO today?

Or will we wait until a later time, like during the regular controlled access period during the week?

-Tyler
________________________________
From: Valery Kubarovsky <vpk at jlab.org>
Sent: Sunday, December 12, 2021 12:32 PM
To: Tyler Lemon <tlemon at jlab.org>; dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>
Subject: Re: RICH-1 EP cRIO failed

Tyler,
We cannot rely only on the software FPGA interlock. Just recently the communication with the RICH mainframe was lost. It means that the software interlock did not work at that time. So, I am not fine to run RICH without hardware interlock.

We started new run without RICH. HV and LV are off. We will wait your actions to recover cRIO.
Valery


________________________________
From: Tyler Lemon <tlemon at jlab.org>
Sent: Sunday, December 12, 2021 12:24
To: Valery Kubarovsky <vpk at jlab.org>; dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>; Nathan Baltzell <baltzell at jlab.org>
Subject: Re: RICH-1 EP cRIO failed

There aren't really any N2 volume temperature sensors near the electronic panel so I would not use those as an indication of temperature in the EP.

If you're fine with running RICH with no hardware interlock temperature sensors in the electronic panel and relying on the software interlocks using FGPA temperature readout, then it can wait.

I may be able to remotely override the EP cRIO so RICH can be turned back on. If that doesn't work, someone will have to go turn a switch on the chassis in the hall.

Regardless of course of action, I'm going to plan on at least going into the EEL to get a spare cRIO ready for whenever the hall is accessible.
If you still want to discuss it on the phone, I'll call once I'm in the EEL (it should be by 1:00PM)

-Tyler


________________________________
From: Valery Kubarovsky <vpk at jlab.org>
Sent: Sunday, December 12, 2021 12:16 PM
To: Tyler Lemon <tlemon at jlab.org>; dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>; Nathan Baltzell <baltzell at jlab.org>
Subject: Re: RICH-1 EP cRIO failed

Unfortunately beam is coming back.
I see that N2 volume sensors are working.
What do you suggest?
Is it safe to take data with only N2 sensors ON?
Can we talk by phone?
My cell is 747-232-9658.
________________________________
From: Tyler Lemon <tlemon at jlab.org>
Sent: Sunday, December 12, 2021 12:14
To: dsg-hallb_rich at jlab.org <dsg-hallb_rich at jlab.org>; Valery Kubarovsky <vpk at jlab.org>; Nathan Baltzell <baltzell at jlab.org>
Subject: RICH-1 EP cRIO failed

Hi Valery,

I tried to reboot the cRIO for the RICH-1 electronic panel hardware interlock system to get it back online.

It didn't work.

Now it's saying that the cRIO does not recognize any of the modules at all.

Signs are pointing to a hardware failure on the cRIO so it should be replaced.

You mentioned there isn't beam to the hall currently, is it expected to stay that way for a while?

I can go in to swap to a spare cRIO.




Brian, Marc, Peter,

It looks like I'll have to swap out the cRIO. Do any of you know if we have a cRIO anywhere that isn't being used?

If I don't hear from any of you, my plan is to take the one from the development chassis on Peter's desk to swap them out.

-Tyler

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/dsg-hallb_rich/attachments/20211212/910f48d1/attachment.html>