[Hallb-engineering] [Esr-users] [Revised Logentry] Root Cause Search for ESR Trip 2/6/2018

[Paul Brindza]

Hi

The SHMS Dipole control net module in Hall C stopped working last evening resulting in a loss of ALL signals from SHMS Dipole and this caused the SHMS Dipole PSU to shut down.
The HMS Dipole did a fast dump.
This puts a lot of helium in the warm return.
This is the first time a control net module has ever stopped working.
A quick power cycle returned it to life.
This device is in the SHMS bunker.
There also was an issue more or less simultaneously with HB and Q1 that caused an overfill and over pressure in the magnet.
According to the time stamps the SHMS Dipole went first.

All Hall C magnets recovered normally

This is a candidate for the ESR trip but more information is needed.

Paul Brindza

----- Original Message -----
From: "Dave Kashy" <kashy at jlab.org>
To: hallb-engineering at jlab.org, esr-users at jlab.org
Sent: Tuesday, February 6, 2018 11:45:01 PM
Subject: [Esr-users] [Revised Logentry] Root Cause Search for ESR Trip	2/6/2018

Root Cause Search for ESR Trip 2/6/2018 


Lognumber 3528416 . Submitted by kashy on Tue, 02/06/2018 - 22:41. 

Last updated on Tue, 02/06/2018 - 23:42 Logbooks: 	HALOG HBLOG HCLOG CLOG ELOG 
Entry Makers: 	kashy 
Backlinks: 	Follow-up Re: Root Cause Search for ESR Trip 2/6/2018 


I looked though many signals in the Mya Viewer Archive. I found the following: 
In time order (times per the data logger, there may be time shifts due to data collection rates etc.) 
At 16:34:33 the SHMS Dipole started 1 rampdown that lasted about 1 minute 
At 16:34:40 the lead flow of the Hall C SHMS HB magnet went from 100 steady up to 113 then back down to 100 at 13:35:17 
At 16:35:18-23 all other Hall C SHMS started rampdowns that lasted sless than 1 minute 
At 16:35:19 the helium level in the Hall C SHMS HB went from 70 to 51 
At 16:35:26 the lead flow in the Hall C SHMS Q3 went form 88 to 85 (not a big change) 
At 16:35:37 the three pressure sensors on the quench headers of Halls A, B, C (CPI603QA, QB, QC) started to slowly rise from their flatline values 
At 16:35:40 the helium level of the Hall C SHMS Q3 went from 70% to 35% 
At 16:35:48 the Hall B Solenoid magnet resevoir started to slowly loose helium level 
At 16:36:02 the Hall A Right Dipole started to slowly ramp down but at 16:37:31 it started to ramp up 
At 16:36:08-12 All Hall C HMS magnets started rundowns that lasted 1-3 minutes 
At 16:37:42 the helium level in the Hall A right dipole started to climb then at 16:39:09 ti dopped form 96% to 26% 
At 16:44:56 ESR Turbines T1 and T2 tripped 
At 16:54:00 the Hall B solenoid fast dumped 
At 17:22:02 the ESR Turbine T2 was restarted 
AT 17:28:25 the Hall B Torus fast dumped 
At 17:30:54 the ESR Turbine T1 was restarted 

I can't make any firm conclusions, because I do not know the accuracy of the time stamps on the above data... 
I do not know all the interlocks on the Turbines but see no big events really close in time to the turbine trips 

_______________________________________________
Esr-users mailing list
Esr-users at jlab.org
https://mailman.jlab.org/mailman/listinfo/esr-users