[Dsg-hallb_magnets] Torus CERNOX board failure

[Brian Eng]

Some of these questions Rama or Nick might need to answer.

1. No, we don't have a good answer as to why it failed, the bad batch is just a guess at this point. The same component is also on the -12V circuit as well, so far we've only seen failures on the +12V side.

2. Since we don't have a solid understanding of why the capacitor failed (it is rated plenty high enough) I suppose removing it could be simply moving the problem elsewhere. However, this is only a power filter/decoupling cap and by the time the voltage gets to this point it is already quite well filtered. Utility power -> JLAB transformers -> UPS -> AC/DC PS in chassis -> PS Breakout PCB -> PCB in question.

I think we should also just leave the boards alone, while having 2 of the exact same component fail is concerning we don't know if it's going to be widespread yet.

Aside from the Cernox sensors this PCB also handles load cells.

> On Oct 12, 2017, at 3:29 PM, Ruben Fair <rfair at jlab.org> wrote:
> 
> Brian,
> 
> This looks very encouraging.
> 
> It certainly sounds like we should remove C34 from all the CERNOX boards.
> 
> However, before we do that, a couple of questions:
> 
> 1. Do we understand why C34 failed after a period of time (about one year for one of the chassis?). I know you said that it might be a bad component batch issue.
> 
> 2. By removing C34, could we be moving the problem elsewhere with another imminent component failure down the road?
> 
> If we cannot answer the above questions satisfactorily, I would recommend that we leave the remaining boards as they are.
> 
> Can you check and give me a list of which CERNOX sensors are interlocked to produce either a controlled ramp down or a fast dump?
> 
> There may be a way that we can avoid fast dumps if the boards go bad during a Physics run, as a splice which is slowly going bad will first produce a slowly increasing voltage followed by a slowly(?) increasing temperature, whereas a board failure causes the splice temperature read back to suddenly jump to a high value.
> 
> Regards
> 
> Ruben
> 
> 
> From: "Brian Eng" <beng at jlab.org>
> To: "Ruben Fair" <rfair at jlab.org>, "dsg-hallb magnets" <dsg-hallb_magnets at jlab.org>
> Cc: "bachiman" <bachiman at jlab.org>, "Probir Ghoshal" <ghoshal at jlab.org>, "Denny Insley" <dinsley at jlab.org>
> Sent: Thursday, October 12, 2017 3:17:42 PM
> Subject: Re: Torus CERNOX board failure
> 
> Forgot to send to mailing list, here's the results of swapping the boards out: https://logbooks.jlab.org/entry/3488318
> 
> I'm not sure what the expected accuracy/precision of the LV chassis and/or Cernox is, but it seems like a few mK difference is negligible?
> 
> > On Oct 11, 2017, at 10:51 AM, Brian Eng <beng at jlab.org> wrote:
> > 
> > Hi Denny,
> > 
> > I plan on swapping the PCB out today sometime before lunch.
> > 
> > So there will definitely be some alarms. I'll also need to restart the cRIO once the swap out is complete, which might cause additional alarms.
> > 
> >> On Oct 10, 2017, at 10:27 AM, Ruben Fair <rfair at jlab.org> wrote:
> >> 
> >> Brian,
> >> 
> >> Torus Chassis #4 should be fine.
> >> 
> >> Please just let Denny know before you swap it out as the splice temperature on the hex beam will trip the interlock again as it did last time.
> >> 
> >> Ruben