[Dsg-rich] scalers for PMTs on one RICH tile not updating

[Tyler Lemon]

Hello, 
After our email, DSG looked at the data available on EPICS for tile 66 with all PMTs stuck at zero. We saw that it is getting the correct low voltage and the PMTs are getting the correct high voltage. We also saw that the FPGA is reading out temperatures. From this, it leads DSG to believe that it is a problem with the hardware responsible for reading the PMT data. 


Hello Ben, 
>From your email below, it seems that there is an issue with communicating to the tile in question. What would cause a board to stop working correctly at seemingly random time? 

You mentioned that a power-cycle of LV and the DAQ should fix the issue. Has this been the procedure to fix a "frozen" tile that is not responding in the past? 

Best regards, 
Tyler 



From: "Benjamin Raydo" <braydo at jlab.org> 
To: "Valery Kubarovsky" <vpk at jlab.org>, tlemon at jlab.org 
Sent: Thursday, February 22, 2018 11:28:40 AM 
Subject: Re: [Dsg-rich] scalers for PMTs on one RICH tile not updating 

Hi Tyler and Valery, 

I just checked the tile status and see that the link is up to the unit, but it failed to identify the ASIC configuration (so the system doesn't know whether it is a 2 or 3 ASIC tile, so it puts 0) - you can see that in the status printout below for SSP slot 5 fiber 1 (#ASIC is 0, but ChUp is 1 saying a tile exists). This is observed from time-to-time and likely can be fixed by restarting the DAQ, but a power cycle of the low voltage for that tile and CODA restart may be necessary to fix this. This behavior is on the list of issues to clean up - please keep me informed if it becomes a serious problem for you all and I may be able to make it more of a priority. 

Thanks, 
Ben 


clasrun at clondaq6:clasrun> tcpClient rich4 'sspRich_PrintFiberStatus(5)' 
SSP RICH Fiber Status (slot=5): 
------------------------------------------------------------------------ 
Fiber Rst ChUp HrdErr FrmErr ErrCnt EvtEn Valid #ASIC FWRev 
------------------------------------------------------------------------ 
0 0 1 0 0 0 1 1 3 00000001 
1 0 1 0 0 0 1 1 0 00000001 
2 0 1 0 0 0 1 1 3 00000001 
3 0 1 0 0 0 1 1 3 00000001 
4 0 1 0 0 0 1 1 2 00000001 
5 0 1 0 0 0 1 1 3 00000001 
6 0 1 0 0 0 1 1 3 00000001 
7 0 1 0 0 0 1 1 3 00000001 
8 0 1 0 0 0 1 1 3 00000001 
9 0 1 0 0 0 1 1 3 00000001 
10 0 1 0 0 0 1 1 3 00000001 
11 0 1 0 0 0 1 1 3 00000001 
12 0 1 0 0 0 1 1 3 00000001 
13 0 1 0 0 0 1 1 3 00000001 
14 0 1 0 0 0 1 1 3 00000001 
15 0 0 0 0 0 0 0 0 00000000 
16 0 1 0 0 0 1 1 3 00000001 
17 0 1 0 0 0 1 1 3 00000001 
18 0 1 0 0 0 1 1 3 00000001 
19 0 1 0 0 0 1 1 2 00000001 
20 0 1 0 0 0 1 1 2 00000001 
21 0 1 0 0 0 1 1 3 00000001 
22 0 1 0 0 0 1 1 3 00000001 
23 0 1 0 0 0 1 1 3 00000001 
24 0 1 0 0 0 1 1 3 00000001 
25 0 1 0 0 0 1 1 3 00000001 
26 0 1 0 0 0 1 1 3 00000001 
27 0 1 0 0 0 1 1 3 00000001 
28 0 1 0 0 0 1 1 2 00000001 
29 0 1 0 0 0 1 1 3 00000001 
30 0 1 0 0 0 1 1 3 00000001 
31 0 1 0 0 0 1 1 3 00000001 


----- Original Message ----- 
From: "Valery Kubarovsky" <vpk at jlab.org> 
To: "Benjamin Raydo" <braydo at jlab.org> 
Sent: Thursday, February 22, 2018 8:59:15 AM 
Subject: Fwd: [Dsg-rich] scalers for PMTs on one RICH tile not updating 

Ben, 
Please reply to DSG group about the reasons why we have sometimes frozen scalers. 
Thanks, 
Valery 


Begin forwarded message: 

> From: Tyler Lemon <tlemon at jlab.org> 
> Date: February 22, 2018 at 08:22:50 EST 
> To: dsg-rich <dsg-rich at jlab.org> 
> Subject: [Dsg-rich] scalers for PMTs on one RICH tile not updating 
> 
> Hello, 
> 
> DSG noticed this morning that there is there is a group of three PMTs on the scaler map EPICS screen that are not updating. They have all been at zero for a few hours this morning. Attached is a screenshot with the three PMT scalers as zero. 
> 
> The PMTs in question are PMT 182, 183, and 184. They all look to be on the same electronic board (tile 66) that is read out in SSP slot 5, fiber 1. The temperatures for this board are still being read. 
> 
> Scalers for PMTs not updating for long periods until the IOC is rebooted or something else is done seems to be a common occurrence. 
> 
> What would be causing the scalers to stop updating while the temperatures are still read? 
> 
> Best regards, 
> Tyler 
> _______________________________________________ 
> Dsg-rich mailing list 
> Dsg-rich at jlab.org 
> https://mailman.jlab.org/mailman/listinfo/dsg-rich 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/dsg-hallb_rich/attachments/20180222/7ca8b564/attachment-0002.html>