[Dsg-halla_ecal] heater controls

[Donald Jones]

Hi Marc,
Thanks. Many of my requests were long term wants not short term needs. What I need in the short term to continue testing is a solution to the crashing. It's not a safety issue since the independent process controller is always active but it does make it difficult to demonstrate stable performance. I have requested that the device be moved to the hall next week and I was hoping to have a few days of successful data stable at the desired set temperatures before then. We have already shown the system can get there but haven't demonstrated long term performance. Do you have any ideas? Is there a remote way to reset the program?
-Don

Donald Jones
Hall A/C Staff Scientist
Jefferson Lab
Newport News, VA
________________________________
From: Dsg-halla_ecal <dsg-halla_ecal-bounces at jlab.org> on behalf of Marc Mcmullen via Dsg-halla_ecal <dsg-halla_ecal at jlab.org>
Sent: Monday, August 21, 2023 12:02 PM
To: dsg-halla_ecal at jlab.org <dsg-halla_ecal at jlab.org>
Subject: Re: [Dsg-halla_ecal] heater controls

Hi Don,

We have reviewed your findings and are optimizing to the controls software. I'll send an update later this week with the software changes.

Regards,
Marc
________________________________
From: Donald Jones <jonesdc at jlab.org>
Sent: Monday, August 21, 2023 12:24 AM
To: Marc Mcmullen <mcmullen at jlab.org>; Brian Eng <beng at jlab.org>
Cc: Jimmy Caylor <jcaylor at jlab.org>
Subject: heater controls

Hi Marc, Brian,
I have been running the heater system since Thursday afternoon when we were given the go ahead again. I'd like to discuss a few things with you but since I have to report for jury duty tomorrow, that may have to wait. Instead I will start the conversation here.
A few things I have noticed:

  *   There is some weird behavior in the controls where if I change the setpoint on one zone it immediately affects the output of other zones on the same PID. I would not describe this as happening every time, but quite often. For example, if I start by setting all three SM zones to the same temperature and then suddenly change one zone to a much higher temperature value than the others, often within a second or two, the other two zones will relax to very low output. This is well before any real temperature increase could be felt by the other zones. It feels like there is cross-talk somehow between channels. I have also seen when I reset the value of one zone and leave the other two unchanged, that the other two just stop putting out any power. I am imagining a loop with a shift register to cycle back the set values from the end of the previous loop to the start of the next and with one being changed, sometimes the other two are resetting to 0 or something like this.
  *   The system has crashed 3 times since Thursday, The first time happened Friday when I was sitting at the terminal. The second time happened while I was absent on Saturday and the crystals had cooled down by 80 degrees before I came back to check and found the issue. The last time it happened was tonight Sunday and when I came to check I found the system unresponsive and ~30 degrees hotter than the setpoints. So it's good to know it can freeze and go either way (hotter or colder). Also, the only way I can restart it is by physically pushing the reset switch, which isn't going to work in the hall. I have been conservatively running the Omega setpoint much lower (250 degC) in case of a malfunction and it was at 246 tonight, so not all that hot, but more than the usual 220-230.

I shut it down since the crashes are too frequent. We have to find a solution to the frequent crashes. Perhaps you already have ideas.
In addition to these issues, I have a few suggestions that I think would improve the program.

  *   Separate PID for each zone. We need more flexibility.
  *   Make output of PID go from 0 to 100 and then set the output voltage to the square root of the PID output so that the output voltage scales with heater power not heater voltage.
  *   We should average the PID output over many seconds before updating. Right now it updates every second and sometimes there are large swings from one second to the next. I suspect that these power supplies aren't built to manage continuous adjustment thousands of times per hour with potentially large swings in current. I would think something like a 30 second update would be sufficient but we could make the number of seconds a control for more flexibility. I think the graphs and readbacks should still update every second, but the voltage output should not.
  *   The same applies to the software controlled relay. I have seen it turn on and off several times very quickly when it is close to the set threshold. Averaging the deltaT for 30 seconds before turning it back on or off makes sense.
  *   It would be nice to have the PID voltage output on separate graph from the temps

Let me know what you think. There will likely be more suggestions and we can discuss the ones I bring up here at a convenient time.
-Don

Donald Jones
Hall A/C Staff Scientist
Jefferson Lab
Newport News, VA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/dsg-halla_ecal/attachments/20230821/5f3526f0/attachment-0001.html>