[Hps-controls] clonioc1 problem

Hansson Adrian, Per Ola phansson at slac.stanford.edu
Mon Mar 2 14:45:53 EST 2015


Ok. I logged in and killed them. 

Because of travel issues (stuck half-way) I won’t be able to work on this until tomorrow.

In  the next few days we can turn them on and off when we are actually running things to avoid causing problems. Will only be Omar, Sho, me and Ryan I think. 

/Pelle

On Mar 2, 2015, at 11:55 AM, Hansson Adrian, Per Ola <phansson at slac.stanford.edu> wrote:

> No they are not supposed to do that. 
> 
> Sent from my phone
> 
>> On Mar 2, 2015, at 09:15, Serguei Boiarinov <boiarino at jlab.org> wrote:
>> 
>> Hi Pelle,
>> your IOCs took all 4G memory on clonioc1, is it suppose to be ?
>> Why it needs so much memory ? Machine is really slow now ...
>> Sergey
>> 
>> ----- Original Message -----
>> From: "Hansson Adrian" <phansson at slac.stanford.edu>
>> To: "Wesley Moore" <wmoore at jlab.org>
>> Cc: "Serguei Boiarinov" <boiarino at jlab.org>, hps-controls at jlab.org
>> Sent: Saturday, February 28, 2015 1:04:17 PM
>> Subject: Re: [Hps-controls] clonioc1 problem
>> 
>> Done. It took me a while to figure out that they were on clonioc2...
>> 
>> 
>> On Feb 28, 2015, at 12:53 PM, Wesley Moore <wmoore at jlab.org<mailto:wmoore at jlab.org>> wrote:
>> 
>> Pelle,
>> 
>> Do me a favor... I'm out if town right now.
>> 
>> Please launch the production iocs with:
>> procServMgr start
>> 
>> Wesley
>> 
>> 
>> Sent from my Verizon Wireless 4G LTE smartphone
>> 
>> 
>> -------- Original message --------
>> From: Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>> Date: 02/28/2015 12:16 PM (GMT-05:00)
>> To: Hansson Adrian <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>> Cc: Wesley Moore <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>, Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>> Subject: Re: [Hps-controls] clonioc1 problem
>> 
>> 
>> Then it should not be you. Please feel free to reboot it if necessary, right now
>> clonioc's are used for HPS experiment only. Starting Monday CLAS12 ioc's will be
>> operational as well.
>> 
>> ----- Original Message -----
>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>> To: "Serguei Boiarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>> Cc: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>> Sent: Saturday, February 28, 2015 12:12:52 PM
>> Subject: Re: [Hps-controls] clonioc1 problem
>> 
>> Nope I didn’t :)
>> 
>> 
>>> On Feb 28, 2015, at 12:06 PM, Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>> wrote:
>>> 
>>> Did you run 'stuff' as root ?
>>> 
>>> ----- Original Message -----
>>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>> To: "Serguei Boiarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>>> Cc: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>> Sent: Saturday, February 28, 2015 11:08:23 AM
>>> Subject: Re: [Hps-controls] clonioc1 problem
>>> 
>>> Hmmm, now clonioc2 has a problem. I was testing stuff so now I’m very suspicious of my stuff crashing something.
>>> 
>>> Can I just reboot it?
>>> 
>>> 
>>>> On Feb 28, 2015, at 9:17 AM, Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>> wrote:
>>>> 
>>>> Thanks !
>>>> Was it saying 'Press F1 ....' ?
>>>> 
>>>> ----- Original Message -----
>>>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>>> To: "Serguei Boiarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>>>> Cc: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>>> Sent: Saturday, February 28, 2015 9:06:40 AM
>>>> Subject: Re: [Hps-controls] clonioc1 problem
>>>> 
>>>> Yeah it was stupid. Stepan came in and said that I need to physically attach the keyboard and check that it’s not stuck. It was but it’s booting now.
>>>> 
>>>>> On Feb 28, 2015, at 9:00 AM, Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>> wrote:
>>>>> 
>>>>> If you have a time, please do it, maybe it is fixable.
>>>>> 
>>>>> ----- Original Message -----
>>>>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>>>> To: "Serguei Boiarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>>>>> Cc: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>>>> Sent: Saturday, February 28, 2015 8:57:05 AM
>>>>> Subject: Re: [Hps-controls] clonioc1 problem
>>>>> 
>>>>> Do you want me to plug in a screen and look? Or should I just let you guys do it when you have time?
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Feb 28, 2015, at 8:55 AM, Serguei Boiarinov <boiarino at jlab.org<mailto:boiarino at jlab.org>> wrote:
>>>>>> 
>>>>>> Seems clonioc1 does not came up ? For that case we have clonioc2, clonioc3 etc as backups.
>>>>>> 
>>>>>> ----- Original Message -----
>>>>>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>>>>> To: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>
>>>>>> Cc: "Serguei Boiarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>>>>> Sent: Saturday, February 28, 2015 8:52:36 AM
>>>>>> Subject: Re: [Hps-controls] clonioc1 problem
>>>>>> 
>>>>>> I’m playing with the daq IOCs on clonioc2 so don’t enable them in the procServ right now..
>>>>>> 
>>>>>> I also have an issue doing “su epics”, it won’t recognize the password you sent. It worked yesterday.
>>>>>> 
>>>>>>> On Feb 28, 2015, at 8:48 AM, Wesley Moore <wmoore at jlab.org<mailto:wmoore at jlab.org>> wrote:
>>>>>>> 
>>>>>>> I'll check the softiocs when it comes up.  If there's an issues with it (booting, etc), I'll start up the iocs on clonioc2.
>>>>>>> 
>>>>>>> Wesley
>>>>>>> 
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Sergey Boyarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>>>>>>>> To: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>>>>>>> Cc: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>, hps-controls at jlab.org<mailto:hps-controls at jlab.org>, "Sergey Boyarinov" <boiarino at jlab.org<mailto:boiarino at jlab.org>>
>>>>>>>> Sent: Saturday, February 28, 2015 8:29:48 AM
>>>>>>>> Subject: Re: [Hps-controls] clonioc1 problem
>>>>>>> 
>>>>>>>> Pelle,
>>>>>>>> please restart clonioc1.
>>>>>>>> Sergey
>>>>>>>> 
>>>>>>>> ----- Original Message -----
>>>>>>>> From: "Hansson Adrian" <phansson at slac.stanford.edu<mailto:phansson at slac.stanford.edu>>
>>>>>>>> To: "Wesley Moore" <wmoore at jlab.org<mailto:wmoore at jlab.org>>
>>>>>>>> Cc: hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>>>>>>> Sent: Saturday, February 28, 2015 8:13:17 AM
>>>>>>>> Subject: Re: [Hps-controls] clonioc1 problem
>>>>>>>> 
>>>>>>>> I’m in the control room right now so if you want me to reboot it I can. But I
>>>>>>>> would think there is some remote restart possible on those racks?
>>>>>>>> 
>>>>>>>> On Feb 28, 2015, at 8:11 AM, Wesley Moore < wmoore at jlab.org<mailto:wmoore at jlab.org> > wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> My guess is, some network fallout from yesterday's issues. I'll try to take a
>>>>>>>> look.
>>>>>>>> 
>>>>>>>> Wesley
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Sent from my Verizon Wireless 4G LTE smartphone
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -------- Original message --------
>>>>>>>> From: Nathan Baltzell < baltzell at jlab.org<mailto:baltzell at jlab.org> >
>>>>>>>> Date: 02/28/2015 6:17 AM (GMT-05:00)
>>>>>>>> To: hps-controls at jlab.org<mailto:hps-controls at jlab.org>
>>>>>>>> Subject: [Hps-controls] clonioc1 problem
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Something changed regarding access/networking to clonioc1.
>>>>>>>> This is the machine that was running all the softiocs for monitoring
>>>>>>>> ECal HV, temperature, chiller.
>>>>>>>> 
>>>>>>>> It does respond to a ping, but I cannot ssh to it, cagets timeout, and
>>>>>>>> medm guis are no longer able to communicate with their IOCs on
>>>>>>>> clonioc1. From EPICS strip charts, it looks like this happened around
>>>>>>>> midnight.
>>>>>>>> 
>>>>>>>> -Nathan
>>>>>>>> _______________________________________________
>>>>>>>> Hps-controls mailing list
>>>>>>>> Hps-controls at jlab.org<mailto:Hps-controls at jlab.org>
>>>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls
>>>>>>>> _______________________________________________
>>>>>>>> Hps-controls mailing list
>>>>>>>> Hps-controls at jlab.org<mailto:Hps-controls at jlab.org>
>>>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> Hps-controls mailing list
>>>>>>>> Hps-controls at jlab.org<mailto:Hps-controls at jlab.org>
>>>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls
> 
> _______________________________________________
> Hps-controls mailing list
> Hps-controls at jlab.org
> https://mailman.jlab.org/mailman/listinfo/hps-controls




More information about the Hps-controls mailing list