[Hps-controls] clonioc1 problem

Mon Mar 2 15:09:45 EST 2015

iocsvtDaq* iocs are now deployed on clonioc4 under epics user. Here is the info right after startup. 

top - 15:07:38 up 96 days, 2:48, 3 users, load average: 0.06, 0.02, 0.00 
Tasks: 265 total, 1 running, 260 sleeping, 0 stopped, 4 zombie 
Cpu(s): 1.2%us, 0.1%sy, 0.0%ni, 98.4%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st 
Mem: 33233972k total, 561292k used, 32672680k free, 66236k buffers 
Swap: 66559888k total, 0k used, 66559888k free, 281924k cached 

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 
29517 epics 15 0 33308 12m 2944 S 7.3 0.0 0:00.73 svtDaqDpm7.cmd 
29509 epics 15 0 31812 9900 3020 S 1.0 0.0 0:00.27 st.cmd 
29513 epics 15 0 25084 4572 2952 S 0.7 0.0 0:00.08 svtDaqDpm3.cmd 
... 

Wesley 

> From: "Hansson Adrian" <phansson at slac.stanford.edu>
> To: "Hovanes Egiyan" <hovanes.egiyan at gmail.com>
> Cc: hps-controls at jlab.org
> Sent: Monday, March 2, 2015 3:05:15 PM
> Subject: Re: [Hps-controls] clonioc1 problem

> Ok. That helps. I should still fix my small mem leak.

> Sent from my phone

> On Mar 2, 2015, at 13:01, Hovanes Egiyan < hovanes.egiyan at gmail.com > wrote:

>> Wesley will move svt daq iocs to clonioc4
>> which has 32GB memory instead of
>> 4GB.

>> Sent via the Samsung Galaxy Note® 4, an AT&T 4G LTE smartphone

>> -------- Original message --------
>> From: "Hansson Adrian, Per Ola" < phansson at slac.stanford.edu >
>> Date: 03/02/2015 11:55 AM (GMT-05:00)
>> To: Serguei Boiarinov < boiarino at jlab.org >
>> Cc: hps-controls at jlab.org
>> Subject: Re: [Hps-controls] clonioc1 problem

>> No they are not supposed to do that.

>> Sent from my phone

>> > On Mar 2, 2015, at 09:15, Serguei Boiarinov < boiarino at jlab.org > wrote:

>> > Hi Pelle,
>> > your IOCs took all 4G memory on clonioc1, is it suppose to be ?
>> > Why it needs so much memory ? Machine is really slow now ...
>> > Sergey

>> > ----- Original Message -----
>> > From: "Hansson Adrian" < phansson at slac.stanford.edu >
>> > To: "Wesley Moore" < wmoore at jlab.org >
>> > Cc: "Serguei Boiarinov" < boiarino at jlab.org >, hps-controls at jlab.org
>> > Sent: Saturday, February 28, 2015 1:04:17 PM
>> > Subject: Re: [Hps-controls] clonioc1 problem

>> > Done. It took me a while to figure out that they were on clonioc2...

>>> On Feb 28, 2015, at 12:53 PM, Wesley Moore < wmoore at jlab.org <
>> > mailto:wmoore at jlab.org >> wrote:

>> > Pelle,

>> > Do me a favor... I'm out if town right now.

>> > Please launch the production iocs with:
>> > procServMgr start

>> > Wesley

>> > Sent from my Verizon Wireless 4G LTE smartphone

>> > -------- Original message --------
>> > From: Serguei Boiarinov < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>> > Date: 02/28/2015 12:16 PM (GMT-05:00)
>>> To: Hansson Adrian < phansson at slac.stanford.edu <
>> > mailto:phansson at slac.stanford.edu >>
>>> Cc: Wesley Moore < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>>> hps-controls at jlab.org < mailto:hps-controls at jlab.org >, Serguei Boiarinov <
>> > boiarino at jlab.org < mailto:boiarino at jlab.org >>
>> > Subject: Re: [Hps-controls] clonioc1 problem

>> > Then it should not be you. Please feel free to reboot it if necessary, right now
>> > clonioc's are used for HPS experiment only. Starting Monday CLAS12 ioc's will be
>> > operational as well.

>> > ----- Original Message -----
>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> > mailto:phansson at slac.stanford.edu >>
>> > To: "Serguei Boiarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>>> Cc: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>> > hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> > Sent: Saturday, February 28, 2015 12:12:52 PM
>> > Subject: Re: [Hps-controls] clonioc1 problem

>> > Nope I didn’t :)

>>>> On Feb 28, 2015, at 12:06 PM, Serguei Boiarinov < boiarino at jlab.org <
>> >> mailto:boiarino at jlab.org >> wrote:

>> >> Did you run 'stuff' as root ?

>> >> ----- Original Message -----
>>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >> mailto:phansson at slac.stanford.edu >>
>> >> To: "Serguei Boiarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>>>> Cc: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>> >> hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >> Sent: Saturday, February 28, 2015 11:08:23 AM
>> >> Subject: Re: [Hps-controls] clonioc1 problem

>>>> Hmmm, now clonioc2 has a problem. I was testing stuff so now I’m very suspicious
>> >> of my stuff crashing something.

>> >> Can I just reboot it?

>>>>> On Feb 28, 2015, at 9:17 AM, Serguei Boiarinov < boiarino at jlab.org <
>> >>> mailto:boiarino at jlab.org >> wrote:

>> >>> Thanks !
>> >>> Was it saying 'Press F1 ....' ?

>> >>> ----- Original Message -----
>>>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >>> mailto:phansson at slac.stanford.edu >>
>> >>> To: "Serguei Boiarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>>>>> Cc: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>> >>> hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >>> Sent: Saturday, February 28, 2015 9:06:40 AM
>> >>> Subject: Re: [Hps-controls] clonioc1 problem

>>>>> Yeah it was stupid. Stepan came in and said that I need to physically attach the
>> >>> keyboard and check that it’s not stuck. It was but it’s booting now.

>>>>>> On Feb 28, 2015, at 9:00 AM, Serguei Boiarinov < boiarino at jlab.org <
>> >>>> mailto:boiarino at jlab.org >> wrote:

>> >>>> If you have a time, please do it, maybe it is fixable.

>> >>>> ----- Original Message -----
>>>>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >>>> mailto:phansson at slac.stanford.edu >>
>> >>>> To: "Serguei Boiarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>>>>>> Cc: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>> >>>> hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >>>> Sent: Saturday, February 28, 2015 8:57:05 AM
>> >>>> Subject: Re: [Hps-controls] clonioc1 problem

>>>>>> Do you want me to plug in a screen and look? Or should I just let you guys do it
>> >>>> when you have time?

>>>>>>> On Feb 28, 2015, at 8:55 AM, Serguei Boiarinov < boiarino at jlab.org <
>> >>>>> mailto:boiarino at jlab.org >> wrote:

>>>>>>> Seems clonioc1 does not came up ? For that case we have clonioc2, clonioc3 etc
>> >>>>> as backups.

>> >>>>> ----- Original Message -----
>>>>>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >>>>> mailto:phansson at slac.stanford.edu >>
>> >>>>> To: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>
>>>>>>> Cc: "Serguei Boiarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>,
>> >>>>> hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >>>>> Sent: Saturday, February 28, 2015 8:52:36 AM
>> >>>>> Subject: Re: [Hps-controls] clonioc1 problem

>>>>>>> I’m playing with the daq IOCs on clonioc2 so don’t enable them in the procServ
>> >>>>> right now..

>>>>>>> I also have an issue doing “su epics”, it won’t recognize the password you sent.
>> >>>>> It worked yesterday.

>>>>>>>> On Feb 28, 2015, at 8:48 AM, Wesley Moore < wmoore at jlab.org <
>> >>>>>> mailto:wmoore at jlab.org >> wrote:

>>>>>>>> I'll check the softiocs when it comes up. If there's an issues with it (booting,
>> >>>>>> etc), I'll start up the iocs on clonioc2.

>> >>>>>> Wesley

>> >>>>>> ----- Original Message -----
>> >>>>>>> From: "Sergey Boyarinov" < boiarino at jlab.org < mailto:boiarino at jlab.org >>
>>>>>>>>> To: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >>>>>>> mailto:phansson at slac.stanford.edu >>
>>>>>>>>> Cc: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>,
>>>>>>>>> hps-controls at jlab.org < mailto:hps-controls at jlab.org >, "Sergey Boyarinov" <
>> >>>>>>> boiarino at jlab.org < mailto:boiarino at jlab.org >>
>> >>>>>>> Sent: Saturday, February 28, 2015 8:29:48 AM
>> >>>>>>> Subject: Re: [Hps-controls] clonioc1 problem

>> >>>>>>> Pelle,
>> >>>>>>> please restart clonioc1.
>> >>>>>>> Sergey

>> >>>>>>> ----- Original Message -----
>>>>>>>>> From: "Hansson Adrian" < phansson at slac.stanford.edu <
>> >>>>>>> mailto:phansson at slac.stanford.edu >>
>> >>>>>>> To: "Wesley Moore" < wmoore at jlab.org < mailto:wmoore at jlab.org >>
>> >>>>>>> Cc: hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >>>>>>> Sent: Saturday, February 28, 2015 8:13:17 AM
>> >>>>>>> Subject: Re: [Hps-controls] clonioc1 problem

>> >>>>>>> I’m in the control room right now so if you want me to reboot it I can. But I
>> >>>>>>> would think there is some remote restart possible on those racks?

>>>>>>>>> On Feb 28, 2015, at 8:11 AM, Wesley Moore < wmoore at jlab.org <
>> >>>>>>> mailto:wmoore at jlab.org > > wrote:

>> >>>>>>> My guess is, some network fallout from yesterday's issues. I'll try to take a
>> >>>>>>> look.

>> >>>>>>> Wesley

>> >>>>>>> Sent from my Verizon Wireless 4G LTE smartphone

>> >>>>>>> -------- Original message --------
>> >>>>>>> From: Nathan Baltzell < baltzell at jlab.org < mailto:baltzell at jlab.org > >
>> >>>>>>> Date: 02/28/2015 6:17 AM (GMT-05:00)
>> >>>>>>> To: hps-controls at jlab.org < mailto:hps-controls at jlab.org >
>> >>>>>>> Subject: [Hps-controls] clonioc1 problem

>> >>>>>>> Something changed regarding access/networking to clonioc1.
>> >>>>>>> This is the machine that was running all the softiocs for monitoring
>> >>>>>>> ECal HV, temperature, chiller.

>> >>>>>>> It does respond to a ping, but I cannot ssh to it, cagets timeout, and
>> >>>>>>> medm guis are no longer able to communicate with their IOCs on
>> >>>>>>> clonioc1. From EPICS strip charts, it looks like this happened around
>> >>>>>>> midnight.

>> >>>>>>> -Nathan
>> >>>>>>> _______________________________________________
>> >>>>>>> Hps-controls mailing list
>> >>>>>>> Hps-controls at jlab.org < mailto:Hps-controls at jlab.org >
>> >>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls
>> >>>>>>> _______________________________________________
>> >>>>>>> Hps-controls mailing list
>> >>>>>>> Hps-controls at jlab.org < mailto:Hps-controls at jlab.org >
>> >>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls

>> >>>>>>> _______________________________________________
>> >>>>>>> Hps-controls mailing list
>> >>>>>>> Hps-controls at jlab.org < mailto:Hps-controls at jlab.org >
>> >>>>>>> https://mailman.jlab.org/mailman/listinfo/hps-controls

>> _______________________________________________
>> Hps-controls mailing list
>> Hps-controls at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/hps-controls

> _______________________________________________
> Hps-controls mailing list
> Hps-controls at jlab.org
> https://mailman.jlab.org/mailman/listinfo/hps-controls
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.jlab.org/pipermail/hps-controls/attachments/20150302/9f102112/attachment-0001.html 