[Halld-offline] [EXTERNAL] CCDB interruption today

[Mark Ito]

I should have said this explicitly: you can go ahead and start writing 
new constants now.

On 2/19/20 3:45 PM, Mark Ito wrote:
> People,
>
> Marty has complete the restore from backup of the CCDB. Recall that 
> any constants written between 3:45 am and a bit ago have been lost. 
> This restoration appears to have propagated correctly to all replicas 
> (e.g., hallddb-farm). Please let me know if you see problems using the 
> CCDB from the MySQL servers.
>
> One note. There were a few active connections to the server when the 
> restore was attempted. Those connections (about 20 of them) had to be 
> terminated by hand. Most of these jobs probably went through without 
> any bad effects, but some of them may have crashed and will have to be 
> restarted.
>
>   -- Mark
>
> On 2/19/2020 3:09 PM, Mark Ito wrote:
>> People,
>>
>> To make sure we restore all of the data from this incident we are 
>> rolling back to a backup of the CCDB database done at 3:45 am this 
>> morning. Please refrain from writing new constants to the CCDB until 
>> this is done (on the master server, hallddb.jlab.org). Anything 
>> written now will be lost. Marty Wise of the computer center is doing 
>> the restore and should be done in an hour or so.
>>
>>   -- Mark
>>
>> On 2/19/2020 2:44 PM, Sean Dobbs wrote:
>>> Hi Offliners,
>>>
>>> Due to the temporary disappearance of a CCDB table, jobs
>>> reconstructing raw data that accessed the master CCDB would have
>>> crashed from about 11 am - 2:30 pm today.  This should be fixed now.
>>> Please let me know if you see any other problems.
>>>
>>> Cheers,
>>> Sean
>>> _______________________________________________
>>> Halld-offline mailing list
>>> Halld-offline at jlab.org
>>> https://mailman.jlab.org/mailman/listinfo/halld-offline
>> _______________________________________________
>> Halld-offline mailing list
>> Halld-offline at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/halld-offline
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline