[Halld-offline] [EXTERNAL] CCDB interruption today

[Sean Dobbs]

Someone made a mistake, it is fixed now.

---Sean

On Wed, Feb 19, 2020 at 4:51 PM Beni Zihlmann <zihlmann at jlab.org> wrote:
>
> worked a minute ago now it does not anymore
>
> Error [1190]: in [MySQLDataProvider::GetAssignmentShort] Type table was not found: '/GEOMETRY/TRDGEM_HDDS.xml'
> JANA ERROR>> Failed to get XML from Calib DB for /GEOMETRY/TRDGEM_HDDS.xml! Geometry info not available!!
>
>
> On 2/19/20 4:02 PM, Mark Ito wrote:
>
> I should have said this explicitly: you can go ahead and start writing new constants now.
>
> On 2/19/20 3:45 PM, Mark Ito wrote:
>
> People,
>
> Marty has complete the restore from backup of the CCDB. Recall that any constants written between 3:45 am and a bit ago have been lost. This restoration appears to have propagated correctly to all replicas (e.g., hallddb-farm). Please let me know if you see problems using the CCDB from the MySQL servers.
>
> One note. There were a few active connections to the server when the restore was attempted. Those connections (about 20 of them) had to be terminated by hand. Most of these jobs probably went through without any bad effects, but some of them may have crashed and will have to be restarted.
>
>   -- Mark
>
> On 2/19/2020 3:09 PM, Mark Ito wrote:
>
> People,
>
> To make sure we restore all of the data from this incident we are rolling back to a backup of the CCDB database done at 3:45 am this morning. Please refrain from writing new constants to the CCDB until this is done (on the master server, hallddb.jlab.org). Anything written now will be lost. Marty Wise of the computer center is doing the restore and should be done in an hour or so.
>
>   -- Mark
>
> On 2/19/2020 2:44 PM, Sean Dobbs wrote:
>
> Hi Offliners,
>
> Due to the temporary disappearance of a CCDB table, jobs
> reconstructing raw data that accessed the master CCDB would have
> crashed from about 11 am - 2:30 pm today.  This should be fixed now.
> Please let me know if you see any other problems.
>
> Cheers,
> Sean
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline