[Halld-online] auto-deletion of files on RAID

[David Lawrence]

Hi All,

  We are fast approaching the point where some files will need to be deleted from the RAID disks in order
to make room for new data. Initially, this can be done "by hand”, removing older runs. However, I’m working
on a script to auto-delete files based on some algorithm that takes into account both the file size and the last 
access time. This could potentially delete anything on the disk. It seems a lot of files that are not raw data files
are currently residing on the RAIDs. You can have a look at the summary below for gluonraid1 to get an idea
of the biggest ones. 

  So, we need to decide on a policy that will be implemented in the auto-deletion script. Specifically, if there
are certain directories that should not be touched. We can discuss this at the Online meeting on Wednesday,
but people should be thinking about it ahead of time so the discussion can be meaningful and efficient.


Biggest Users
---------------------
55.41 TB hdops
1.48 TB hdsys
1.25 TB hdcdcops
1.24 TB dalton
935.44 GB somov
454.54 GB hdbcalops
398.86 GB davidl
317.37 GB hdtofops
272.66 GB yqiang
218.42 GB zihlmann
 
Biggest Directories
---------------------
4.99 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run001998
2.88 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002141
2.68 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002129
1.64 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002126
1.61 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002127
1.50 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002123
1.38 TB /raid/Users/pxi
1.30 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run001852
1.29 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run001824
1.24 TB /raid/rawdata/volatile/RunPeriod-2014-10/rawdata/Run002016
 
Biggest files
---------------------
253.84 GB /raid/Users/hdcdcops/fdcdata/adcdata_testROCFDC10_ALLslots.dat
93.18 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1513.root
93.16 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1514.root
93.14 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1613.root
61.48 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/cosmic/bcal_1993.root
41.42 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1525.root
28.84 GB /raid/Users/hdcdcops/cdcdata/adcdata_ROCCDC4_999_ALLslots.dat
26.27 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1514_1.root
21.66 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1163.root
20.65 GB /raid/rawdata/active/RunPeriod000/BCAL/rootfiles/commish/bcal_1280.root
 
Scan Info.
---------------------
               user account = hdsys
                  scan host = gluonraid1-daq.jlab.org
                     topdir = /raid
            scan start time = Sun, 14 Dec 2014 10:37:47 +0000
directory update start time = Sun, 14 Dec 2014 11:37:24 +0000
    user summary start time = Sun, 14 Dec 2014 11:59:13 +0000
              scan end time = Sun, 14 Dec 2014 11:59:13 +0000
 


Regards,
-David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.jlab.org/pipermail/halld-online/attachments/20141215/d26b560c/attachment.html