[Hallcsw] Enabled CODA file cleanup after move to tape on Hall C cluster
Brad Sawatzky
brads at jlab.org
Mon Sep 25 16:40:09 EDT 2017
I have turned on the clean-up part of the CODA data mover script. CODA
files will now be removed from the cdaq cluster file system
(/net/cdaq/cdaql3data/coda/data/raw/) after they are verified to be on
tape.
DO NOT START COPYING CODA FILES TO PERSONAL DIRECTORIES TO "FIX" THIS.
SUCH FILES WILL BE DELETED WITHOUT WARNING IF THEY ARE ON A SHARED SYSTEM.
The current system makes the files available at
/cache/mss/hallc/spring17/raw/
immediately after storing them on tape. Since there isn't a huge amount of
pressure on the /cache system, they are staying there for a long time (ie. I
don't think any of our files have been flushed off since last Fall...)
Modifications have been made to hcana (really podd) and the hallc_replay
scripts that allow the analyzer to look in multiple locations for a
filename. With proper setup, this will make the actual location of the fi
le transparent to the analysis (you don't have to monitor whether the file
is 'local' or on the tape system -- it will "just work").
If you are working on the Hall C cluster, then you should do the
following:
* Update any personal versions of hcana and hallc_replay scripts from
github's devel branches (follow the wiki instructions for extra
details). For example, to update hcana do something like this:
% cd hcana
% git checkout devel # if you're not already on that branch
% git submodule update
% scons
* Make a new symlink to the cache location in your replay directory
(keep the existing symlink 'raw'). For example:
% cd hallc_replay
% ln -s /cache/mss/hallc/spring17/raw cache
If you have personal scripts (that are not part of the updated
hallc_replay repo), then you will want to do something like this where y!
ou open your CODA files in your script:
const char* RunFileNamePattern = "shms_all_%05d.dat";
vector<TString> pathList;
pathList.push_back(".");
pathList.push_back("./raw");
pathList.push_back("./cache");
THaRun* run = new THaRun( pathList, Form(RunFileNamePattern, RunNumber)
);
If you're lost, talk to an expert. Do NOT copy files your your local
directory.
-- Brad
[PS: Don't copy files into your local directory.]
--- This is a plain text email for clients that cannot display HTML. The
full logentry can be found online at https://logbooks.jlab.org/entry/3485641
--
Brad Sawatzky, PhD <brads at jlab.org> -<>- Jefferson Lab / Hall C / C111
Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page at jlab.org
The most exciting phrase to hear in science, the one that heralds new
discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov
More information about the Hallcsw
mailing list