[Halld-offline] REST Format Minor Changes, Versioning?

[Paul Mattione]

Thanks Richard.  I did the change the meaning of some variables (e.g. BCAL / track DOCA was being recorded as the total distance, and now I'm splitting it up into it's delta-z/phi components).  I'm traveling today, I won't be able to get a change in before we launch today's jobs.  So: 

1) For now, temporarily, old REST data (e.g. DC2) is unreadable with the current version of sim-recon-commissioning

2) Today's jobs will run with the new format. 

3) Before we launch our full skim jobs over the entire dataset in January, I'll modify the REST format so that it supports both old (pre-now) and new (full-skim-jobs and later) data. 

4) Once I make that change, the data generated over the next weeks (from now until we launch the full skim jobs) will no longer be readable.  

 - Paul

On Dec 19, 2014, at 11:16 AM, Richard Jones wrote:

> Hello Paul,
> 
> I suggest the following strategy for incremental evolution of the REST format.
> If all you are doing is adding new information without changing the meaning of the existing tags in the hddm data model (rest.hddm file), create a subtag to hold the new information, and set minOccurs="0" on that subtag. A subtag is a tag that sits inside the prior tag that it modifies, and the prior tag becomes the enclosing element. This way when you go to read old data files, the subtag will simply be absent, but the library will read it anyway.
> If you want to actually replace an old tag with a new way of representing the same information, or change the number or meaning of an existing tag, add a new tag with a different tag name. For example, you could change BCALshowers to BCALshowersRPhi or BCALshowers_v2. If you set minOccurs="0" on both the new and old tags, then the library will be able to read files with either type of tag. Of course the DEventSource implementation will have to support unpacking the information from either type of tag and populate the JANA object appropriately. If this is not possible because the JANA object now needs some of the new information then I suggest that you follow method (1) above, and add the new information as a subtag of the old one. If the new information is not available because the REST file is an old one, you have to decide how to configure the JANA object to fill in the missing pieces.
> Either way is fine with me, but I think that method 1 is the cleaner of the two. If some of the new information replaces what the original tag was supposed to convey, you can add a comment in the xml source to the effect that "this subtag when present overrides the 'x' and 'y' attributes of the parent tag."
> 
> -Richard Jones
> 
> On Fri, Dec 19, 2014 at 10:18 AM, Paul Mattione <pmatt at jlab.org> wrote:
> I've made some minor changes to the REST format:
> 
> 1) Breakup BCAL / Track DOCA into delta-phi and delta-z components (significantly different dependence)
> 2) Breakup TOF / Track DOCA into delta-x and delta-y components (can be significantly different (e.g. single-ended paddles))
> 3) Include TOF point "Status" (convolution of which bars were hit, and which ends had hits > threshold energy)
> 
> These changes are important for evaluating the quality of the hit / track matching, evaluating whether "neutral" showers might instead be parts of hadronic showers, and for matching tracks to TOF hits that don't have well-defined positions (e.g. hits in single-ended paddles that don't match to the other plane).
> 
> However, these changes break backwards compatibility.  So you'll no longer be able to analyze (e.g.) Data-Challenge-2 data with code versions from here on.  These changes needed to be done eventually though, and I thought it was important to do them now, before we create any official REST skims of the commissioning data.
> 
> Maybe we should start specifying/using different versioning for the REST files ... Richard, is this possible?  If you could give me some instructions and sample code early tomorrow, maybe I can implement this before we launch ...
> 
> - Paul
> 
> 
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20141219/2157a798/attachment-0002.html>