[Halld-offline] halld_recon versioning

[Sean Dobbs]

Hi all,

I agree that Thomas is correct about the long term solution being a
split off of the analysis code that is upstream of producing REST
files, so that we can independently track its progression.  Splitting
sim from recon took a fair amount of time and effort, so we should
start planning for this soon.

We should think about a good short term solution as well.  hd_root is
statically linked to these libraries, so I think we should use
separate software stacks for both activities and not mix plugins.

It might be a good idea to schedule a meeting after the Collaboration
Meeting to come up with a consistent plan and make sure we are
correctly capturing and storing the details of all of the data we are
producing.

In regard to J/psi, the trees for this analysis were produced shortly
after the reconstructed data was available, and the same software
stack was used for the simulated data, so it was not affected by the
divergence in the analysis libraries which has only come up in the
past month or two.

Cheers,
Sean


On Thu, May 2, 2019 at 11:47 AM Shepherd, Matthew <mashephe at indiana.edu> wrote:
>
>
> Hi all,
>
> I agree this is a major issue that we should try to resolve ASAP.  It seems hard to do analysis if we can't easily generate signal MC and process it in the same way that we did the data.
>
> (It also makes me want to ask how the J/psi folks ensured their MC matched the data, as they use kinematic fitting.)
>
> Care should be taken with Thomas' proposal to check linking issues.  Some symbols are resolved dynamically with plugins, but our default link (IIRC) is static for executables.  If we are going to put newer dynamically linked plugins into a different version of hd_root we have to be sure there is not a conflict in symbol resolution.
>
> Matt
>
>
> > On May 2, 2019, at 8:48 AM, Thomas Britton <tbritton at jlab.org> wrote:
> >
> > I should have said “halld_ana” as a name for the new packaged. And yes. If the changes to the analysis libraries (kinfitter included) only effect the plugins that use them (reaction filter) then the short term patch will be much easier. Also it would be easier to split out in this case too.
> >
> > Thomas Britton
> >
> > On May 2, 2019, at 8:43 AM, Beni Zihlmann <zihlmann at jlab.org> wrote:
> >
> >> Hi Thomas,
> >> I do not understand you statements.
> >> hd_root and hd_anal are based on the same halld_recon code. There is no difference other than hd_root will create root trees.
> >> However the when running over REST files the kin fitter code is in halld_recon too and has to be  tracked.
> >> are you suggesting to break that part out of halld_recon?
> >>
> >> cheers,
> >> Beni
> >>
> >>
> >>
> >>
> >>
> >> On 5/2/19 8:30 AM, Thomas Britton wrote:
> >>> The correct long-term solution is to break the analysis things out into its own repository as was done with sim-recon. Remember, sim-recon was split because the reconstruction was really an anchor software set while sim could be rapidly iterated. The same thing applies here. Fortunately this split should be easier in principle as we know how to and The ana bits are less intertwined with the recon bits (although halld_ana would still be dependent on recon).  Moving forward, on the short term, the version sets used in each analysis launch should be recorded and saved in its entirety. MCwrapper could use the second xml or provided reaction filter. Is hd_root un modified if just the analysis bits change? If this is true the Jana configuration file would just need to change. David thinks that if you give a path to a plug-in in the Jana configuration file it will work.
> >>>
> >>> So I propose to split halld_recon to truly leave an anchoring software (split the workflows up). And if the changes really on effects reaction filter (and other plug-ins) then we record the software used in the analysis launches more rigorously and modify the Jana config file to use the specific plugin version appropriate to produce the trees.
> >>>
> >>> Thomas Britton
> >>>
> >>> On May 2, 2019, at 8:11 AM, Peter Pauli <ppauli at jlab.org> wrote:
> >>>
> >>>> Hi everybody,
> >>>>
> >>>> as Thomas already mentioned in the Analysis and Production meeting yesterday there is a problem the way most people probably use MCwrapper to produce their MC. The version.xml that can be specified contains a halld_recon version that should be chosen to match the halld_recon at the time of data production (for the latest reconstruction of spring17 that would be recon-2017_01-ver03_hdr as e.g. in version_recon-2017_01-ver03_8.xml). So the problem is that the analysis launches that produced the trees afterwards used more recent halld_recon builds with sometimes substantial changes to the analysis libraries. That means that if one produces MC with version_recon-2017_01-ver03_8.xml the reconstruction matches what was used for real data but the trees produced by ReactionFilter will not match what was produced in a subsequent analysis launch.
> >>>> If this is a big or a small problem is very much channel dependent but surely we all agree that we want to be able to produce our MC such that it matches the trees that we analyse and not just the REST files. A quick fix to this problem is that each individual user needs to check the version.xml used for the analysis launch in question and rerun over the REST files produced by MCwrapper. This should yield the correct trees. Although this will work I don’t think it is a good solution in the long run and we should come up with something better in this working group.
> >>>>
> >>>> Cheers,
> >>>> Peter
> >>>> _______________________________________________
> >>>> Halld-offline mailing list
> >>>> Halld-offline at jlab.org
> >>>> https://mailman.jlab.org/mailman/listinfo/halld-offline
> >>>
> >>>
> >>> _______________________________________________
> >>> Halld-offline mailing list
> >>>
> >>> Halld-offline at jlab.org
> >>> https://mailman.jlab.org/mailman/listinfo/halld-offline
> >>
> >> _______________________________________________
> >> Halld-offline mailing list
> >> Halld-offline at jlab.org
> >> https://mailman.jlab.org/mailman/listinfo/halld-offline
> > _______________________________________________
> > Halld-offline mailing list
> > Halld-offline at jlab.org
> > https://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline