[Halld-offline] [EXTERNAL] Re: Software versions and simulation issues

[Sean Dobbs]

Matt,  yes that's right.  The cleanest solution to have both the data and
MC depend on one set of constants, so that we didn't have to manage the
variations separately, would have been to introduce an entirely new table
and make both mcsmear and the reconstruction code use this new table.  But
I agree that this would be getting overly complicated for this number which
is unlikely to change in the future.

---Sean

On Thu, Sep 26, 2019 at 7:16 PM Shepherd, Matthew <mashephe at indiana.edu>
wrote:

>
>
> Right - this now has the feature that if someone adjusts the digi_scale
> meant to calibrate data one must also adjust MC/digi_scale or else the MC
> code won’t perform as desired.  So we’ve introduced a new burden and
> potential pitfall going forward.  Fortunately it seems unlikely this number
> will change significantly.
>
> The real problem was with the old code, which used the digi_scale
> parameter in two completely different ways:  the smearing code used it to
> reconcile the scale difference between true energy and Geant-reported
> energy and the data hit creation code used to scale FADC to energy.  By
> construction one and only one of these algorithms would run on either an MC
> or data hit.  The problem came when one wanted to implement features of the
> digitization, that depended on knowing the FADC scale, into the smearing,
> which was written to use the same constant in a different way.
>
> Rather than just solving the problem, we’ve introduced a new problem to
> make solving the problem less painful.  If just doing it right is really an
> impediment, then this seems like the best solution, especially since this
> constant is unlikely to change.
>
> Matt
>
>
> On Sep 26, 2019, at 1:40 PM, Sean Dobbs <seandobbs at gmail.com> wrote:
>
> Hi all,
>
> As discussed in the last meeting, the implementation of the FCAL smearing
> that I approved had a design flaw, and backwards compatibility should have
> been supported.  I checked in a change which was merged today that resolves
> this issue, along with a modification of CCDB.  Basically, the values for
> the mc variation of /FCAL/digi_scales were restored for backward
> compatibility, and appropriate values for the new smearing scheme were put
> in a new table,  /FCAL/MC/digi_scales. This is not as elegant as was hoped
> for, but properly backward compatible, and hopefully more clear what all of
> the MC factors mean.
>
> A new halld_sim version should be tagged, presumably with a version number
> of 4.8.0.
>
> What does this mean for users?  Here are some guidelines depending on
> which halld_sim version one is using (n.b. the latest version is also
> recommended, but not available in all cases):
> latest (4.8.0?):  the current CCDB constants are fine
> 4.6.0 and 4.7.0:  use a CCDB timestamp between 2019-07-27 and 2019-09-25
> 4.5.0 and earlier:  the current CCDB constants are fine
>
>
> Cheers,
> Sean
>
> On Tue, Sep 17, 2019 at 10:36 AM Sean Dobbs <sdobbs at fsu.edu> wrote:
>
>> Hi all,
>>
>> I've been working on tracking down some problems that were reported to me
>> about poor simulation yields which seem to have exposed some issues in how
>> we're dealing with the interaction of software and the CCDB in the current
>> frame work.  Thanks to all who helped in tracking this down.
>>
>> The issue started on July 26, when some modifications to the CCDB were
>> made to support changes made to mcsmear to improve the FCAL smearing.  A
>> new halld_sim version was rolled out at this point.  The CCDB changes were
>> not necessarily backward compatible to older halld_im versions, or at least
>> weren't checked - which is fine, that's how our system has been designed so
>> far - but the changes were not announced widely enough, which is a mea
>> culpa,
>>
>> For people who were generating simulations themselves, they are usually
>> using the latest CCDB and the straightforward fix is to use the latest
>> version of halld_sim (or a version >= 4.6.0)
>>
>> For the centralized submission page, the situation is more problematic.
>> The default for generating simulations for 2017 data is halld_sim 4.2.0,
>> which is outdated, although there are newer versions that are built.  For
>> other data sets, no up-to-date version of halld_sim exists.  Also, there is
>> no ability in the form to set a CCDB timestamp that would give a version of
>> constants that would work with these older constants.
>>
>> So, we should probably discuss this in some more detail in our software
>> meeting this afternoon, and see if there are ways we can improve our
>> defaults and communication to avoid such issues in the future.
>>
>> Cheers,
>> Sean
>>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20190927/0abbffae/attachment-0002.html>