[Hps-analysis] Vertex Chisq Probability Cut?

Thu Jan 31 14:46:47 EST 2019

Hi Miriam,

I think I put that cut, because I feel it is dangerous just looking into 
pass1 data, 0.00001 was corresponding to chi2/NDF = around 6,
and put 1.7e-16 corresponds to chi2/NDF=20 (if I am not wrong).

This was discussed in the analysis meetings, before we put this in the 
steering file.

Rafo

On 1/31/19 2:35 PM, Solt, Matthew Reagan wrote:
>
> Miriam,
>
>
> For some reason, it's set at 1.7e-16 in the steering file: 
> https://github.com/JeffersonLab/hps-java/blob/master/steering-files/src/main/resources/org/hps/steering/recon/PhysicsRun2016FullRecon.lcsim 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJeffersonLab%2Fhps-java%2Fblob%2Fmaster%2Fsteering-files%2Fsrc%2Fmain%2Fresources%2Forg%2Fhps%2Fsteering%2Frecon%2FPhysicsRun2016FullRecon.lcsim&data=02%7C01%7Crafopar%40jlab.org%7Cb18072903ac543c4cc8608d687b33d45%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636845601240365635&sdata=zowqSQvWApwhIPQYZmnOji9z3ZKVVRA7MXDZHQ%2FTZhw%3D&reserved=0>
>
>
> No idea why, but I think this value helped us a bit.
>
>
> Matt Solt
>
> ------------------------------------------------------------------------
> *From:* Diamond, Miriam <mdiamond at slac.stanford.edu>
> *Sent:* Thursday, January 31, 2019 11:22:26 AM
> *To:* Solt, Matthew Reagan; Graf, Norman A.; Graham, Mathew Thomas; 
> Maurik Holtrop
> *Cc:* hps-analysis at jlab.org
> *Subject:* Re: [Hps-analysis] Vertex Chisq Probability Cut?
> Wait, how did the cut value get to be 1.7e-16?  The default is 0.00001 
> (https://github.com/JeffersonLab/hps-java/blob/master/record-util/src/main/java/org/hps/record/StandardCuts.java#L144 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJeffersonLab%2Fhps-java%2Fblob%2Fmaster%2Frecord-util%2Fsrc%2Fmain%2Fjava%2Forg%2Fhps%2Frecord%2FStandardCuts.java%23L144&data=02%7C01%7Crafopar%40jlab.org%7Cb18072903ac543c4cc8608d687b33d45%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636845601240375640&sdata=A0Kcpgc8Tt9Gn9ZTY5q3CkUNOE%2F47dsywq46DW5ozjc%3D&reserved=0>)
> ------------------------------------------------------------------------
> *From:* Solt, Matthew Reagan <mrsolt at slac.stanford.edu>
> *Sent:* Thursday, January 31, 2019 10:49:36 AM
> *To:* Graf, Norman A.; Graham, Mathew Thomas; Maurik Holtrop
> *Cc:* Diamond, Miriam; hps-analysis at jlab.org
> *Subject:* Re: [Hps-analysis] Vertex Chisq Probability Cut?
>
> I think I agree with Norman here, that I don't really trust the 
> probabilities now even if it's a simple calculation. Has anyone 
> studied the value in detail? I'm not sure how much data this gets rid 
> of right now, my guess is not a lot. The cut itself is already pretty 
> loose at 1.7e-16, but I don't really know what vertices this gets rid 
> of. I would feel more comfortable if this cut were removed. But if 
> someone shows me that it has been studied and doesn't cut into 
> displaced signal much, I would be happy to keep it in.
>
>
> I do, however, like Maurik's change that that cut only should apply to 
> unconstrained so that every vertex is guaranteed one type of 
> constraint (whether we apply that cut now or in the future).
>
>
> Matt Solt
>
> ------------------------------------------------------------------------
> *From:* Graf, Norman A. <ngraf at slac.stanford.edu>
> *Sent:* Thursday, January 31, 2019 10:32:42 AM
> *To:* Solt, Matthew Reagan; Graham, Mathew Thomas; Maurik Holtrop
> *Cc:* Diamond, Miriam; hps-analysis at jlab.org
> *Subject:* Re: [Hps-analysis] Vertex Chisq Probability Cut?
>
> I, for one, don't trust the current chi-squared values for the 
> vertices (nor for the track. And to be honest I'm suspicious about the 
> hit uncertainties, too.) So unless it's a yuge number of vertices that 
> this cut is eliminating I would prefer to loosen/eliminate this cut 
> and try to understand what those vertices are.
>
>
> Norman
>
>
>
> ------------------------------------------------------------------------
> *From:* Solt, Matthew Reagan <mrsolt at slac.stanford.edu>
> *Sent:* Thursday, January 31, 2019 10:11 AM
> *To:* Graham, Mathew Thomas; Maurik Holtrop
> *Cc:* Diamond, Miriam; hps-analysis at jlab.org; Graf, Norman A.
> *Subject:* Re: [Hps-analysis] Vertex Chisq Probability Cut?
>
> Thanks Maurik and Miriam,
>
>
> I think only having the cut on unconstrained, and writing out other 
> collections based on that, is a viable solution. As long as we trust 
> the unconstrained chisq probabilities enough to do this, which I have 
> never looked at before yesterday. But, the question is do we?
>
>
> Matt Solt
>
> ------------------------------------------------------------------------
> *From:* Graham, Mathew Thomas <mgraham at slac.stanford.edu>
> *Sent:* Thursday, January 31, 2019 9:24:03 AM
> *To:* Maurik Holtrop
> *Cc:* Solt, Matthew Reagan; Diamond, Miriam; hps-analysis at jlab.org; 
> Graf, Norman A.
> *Subject:* Re: [Hps-analysis] Vertex Chisq Probability Cut?
> Great!  Thanks Maurik.
>
>> On Jan 31, 2019, at 10:20 AM, maurik <maurik at physics.unh.edu 
>> <mailto:maurik at physics.unh.edu>> wrote:
>>
>> Matt, Norman,
>>
>> I started iss398 branch and am testing the code right now. It seems 
>> to be passing the Integration tests.
>>
>> I will process an EVIO file locally and send it to Jlab or SLAC. Let 
>> me know where to put that at SLAC.
>> Should have it available at the end of the day for testing.
>>
>> Best,
>> Maurik
>>
>>> On Jan 31, 2019, at 12:06 PM, maurik <maurik at physics.unh.edu 
>>> <mailto:maurik at physics.unh.edu>> wrote:
>>>
>>> Thanks Miriam! That is really helpful.
>>>
>>> I am going to start an issue with a new branch, just to suggest how 
>>> we could modify this part of the code so the chisquared check is 
>>> ONLY on the UNCONSTRAINED vertexes. If it passes the unconstrained 
>>> vertex, then all versions of constrained are written out.
>>>
>>> - Maurik
>>>
>>>
>>>> On Jan 31, 2019, at 11:49 AM, Diamond, Miriam 
>>>> <mdiamond at slac.stanford.edu <mailto:mdiamond at slac.stanford.edu>> wrote:
>>>>
>>>> It is indeed part of the MOUSE cuts: MinVertexChisqProb. The cut 
>>>> value can be set via
>>>> https://github.com/JeffersonLab/hps-java/blob/master/recon/src/main/java/org/hps/recon/particle/HpsReconParticleDriver.java#L281 
>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJeffersonLab%2Fhps-java%2Fblob%2Fmaster%2Frecon%2Fsrc%2Fmain%2Fjava%2Forg%2Fhps%2Frecon%2Fparticle%2FHpsReconParticleDriver.java%23L281&data=02%7C01%7Crafopar%40jlab.org%7Cb18072903ac543c4cc8608d687b33d45%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636845601240385649&sdata=xNvdKxBG8zDXIzTcPXN8kREe6eTitJyUWX3%2FVVMgr5o%3D&reserved=0> .; 
>>>> you could essentially disable it by setting it to 0.
>>>>
>>>> The cut is applied here: 
>>>> https://github.com/JeffersonLab/hps-java/blob/master/recon/src/main/java/org/hps/recon/particle/HpsReconParticleDriver.java#L626 
>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJeffersonLab%2Fhps-java%2Fblob%2Fmaster%2Frecon%2Fsrc%2Fmain%2Fjava%2Forg%2Fhps%2Frecon%2Fparticle%2FHpsReconParticleDriver.java%23L626&data=02%7C01%7Crafopar%40jlab.org%7Cb18072903ac543c4cc8608d687b33d45%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636845601240395654&sdata=xnA3cEGf90O9dv7KCYorknQ6ZU8hOWhdmWf8DDFlHDM%3D&reserved=0> 
>>>>
>>>>
>>>> There are no grave consequences to removing this cut, you'll just 
>>>> get more junk vertices in the lcio.
>>>>
>>>> -- Miriam
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:*Solt, Matthew Reagan <mrsolt at slac.stanford.edu 
>>>> <mailto:mrsolt at slac.stanford.edu>>
>>>> *Sent:*Thursday, January 31, 2019 12:15:15 AM
>>>> *To:*hps-analysis at jlab.org <mailto:hps-analysis at jlab.org>
>>>> *Cc:*Diamond, Miriam; Nelson, Timothy Knight; Graham, Mathew 
>>>> Thomas; Graf, Norman A.
>>>> *Subject:*Vertex Chisq Probability Cut?
>>>> Hi all,
>>>>
>>>> Long email, executive summary: we should remove the vertex chisq 
>>>> probability cut in the reconstruction. Details below.
>>>>
>>>> There is a cut in the reconstruction on the vertex chisq 
>>>> probability, which I think is part of the MOUSE cuts, that is 
>>>> causing issues for displaced A's. Where has this cut been verified 
>>>> (specifically for displaced A's)? Plots are attached that show the 
>>>> following story.
>>>>
>>>> There is currently a cut at 1.7e-16 for the current reconstruction 
>>>> which seems fine, except the distributions for the beamspot 
>>>> constrained and unconstrained probabilities look crazy, with bsc 
>>>> having a significant number events at a probability of 0 and the 
>>>> unconstrained distributions have events closer to 1 (I didn't check 
>>>> target constrained). For the same vertex, the reconstruction can 
>>>> write an unconstrained vertex that passes the cut, but not a bsc 
>>>> that doesn't pass the cut (which is definitely not what we want). 
>>>> This particularly effects high z events, including clean signal 
>>>> events, as bsc probability is highly anti-correlated with z. This 
>>>> is actually how I just noticed it today, while looking at displaced 
>>>> A's which have very few reconstructed high z events. This effect is 
>>>> not noticeable in either data or MC because the it really only 
>>>> effects high z events (both good signal and background).
>>>>
>>>> What's worse is that apparently the tuple maker requires a vertex 
>>>> to have both a bsc and unconstrained collection, so if it doesn't 
>>>> have a bsc vertex it throws them both away even if the 
>>>> unconstrained passes the cut (not true in the lcio). I first 
>>>> noticed it in the displaced A' tuples (which have the default cut 
>>>> of 0.00001, so even worse than the data reconstruction).
>>>>
>>>> What affect does this have on current pass4? It doesn't effect the 
>>>> core distributions much at all, which is why I didn't catch it yet. 
>>>> It only really affects high z signal and background (which is 
>>>> obviously important). The unconstrained collection is probably ok 
>>>> in the lcio files, it's more just the beamspot constrained 
>>>> collection and the ntuples.
>>>>
>>>> So, why do the vertex chisq probability distributions look weird? I 
>>>> don't know, I haven't had time to dig that far. But since I don't 
>>>> think we understand this effect, and we are trying to compute our 
>>>> data ASAP, we should remove the cut in the reconstruction for now. 
>>>> If there are consequences to removing this cut, someone should 
>>>> speak up (Tim, Miriam, Matt, Norman?).
>>>>
>>>> I hope I was clear, but I'm sure some of you will have questions. 
>>>> Also, if someone gets a chance, can someone verify these 
>>>> probability plots? Thanks.
>>>>
>>>> Matt Solt
>>>> _______________________________________________
>>>> Hps-analysis mailing list
>>>> Hps-analysis at jlab.org <mailto:Hps-analysis at jlab.org>
>>>> https://mailman.jlab.org/mailman/listinfo/hps-analysis
>>>
>>> _______________________________________________
>>> Hps-analysis mailing list
>>> Hps-analysis at jlab.org <mailto:Hps-analysis at jlab.org>
>>> https://mailman.jlab.org/mailman/listinfo/hps-analysis
>>
>> _______________________________________________
>> Hps-analysis mailing list
>> Hps-analysis at jlab.org <mailto:Hps-analysis at jlab.org>
>> https://mailman.jlab.org/mailman/listinfo/hps-analysis
>
>
> _______________________________________________
> Hps-analysis mailing list
> Hps-analysis at jlab.org
> https://mailman.jlab.org/mailman/listinfo/hps-analysis

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/hps-analysis/attachments/20190131/b3b7ab02/attachment-0001.html>