[G14_run] Systematic uncertainties on asymmetries - (Tsuneo's question to K Sigma analysis note)
Nicholas Zachariou
nicholas at jlab.org
Wed Nov 8 16:20:44 EST 2017
Hi Andy,
Thank you for the explanation. I do agree that that would be the correct way to do so if we found a systematic error (and not uncertainty). In that case we would need to shift our points to account for the systematic offset. In most cases (and in the analysis we have done for the E observable on K+Sigma-) the systematic studies yield uncertainties (i.e. the systematic effect of specific sources can produce higher or lower in magnitude observables). The same goes for the photon polarization. The 3.4% is a systematic uncertainty and not an error. This means that the true value of the photon polarization can vary from the nominal value by 3.4% in either direction. This effects our observable E directly in magnitude, so I think its more proper to include that in the total uncertainty by adding it to the statistical in quadrature. Maybe I am missing something and would love to discuss more on this during the meeting.
Best regards,
Nick
> On 8 Nov 2017, at 17:35, Dr. A.M. Sandorfi <sandorfi at jlab.org> wrote:
>
> Nick,
>
> A systematic scale uncertainty represents a shift to the entire data set as a hole. For example, imagine you find out after publication that a polarization (beam or target) was to high by 5%. How do you correct that – you multiply every data point by 0.95. Essentially, that just changes the scale of the axes in a plot of the data. The point-to-point statistical fluctuations do not change. That is not the equivalent to increasing the size of the point-to-point fluctuations.
>
> Now there can be some components of the systematic uncertainty that change with kinematic bins, and these parts could be added in quadrature with statistical errors. But those do not usually dominate the total systematic error. For this reason, when it comes time to publish, point-to-point systematics and scale uncertainties should be clearly quoted separated – as we have done in our recent pi-p PRL.
>
> Andy
>
>
>
>
>
> On 11/7/17, 11:02 PM, "Nicholas Zachariou" <nicholas at jlab.org <x-msg://22/nicholas@jlab.org>> wrote:
>
>> Hi Andy,
>>
>> Can you elaborate why its not correct to add the systematic in quadrature with the statistical?
>> I am not sure I understand why this "uses data that dont represent the experiment". Maybe we can talk more during the g14 meeting.
>>
>> Nick
>> On Nov 7, 2017, at 22:22, "Dr. A.M. Sandorfi" <sandorfi at jlab.org <x-msg://22/sandorfi@jlab.org>> wrote:
>>> Hi Eugene,
>>>
>>> Yes, your point is well taken - for those data points where the asymmetry is
>>> very close to zero. One has two choices: we could either give an overall
>>> fractional (%) uncertainty, while quoting the absolute value as an exception
>>> when the asymmetry vanishes, or just quote only an absolute uncertainty for
>>> all points.
>>>
>>> The trouble comes at the subsequent stage when the data is used by the
>>> various PWA groups. All PWA groups have their fitting routines set up to
>>> float the scale of a data set while including a chi^2 penalty that is
>>> weighted by a fractional systematic error. If we give them a systematic
>>> uncertainty that is absolute, they will combine it in quadrature with the
>>> statistical error to create an inflated point by point uncertainty, and set
>>> the fitting scale to 1. I can guarantee that this will happen and it is a
>>> completely incorrect way to use the data that doesn't represent the
>>> experiment. So it is better to use the first approach - quote the systematic
>>> uncertainty as a fractional (%) error, while explicitly noting the absolute
>>> value of the systematic uncertainty for those asymmetry points with nearly
>>> zero value. The later qualifying statement will probably be ignored in PWA
>>> analyses, but at least most of the data will have been included properly.
>>>
>>> Andy
>>>
>>>
>>>
>>>
>>> On 11/4/17, 11:37 PM, "Eugene Pasyuk" <pasyuk at jlab.org <x-msg://22/pasyuk@jlab.org>> wrote:
>>>
>>>>
>>>> Any asymmetry can be anything between -1 and +1, 0 included. For any
>>>> observable which may be equal to 0 relative uncertainty does not make sense.
>>>> Only absolute uncertainty must be used.
>>>> The second term in Nick's equation is equal to 0 if ObservableValue is always
>>>> equal to 0 regardless of sigma_sys_relative. This is incorrect but good
>>>> illustration why one must not use relative uncertainty for asymmetries.
>>>>
>>>> -Eugene
>>>>
>>>> ----- Original Message -----
>>>>> From: "Tsuneo Kageya" <kageya at jlab.org <x-msg://22/kageya@jlab.org>>
>>>>> To: "Nicholas Zachariou" <nicholas at jlab.org <x-msg://22/nicholas@jlab.org>>
>>>>> Cc: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org>>
>>>>> Sent: Saturday, November 4, 2017 10:57:28 PM
>>>>> Subject: Re: [G14_run] G14_run Digest, Vol 74, Issue 3 (Tuneo's question to
>>>>> K Sigma analysis note)
>>>>
>>>>> Nick,
>>>>>
>>>>> thank you for the response.
>>>>> I would like to know why the absolute is more appropriate.
>>>>> I will look forward the statements.
>>>>>
>>>>> Regards, Tsuneo Kageya.
>>>>>
>>>>> ----- Original Message -----
>>>>> From: "Nicholas Zachariou" <nicholas at jlab.org <x-msg://22/nicholas@jlab.org>>
>>>>> To: "Tsuneo Kageya" <kageya at jlab.org <x-msg://22/kageya@jlab.org>>
>>>>> Cc: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org>>
>>>>> Sent: Saturday, November 4, 2017 3:23:46 PM
>>>>> Subject: Re: [G14_run] G14_run Digest, Vol 74, Issue 3 (Tuneo's question to
>>>>> K
>>>>> Sigma analysis note)
>>>>>
>>>>> Hi Tsuneo,
>>>>>
>>>>> The absolute error is propagated directly to the total uncertainty
>>>>> (sigma_tot=sqrt(sigma_sys_absolute^2+(ObservableValue*sigma_sys_relative)^2 +
>>>>> sigma_statistical^2).
>>>>>
>>>>> I can elaborate more if you like on why thats the case (why absolute are more
>>>>> appropriate in my case). I will include some statements in the note to
>>>>> reflect
>>>>> this.
>>>>>
>>>>> Let me know if you would like to discuss this more.
>>>>>
>>>>> Best regards,
>>>>> Nick
>>>>>
>>>>> On Nov 4, 2017, 19:12, at 19:12, Tsuneo Kageya <kageya at jlab.org <x-msg://22/kageya@jlab.org>> wrote:
>>>>>> Hi Nick,
>>>>>>
>>>>>> Sorry to be late to make a question.
>>>>>> I have a question about the systematic error calculations.
>>>>>>
>>>>>> At page 35, on the table 4, you calculated the total absolute
>>>>>> systematic error
>>>>>> to be 0.10. How this is reflected into the total relative systematic
>>>>>> error ?
>>>>>> On the pi-p analysis, I think we calculated the systematic errors from
>>>>>> cuts in
>>>>>> the similar way and they are combined to the other errors (target and
>>>>>> beam polarizations).
>>>>>> Is this number 0.10 means 10 % or 0.1 % ?
>>>>>>
>>>>>> I may misunderstand this issue. Please let me know.
>>>>>>
>>>>>> Regards, Tsuneo Kageya.
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "g14 run-request" <g14_run-request at jlab.org <x-msg://22/g14_run-request@jlab.org>>
>>>>>> To: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org>>
>>>>>> Sent: Saturday, November 4, 2017 12:00:03 PM
>>>>>> Subject: G14_run Digest, Vol 74, Issue 3
>>>>>>
>>>>>> Send G14_run mailing list submissions to
>>>>>> g14_run at jlab.org <x-msg://22/g14_run@jlab.org>
>>>>>>
>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>> g14_run-request at jlab.org <x-msg://22/g14_run-request@jlab.org>
>>>>>>
>>>>>> You can reach the person managing the list at
>>>>>> g14_run-owner at jlab.org <x-msg://22/g14_run-owner@jlab.org>
>>>>>>
>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>> than "Re: Contents of G14_run digest..."
>>>>>>
>>>>>>
>>>>>> Today's Topics:
>>>>>>
>>>>>> 1. Re: Updated Analysis Note (Reinhard Schumacher)
>>>>>> 2. Re: Updated Analysis Note (Nicholas Zachariou)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Message: 1
>>>>>> Date: Fri, 3 Nov 2017 22:19:24 -0400
>>>>>> From: Reinhard Schumacher <schumacher at cmu.edu <x-msg://22/schumacher@cmu.edu>>
>>>>>> To: g14_run at jlab.org <x-msg://22/g14_run@jlab.org>
>>>>>> Subject: Re: [G14_run] Updated Analysis Note
>>>>>> Message-ID: <a5780718-56ac-1dc0-44e8-6b991dbf849b at cmu.edu <x-msg://22/a5780718-56ac-1dc0-44e8-6b991dbf849b@cmu.edu>>
>>>>>> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>>>>>>
>>>>>> Hi Nick,
>>>>>>
>>>>>> Indeed, noticeably improved.?? I recommend that you put the horizontal
>>>>>> error bars on Figs.? 29 - 32, too.? They are just as important there
>>>>>> since the model curves can vary a lot across one bin.
>>>>>>
>>>>>> Reinhard
>>>>>>
>>>>>>
>>>>>> On 11/3/2017 7:13 PM, Nicholas Zachariou wrote:
>>>>>>> Dear all,
>>>>>>>
>>>>>>> I am attaching the updated note that incorporates and addresses all
>>>>>> comments made. I have noticed that I have previously forgotten to
>>>>>> include the systematic uncertainty associated with the
>>>>>> photon-selection, and is now estimated and included.
>>>>>>> I would like to thank again Shumacher for his time and valuable
>>>>>> insight, and everybody for the comments and feedback. If there is no
>>>>>> other comments, I will be submitting the note early next week.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Nick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> G14_run mailing list
>>>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Reinhard Schumacher Department of Physics, 5000 Forbes Ave.
>>>>>> Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.
>>>>>> phone: 412-268-5177 web: www-meg.phys.cmu.edu/~schumach <http://www-meg.phys.cmu.edu/~schumach <http://www-meg.phys.cmu.edu/~schumach>>
>>>>>>
>>>>>>
>>>>>> -------------- next part --------------
>>>>>> An HTML attachment was scrubbed...
>>>>>> URL:
>>>>>> <https://mailman.jlab.org/pipermail/g14_run/attachments/20171103/651ca313/at <https://mailman.jlab.org/pipermail/g14_run/attachments/20171103/651ca313/at>
>>>>>> tachment-0001.html>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Message: 2
>>>>>> Date: Sat, 04 Nov 2017 07:10:14 +0000
>>>>>> From: Nicholas Zachariou <nicholas at jlab.org <x-msg://22/nicholas@jlab.org>>
>>>>>> To: Reinhard Schumacher <schumacher at cmu.edu <x-msg://22/schumacher@cmu.edu>>
>>>>>> Cc: G14 Run <g14_run at jlab.org <x-msg://22/g14_run@jlab.org>>
>>>>>> Subject: Re: [G14_run] Updated Analysis Note
>>>>>> Message-ID: <d21ed42f-6b13-482d-8c8b-4ea087ad31e1 at jlab.org <x-msg://22/d21ed42f-6b13-482d-8c8b-4ea087ad31e1@jlab.org>>
>>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>>
>>>>>> Hi Reinhard,
>>>>>>
>>>>>> The figures looked a bit busy when I did that (too many lines) and
>>>>>> thats why i left the x-uncertainties out in those, but its takes me 2
>>>>>> minutes to incorborate them. I think that will be more relevant when we
>>>>>> decide exactly how to present our results in the publication.
>>>>>>
>>>>>> In the meantime I was wondering if its OK with the group to share our
>>>>>> preliminary results with the theorists and see if we can get any
>>>>>> insights from them.
>>>>>>
>>>>>> Best regards,
>>>>>> Nick
>>>>>>
>>>>>>
>>>>>> On Nov 4, 2017, 02:19, at 02:19, Reinhard Schumacher
>>>>>> <schumacher at cmu.edu <x-msg://22/schumacher@cmu.edu>> wrote:
>>>>>>> Hi Nick,
>>>>>>>
>>>>>>> Indeed, noticeably improved.?? I recommend that you put the horizontal
>>>>>>
>>>>>>> error bars on Figs.? 29 - 32, too.? They are just as important there
>>>>>>> since the model curves can vary a lot across one bin.
>>>>>>>
>>>>>>> Reinhard
>>>>>>>
>>>>>>>
>>>>>>> On 11/3/2017 7:13 PM, Nicholas Zachariou wrote:
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> I am attaching the updated note that incorporates and addresses all
>>>>>>> comments made. I have noticed that I have previously forgotten to
>>>>>>> include the systematic uncertainty associated with the
>>>>>>> photon-selection, and is now estimated and included.
>>>>>>>> I would like to thank again Shumacher for his time and valuable
>>>>>>> insight, and everybody for the comments and feedback. If there is no
>>>>>>> other comments, I will be submitting the note early next week.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Nick
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> G14_run mailing list
>>>>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Reinhard Schumacher Department of Physics, 5000 Forbes Ave.
>>>>>>> Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.
>>>>>>> phone: 412-268-5177 web: www-meg.phys.cmu.edu/~schumach <http://www-meg.phys.cmu.edu/~schumach <http://www-meg.phys.cmu.edu/~schumach>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> G14_run mailing list
>>>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>> -------------- next part --------------
>>>>>> An HTML attachment was scrubbed...
>>>>>> URL:
>>>>>> <https://mailman.jlab.org/pipermail/g14_run/attachments/20171104/14965baf/at <https://mailman.jlab.org/pipermail/g14_run/attachments/20171104/14965baf/at>
>>>>>> tachment-0001.html>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Subject: Digest Footer
>>>>>>
>>>>>>
>>>>>> G14_run mailing list
>>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> End of G14_run Digest, Vol 74, Issue 3
>>>>>> **************************************
>>>>>>
>>>>>> G14_run mailing list
>>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>>
>>>>> G14_run mailing list
>>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>>
>>>> G14_run mailing list
>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>>
>>>
>>>
>>> G14_run mailing list
>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>> https://mailman.jlab.org/mailman/listinfo/g14_run <https://mailman.jlab.org/mailman/listinfo/g14_run>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/g14_run/attachments/20171108/294f7fe8/attachment-0001.html>
More information about the G14_run
mailing list