[G14_run] Systematic uncertainties on asymmetries - (Tsuneo's question to K Sigma analysis note)

Dr. A.M. Sandorfi sandorfi at jlab.org
Wed Nov 8 16:03:11 EST 2017 

Nick,

Pardon my cavalier interchange of the words ³error² and ³uncertainty². But
that¹s not the point. There is a systematic uncertainty in the beam
polarization, for example, and its variation from the nominal value is
approximated by a gaussian distribution with a 1 sigma of 3.4%. So, someone
else using the data set in some calculation is allowed to consider different
values of that polarization within this distribution. BUT, if an overall fit
to some model, a PWA, ...whatever, prefers a value for the beam polarization
that is different from our quoted nominal, he/she must shift ALL of our data
points together by exactly the same factor, because that¹s how the
measurement responds to a change to that parameter. {For example, If the
assumed beam polarization is lowered by X%, then the deduced asymmetry
values in kinematic bins at W=1800 and cosA= -0.3, and at W=2100 and cosA=
+0.6,  go up by exactly the same X%.} That¹s the point! For this reason one
does not combine uncertainties in the overall scale of an entire data set in
quadrature with point-to-point uncertainties.

Andy





On 11/8/17, 5:20 PM, "Nicholas Zachariou" <nicholas at jlab.org> wrote:

> Hi Andy, 
> 
> Thank you for the explanation. I do agree that that would be the correct way
> to do so if we found a systematic error (and not uncertainty). In that case we
> would need to shift our points to account for the systematic offset. In most
> cases (and in the analysis we have done for the E observable on K+Sigma-) the
> systematic studies yield uncertainties (i.e. the systematic effect of specific
> sources can produce higher or lower in magnitude observables). The same goes
> for the photon polarization. The 3.4% is a systematic uncertainty and not an
> error. This means that the true value of the photon polarization can vary from
> the nominal value by 3.4% in either direction. This effects our observable E
> directly in magnitude, so I think its more proper to include that in the total
> uncertainty by adding it to the statistical in quadrature. Maybe I am missing
> something and would love to discuss more on this during the meeting.
> 
> Best regards, 
> Nick
> 
> 
>> On 8 Nov 2017, at 17:35, Dr. A.M. Sandorfi <sandorfi at jlab.org> wrote:
>> 
>> Re: [G14_run] Systematic uncertainties on asymmetries - (Tsuneo's question to
>> K Sigma analysis note)
>> Nick,
>> 
>> A systematic scale uncertainty represents a shift to the entire data set as a
>> hole. For example, imagine you find out after publication that a polarization
>> (beam or target) was to high by 5%. How do you correct that ­ you multiply
>> every data point by 0.95. Essentially, that just changes the scale of the
>> axes in a plot of the data. The point-to-point statistical fluctuations do
>> not change. That is not the equivalent to increasing the size of the
>> point-to-point fluctuations.
>> 
>> Now there can be some components of the systematic uncertainty that change
>> with kinematic bins, and these parts could be added in quadrature with
>> statistical errors. But those do not usually dominate the total systematic
>> error. For this reason, when it comes time to publish, point-to-point
>> systematics and scale uncertainties should be clearly quoted separated ­ as
>> we have done in our recent pi-p PRL.
>> 
>> Andy
>> 
>> 
>> 
>> 
>> 
>> On 11/7/17, 11:02 PM, "Nicholas Zachariou" <nicholas at jlab.org
>> <x-msg://22/nicholas@jlab.org> > wrote:
>> 
>>> Hi Andy,
>>> 
>>> Can you elaborate why its not correct to add the systematic in quadrature
>>> with the statistical?
>>> I am not sure I understand why this "uses data that dont represent the
>>> experiment". Maybe we can talk more during the g14 meeting.
>>> 
>>> Nick
>>> On Nov 7, 2017, at 22:22, "Dr. A.M.  Sandorfi" <sandorfi at jlab.org
>>> <x-msg://22/sandorfi@jlab.org> > wrote:
>>>> Hi Eugene,
>>>> 
>>>> Yes, your point is well taken - for those data points where the asymmetry
>>>> is
>>>> very close to zero. One has two choices: we could either give an overall
>>>> fractional (%) uncertainty, while quoting the absolute value as an
>>>> exception
>>>> when the asymmetry vanishes, or just quote only an absolute uncertainty for
>>>> all points. 
>>>> 
>>>> The trouble comes at the subsequent stage when the data is used by the
>>>> various PWA groups. All PWA groups have their fitting routines set up to
>>>> float the scale of a data set while including a chi^2 penalty that is
>>>> weighted by a fractional systematic error. If we give them a systematic
>>>> uncertainty that is absolute, they will combine it in quadrature with the
>>>> statistical error to create an inflated point by point uncertainty, and set
>>>> the fitting scale to 1. I can guarantee that this will happen and it is a
>>>> completely incorrect way to use the data that doesn't represent the
>>>> experiment. So it is better to use the first approach - quote the
>>>> systematic
>>>> uncertainty as a fractional (%) error, while explicitly noting the absolute
>>>> value of the systematic uncertainty for those asymmetry points with nearly
>>>> zero value. The later qualifying statement will probably be ignored in PWA
>>>> analyses, but at least most of the data will have been included properly.
>>>> 
>>>> Andy
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 11/4/17, 11:37 PM, "Eugene Pasyuk" <pasyuk at jlab.org
>>>> <x-msg://22/pasyuk@jlab.org> > wrote:
>>>> 
>>>>>  
>>>>>  Any asymmetry can be anything between -1 and +1, 0 included. For any
>>>>>  observable which may be equal to 0 relative uncertainty does not make
>>>>> sense.
>>>>>  Only absolute uncertainty must be used.
>>>>>  The second term in Nick's equation is equal to 0 if ObservableValue is
>>>>> always
>>>>>  equal to 0 regardless of sigma_sys_relative. This is incorrect but good
>>>>>  illustration why one must not use relative uncertainty for asymmetries.
>>>>>  
>>>>>  -Eugene
>>>>>  
>>>>>  ----- Original Message -----
>>>>>>  From: "Tsuneo Kageya" <kageya at jlab.org <x-msg://22/kageya@jlab.org> >
>>>>>>  To: "Nicholas Zachariou" <nicholas at jlab.org
>>>>>> <x-msg://22/nicholas@jlab.org> >
>>>>>>  Cc: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org> >
>>>>>>  Sent: Saturday, November 4, 2017 10:57:28 PM
>>>>>>  Subject: Re: [G14_run] G14_run Digest, Vol 74, Issue 3 (Tuneo's
>>>>>> question to
>>>>>>  K Sigma analysis note)
>>>>>  
>>>>>>  Nick,
>>>>>>  
>>>>>>  thank you for the response.
>>>>>>  I would like to know why the absolute is more appropriate.
>>>>>>  I will look forward the statements.
>>>>>>  
>>>>>>      Regards, Tsuneo Kageya.
>>>>>>  
>>>>>>  ----- Original Message -----
>>>>>>  From: "Nicholas Zachariou" <nicholas at jlab.org
>>>>>> <x-msg://22/nicholas@jlab.org> >
>>>>>>  To: "Tsuneo Kageya" <kageya at jlab.org <x-msg://22/kageya@jlab.org> >
>>>>>>  Cc: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org> >
>>>>>>  Sent: Saturday, November 4, 2017 3:23:46 PM
>>>>>>  Subject: Re: [G14_run] G14_run Digest, Vol 74, Issue 3 (Tuneo's
>>>>>> question to
>>>>>>  K
>>>>>>  Sigma analysis note)
>>>>>>  
>>>>>>  Hi Tsuneo,
>>>>>>  
>>>>>>  The absolute error is propagated directly to the total uncertainty
>>>>>>  
>>>>>> (sigma_tot=sqrt(sigma_sys_absolute^2+(ObservableValue*sigma_sys_relative)
>>>>>> ^2 +
>>>>>>  sigma_statistical^2).
>>>>>>  
>>>>>>  I can elaborate more if you like on why thats the case (why absolute are
>>>>>> more
>>>>>>  appropriate in my case). I will include some statements in the note to
>>>>>>  reflect
>>>>>>  this.
>>>>>>  
>>>>>>  Let me know if you would like to discuss this more.
>>>>>>  
>>>>>>  Best regards,
>>>>>>  Nick
>>>>>>  
>>>>>>  On Nov 4, 2017, 19:12, at 19:12, Tsuneo Kageya <kageya at jlab.org
>>>>>> <x-msg://22/kageya@jlab.org> > wrote:
>>>>>>>  Hi Nick,
>>>>>>>  
>>>>>>>  Sorry to be late to make a question.
>>>>>>>  I have a question about the systematic error calculations.
>>>>>>>  
>>>>>>>  At page 35, on the table 4, you calculated the total absolute
>>>>>>>  systematic error
>>>>>>>  to be 0.10.   How this is reflected into the total relative systematic
>>>>>>>  error ?
>>>>>>>  On the pi-p analysis, I think we calculated the systematic errors from
>>>>>>>  cuts in
>>>>>>>  the similar way and they are combined to the other errors (target and
>>>>>>>  beam polarizations).
>>>>>>>  Is this number 0.10 means 10 % or 0.1 % ?
>>>>>>>  
>>>>>>>  I may misunderstand this issue.  Please let me know.
>>>>>>>  
>>>>>>>     Regards, Tsuneo Kageya.
>>>>>>>  
>>>>>>>  ----- Original Message -----
>>>>>>>  From: "g14 run-request" <g14_run-request at jlab.org
>>>>>>> <x-msg://22/g14_run-request@jlab.org> >
>>>>>>>  To: "g14 run" <g14_run at jlab.org <x-msg://22/g14_run@jlab.org> >
>>>>>>>  Sent: Saturday, November 4, 2017 12:00:03 PM
>>>>>>>  Subject: G14_run Digest, Vol 74, Issue 3
>>>>>>>  
>>>>>>>  Send G14_run mailing list submissions to
>>>>>>>  g14_run at jlab.org <x-msg://22/g14_run@jlab.org>
>>>>>>>  
>>>>>>>  To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>>  https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>>>>  or, via email, send a message with subject or body 'help' to
>>>>>>>  g14_run-request at jlab.org <x-msg://22/g14_run-request@jlab.org>
>>>>>>>  
>>>>>>>  You can reach the person managing the list at
>>>>>>>  g14_run-owner at jlab.org <x-msg://22/g14_run-owner@jlab.org>
>>>>>>>  
>>>>>>>  When replying, please edit your Subject line so it is more specific
>>>>>>>  than "Re: Contents of G14_run digest..."
>>>>>>>  
>>>>>>>  
>>>>>>>  Today's Topics:
>>>>>>>  
>>>>>>>    1. Re: Updated Analysis Note (Reinhard Schumacher)
>>>>>>>    2. Re: Updated Analysis Note (Nicholas Zachariou)
>>>>>>>  
>>>>>>>  
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>>  Message: 1
>>>>>>>  Date: Fri, 3 Nov 2017 22:19:24 -0400
>>>>>>>  From: Reinhard Schumacher <schumacher at cmu.edu
>>>>>>> <x-msg://22/schumacher@cmu.edu> >
>>>>>>>  To: g14_run at jlab.org <x-msg://22/g14_run@jlab.org>
>>>>>>>  Subject: Re: [G14_run] Updated Analysis Note
>>>>>>>  Message-ID: <a5780718-56ac-1dc0-44e8-6b991dbf849b at cmu.edu
>>>>>>> <x-msg://22/a5780718-56ac-1dc0-44e8-6b991dbf849b@cmu.edu> >
>>>>>>>  Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>>>>>>>  
>>>>>>>  Hi Nick,
>>>>>>>  
>>>>>>>  Indeed, noticeably improved.?? I recommend that you put the horizontal
>>>>>>>  error bars on Figs.? 29 - 32, too.? They are just as important there
>>>>>>>  since the model curves can vary a lot across one bin.
>>>>>>>  
>>>>>>>  Reinhard
>>>>>>>  
>>>>>>>  
>>>>>>>  On 11/3/2017 7:13 PM, Nicholas Zachariou wrote:
 Dear all,
 
 I am attaching the updated note that incorporates and addresses all
>>>>>>>  comments made. I have noticed that I have previously forgotten to
>>>>>>>  include the systematic uncertainty associated with the
>>>>>>>  photon-selection, and is now estimated and included.
 I would like to thank again Shumacher for his time and valuable
>>>>>>>  insight, and everybody for the comments and feedback. If there is no
>>>>>>>  other comments, I will be submitting the note early next week.
 
 Best regards,
 Nick
 
 
 
 
 
 
 


 G14_run mailing list
 G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
 https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>>>>  
>>>>>>>  --
>>>>>>> 
>>>>>>> 
>>>>>>>  Reinhard Schumacher         Department of Physics, 5000 Forbes Ave.
>>>>>>>  Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.
>>>>>>>  phone: 412-268-5177         web: www-meg.phys.cmu.edu/~schumach
>>>>>>> <http://www-meg.phys.cmu.edu/~schumach>
>>>>>>> <http://www-meg.phys.cmu.edu/~schumach>
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>>  -------------- next part --------------
>>>>>>>  An HTML attachment was scrubbed...
>>>>>>>  URL:
>>>>>>>  
>>>>>>> <https://mailman.jlab.org/pipermail/g14_run/attachments/20171103/651ca31
>>>>>>> 3/at
>>>>>>>  tachment-0001.html>
>>>>>>>  
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>>  Message: 2
>>>>>>>  Date: Sat, 04 Nov 2017 07:10:14 +0000
>>>>>>>  From: Nicholas Zachariou <nicholas at jlab.org
>>>>>>> <x-msg://22/nicholas@jlab.org> >
>>>>>>>  To: Reinhard Schumacher <schumacher at cmu.edu
>>>>>>> <x-msg://22/schumacher@cmu.edu> >
>>>>>>>  Cc: G14 Run <g14_run at jlab.org <x-msg://22/g14_run@jlab.org> >
>>>>>>>  Subject: Re: [G14_run] Updated Analysis Note
>>>>>>>  Message-ID: <d21ed42f-6b13-482d-8c8b-4ea087ad31e1 at jlab.org
>>>>>>> <x-msg://22/d21ed42f-6b13-482d-8c8b-4ea087ad31e1@jlab.org> >
>>>>>>>  Content-Type: text/plain; charset="utf-8"
>>>>>>>  
>>>>>>>  Hi Reinhard,
>>>>>>>  
>>>>>>>  The figures looked a bit busy when I did that (too many lines) and
>>>>>>>  thats why i left the x-uncertainties out in those, but its takes me 2
>>>>>>>  minutes to incorborate them. I think that will be more relevant when we
>>>>>>>  decide exactly how to present our results in the publication.
>>>>>>>  
>>>>>>>  In the meantime I was wondering if its OK with the group to share our
>>>>>>>  preliminary results with the theorists and see if we can get any
>>>>>>>  insights from them.
>>>>>>>  
>>>>>>>  Best regards,
>>>>>>>  Nick
>>>>>>>  
>>>>>>>  
>>>>>>>  On Nov 4, 2017, 02:19, at 02:19, Reinhard Schumacher
>>>>>>>  <schumacher at cmu.edu <x-msg://22/schumacher@cmu.edu> > wrote:
 Hi Nick,
 
 Indeed, noticeably improved.?? I recommend that you put the horizontal
>>>>>>>  
 error bars on Figs.? 29 - 32, too.? They are just as important there
 since the model curves can vary a lot across one bin.
 
 Reinhard
 
 
 On 11/3/2017 7:13 PM, Nicholas Zachariou wrote:
 Dear all,
 
 I am attaching the updated note that incorporates and addresses all
 comments made. I have noticed that I have previously forgotten to
 include the systematic uncertainty associated with the
 photon-selection, and is now estimated and included.
 I would like to thank again Shumacher for his time and valuable
 insight, and everybody for the comments and feedback. If there is no
 other comments, I will be submitting the note early next week.
 
 Best regards,
 Nick
 
 
 
 
 
 
 


 G14_run mailing list
 G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
 https://mailman.jlab.org/mailman/listinfo/g14_run
 
 --


 Reinhard Schumacher         Department of Physics, 5000 Forbes Ave.
 Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A.
 phone: 412-268-5177         web: www-meg.phys.cmu.edu/~schumach
<http://www-meg.phys.cmu.edu/~schumach>
<http://www-meg.phys.cmu.edu/~schumach>


 
 
 


 


 G14_run mailing list
 G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
 https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>>>>  -------------- next part --------------
>>>>>>>  An HTML attachment was scrubbed...
>>>>>>>  URL:
>>>>>>>  
>>>>>>> <https://mailman.jlab.org/pipermail/g14_run/attachments/20171104/14965ba
>>>>>>> f/at
>>>>>>>  tachment-0001.html>
>>>>>>>  
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>>  Subject: Digest Footer
>>>>>>>  
>>>>>>> 
>>>>>>> 
>>>>>>>  G14_run mailing list
>>>>>>>  G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>>  https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>>>>  
>>>>>>>  
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>>  End of G14_run Digest, Vol 74, Issue 3
>>>>>>>  **************************************
>>>>>>> 
>>>>>>> 
>>>>>>>  G14_run mailing list
>>>>>>>  G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>>  https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>>> 
>>>>>> 
>>>>>>  G14_run mailing list
>>>>>>  G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>>  https://mailman.jlab.org/mailman/listinfo/g14_run
>>>>> 
>>>>> 
>>>>>  G14_run mailing list
>>>>>  G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>>>  https://mailman.jlab.org/mailman/listinfo/g14_run
>>>> 
>>>> 
>>>> 
>>>> 
>>>> G14_run mailing list
>>>> G14_run at jlab.org <x-msg://22/G14_run@jlab.org>
>>>> https://mailman.jlab.org/mailman/listinfo/g14_run
>>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/g14_run/attachments/20171108/d606c552/attachment-0001.html>

More information about the G14_run mailing list