[Halld-offline] [EXTERNAL] Re: heads-up: pull requests require review for halld_recon and halld_sim

Thu Apr 30 10:00:32 EDT 2020

We could use more automated testing.  It would be nice to have a
build+run+compare test instead of just a build test.

in my previous job I wrote the back-end software for this
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.phe-2Dprotectionservices.org.uk_imba_&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=Te_hCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4&m=6VZR-vIePl47fHjWPfbEHD1HtA9LvXNXT8bwIhb3rMI&s=WOY6yKB6qflkluL9BEHMFzRZ48vDQ12gfAFvrLuHLv8&e=    Eventually I wrote an
automated test facility for myself to use to compare results between old
and new versions using many strategically picked scenarios, to check that
the changes did what they were supposed to and nothing else.  Writing the
test facility was a discouraged diversion initially but well worth it a few
years later.

GlueX could use something like this, at least to compare tagged releases.
I am not suggesting checking histograms by eye on a website, I would prefer
an automatic numerical comparison which flags differences and %
differences, for many plugins, many histograms.

Naomi.

On Wed, Apr 29, 2020 at 9:56 PM David Lawrence <davidl at jlab.org> wrote:

> I just meant the analogy doesn’t seem complete since lots of people do
> actually download and build
> the Hall-D software, unlike a data analysis.
>
> Regards,
> -David
>
> -------------------------------------------------------------
> David Lawrence Ph.D.
> Staff Scientist, Thomas Jefferson National Accelerator Facility
> Newport News, VA
> davidl at jlab.org
> (757) 269-5567 W
> (757) 746-6697 C
>
>
> On Apr 29, 2020, at 1:47 PM, Richard Jones <richard.t.jones at uconn.edu>
> wrote:
>
> @David, this is already automated, I thought.
> -rtj
>
> On Wed, Apr 29, 2020 at 1:40 PM David Lawrence <davidl at jlab.org> wrote:
>
>>
>> *Message sent from a system outside of UConn.*
>>
>>
>>
>> The reviewer though will likely never need to download the data and run
>> the analysis whereas someone elese will need to compile and run the code.
>> Just a thought.
>> Regards,
>> -David
>>
>> -------------------------------------------------------------
>> David Lawrence Ph.D.
>> Staff Scientist, Thomas Jefferson National Accelerator Facility
>> Newport News, VA
>> davidl at jlab.org
>> (757) 269-5567 W
>> (757) 746-6697 C
>>
>>
>> On Apr 29, 2020, at 10:32 AM, Richard Jones <richard.t.jones at uconn.edu>
>> wrote:
>>
>> Or how about just emailing the author and asking, have you done this? Or
>> can you do this one more test, and I will approve it. Something like the
>> way peer review of articles works. Usually the reviewer does not download
>> the data and carry out independent checks of the results.
>>
>> -rtj
>>
>> On Wed, Apr 29, 2020 at 9:37 AM Naomi Jarvis <nsj at cmu.edu> wrote:
>>
>>>
>>> *Message sent from a system outside of UConn.*
>>>
>>>
>>> I thought the review process would be something like download the new
>>> branch, make sure it compiles, make sure it runs, make sure it does
>>> whatever it was supposed to do, then go to the website and click the merge
>>> button (and presumably one more now).  That's what I do before approving
>>> PRs (& why I don't approve many).
>>>
>>> Naomi.
>>>
>>> On Wed, Apr 29, 2020 at 9:00 AM Sean Dobbs <sdobbs at fsu.edu> wrote:
>>>
>>>> We could just start with a short comment like “looks good” for now, and
>>>> then revisit the question next software meeting when more people are
>>>> familiar with what functionality is available.
>>>>
>>>> Cheers
>>>> Sean
>>>>
>>>> On Wed, Apr 29, 2020 at 8:46 AM Mark Ito <marki at jlab.org> wrote:
>>>>
>>>>> Mark,
>>>>>
>>>>> That is a good point, well taken. Anyone who was at the meeting
>>>>> yesterday care to take this one?
>>>>>
>>>>>    -- Mark
>>>>>
>>>>> On 4/29/20 8:08 AM, Mark-Macrae Dalton wrote:
>>>>> > It might be useful to spell out more clearly what is expected from a
>>>>> review of a pull request, other than clicking the button.
>>>>> >
>>>>> > Mark
>>>>> >
>>>>> >> On Apr 28, 2020, at 8:06 PM, Mark Ito <marki at jlab.org> wrote:
>>>>> >>
>>>>> >> Cool. Good to know.
>>>>> >>
>>>>> >> On 4/28/20 7:59 PM, Justin Stevens wrote:
>>>>> >>> FYI, just successfully executed the review -> merge PR sequence.
>>>>> There’s basically just one more button to push to “Approve” the review, and
>>>>> then you can merge as usual.
>>>>> >>> -Justin
>>>>> >>>
>>>>> >>>> On Apr 28, 2020, at 5:04 PM, Mark Ito <marki at jlab.org> wrote:
>>>>> >>>>
>>>>> >>>> Folks,
>>>>> >>>>
>>>>> >>>> At the software meeting just now, we decided to require "reviews"
>>>>> of pull requests before they can be merged onto the master branch. For now
>>>>> this is only for halld_sim and halld_recon. The webpages for the individual
>>>>> pull requests now reflect this. The idea is to prevent inadvertent changes
>>>>> to the master branches.
>>>>> >>>>
>>>>> >>>>    -- Mark
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> _______________________________________________
>>>>> >>>> Halld-offline mailing list
>>>>> >>>> Halld-offline at jlab.org
>>>>> >>>> https://mailman.jlab.org/mailman/listinfo/halld-offline
>>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fmailman.jlab.org-2D252Fmailman-2D252Flistinfo-2D252Fhalld-2D2Doffline-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257C27fd9227d73c4da2c00a08d7ec41b3d3-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637237639251846251-2D26sdata-2D3DgkJIGEsJ703IQhrHeWKZ-2D252FMkMnKATp4BMnhUAJVqvY2A-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253DTe-5FhCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4-2526m-253Ds1rydy2wtTm4oNwi0LnpZYCnXrxxs7xi6nYvhktvqrw-2526s-253DUo92-5FuGUaoF8XYALdVBVoyrm-2DcrKFl56IQgWHhHVHVU-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441212059-26sdata-3Dq25ITJgvC9UPE-252BHy-252BJyjUE9ZFtfW1rjXa710XQHW-252F-252F4-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=wDxYdD4bn6Kgtfu0H2wWihJHhvaNAfgmXO8_Fcr0Udc&e=>
>>>>> >> _______________________________________________
>>>>> >> Halld-offline mailing list
>>>>> >> Halld-offline at jlab.org
>>>>> >> https://mailman.jlab.org/mailman/listinfo/halld-offline
>>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fmailman.jlab.org-2D252Fmailman-2D252Flistinfo-2D252Fhalld-2D2Doffline-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257C27fd9227d73c4da2c00a08d7ec41b3d3-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637237639251846251-2D26sdata-2D3DgkJIGEsJ703IQhrHeWKZ-2D252FMkMnKATp4BMnhUAJVqvY2A-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253DTe-5FhCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4-2526m-253Ds1rydy2wtTm4oNwi0LnpZYCnXrxxs7xi6nYvhktvqrw-2526s-253DUo92-5FuGUaoF8XYALdVBVoyrm-2DcrKFl56IQgWHhHVHVU-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441222053-26sdata-3DXZgZ9P33ty0l2HKcrtXzIya5xyOyuaXWqkzAts0B960-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=toL7GnUhgl98mFFc6lLv0-BKTIeK7s4Xwmdmo5tt05Q&e=>
>>>>> _______________________________________________
>>>>> Halld-offline mailing list
>>>>> Halld-offline at jlab.org
>>>>> https://mailman.jlab.org/mailman/listinfo/halld-offline
>>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fmailman.jlab.org-2D252Fmailman-2D252Flistinfo-2D252Fhalld-2D2Doffline-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257C27fd9227d73c4da2c00a08d7ec41b3d3-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637237639251856246-2D26sdata-2D3D0acpsNjdYzHIez-2D252BvBUDisiD6Sj8p6oM78XUE685C98I-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253DTe-5FhCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4-2526m-253Ds1rydy2wtTm4oNwi0LnpZYCnXrxxs7xi6nYvhktvqrw-2526s-253DuVv3yNkBzGPBbTo82Q8EhRnOB6xhe0NLTA-5FaK-5F-2D6WSM-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441222053-26sdata-3DDtel-252BemhWlraSOO06iWWPAPffXJhdtb-252FrE8Y2JiC4Gw-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=vU05M4E25Pbyzf6636BeyPwT223cSsAZZtkGYHLzuMM&e=>
>>>>
>>>> _______________________________________________
>>>> Halld-offline mailing list
>>>> Halld-offline at jlab.org
>>>> https://mailman.jlab.org/mailman/listinfo/halld-offline
>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fmailman.jlab.org-2D252Fmailman-2D252Flistinfo-2D252Fhalld-2D2Doffline-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257C27fd9227d73c4da2c00a08d7ec41b3d3-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637237639251856246-2D26sdata-2D3D0acpsNjdYzHIez-2D252BvBUDisiD6Sj8p6oM78XUE685C98I-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253DTe-5FhCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4-2526m-253Ds1rydy2wtTm4oNwi0LnpZYCnXrxxs7xi6nYvhktvqrw-2526s-253DuVv3yNkBzGPBbTo82Q8EhRnOB6xhe0NLTA-5FaK-5F-2D6WSM-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441232049-26sdata-3DfXoQ-252F5mJ2yyIcNpJhFZbUL1jyq8X-252BQ22YzzjzttTETM-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=gj1qO_u7OQwgqQmgQiyHrlN9Hpb86LGLtzUM2p_h8sc&e=>
>>>
>>> _______________________________________________
>>> Halld-offline mailing list
>>> Halld-offline at jlab.org
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmailman.jlab.org-252Fmailman-252Flistinfo-252Fhalld-2Doffline-26amp-3Bdata-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C27fd9227d73c4da2c00a08d7ec41b3d3-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237639251886232-26amp-3Bsdata-3DdoNOCLzRPLj59h9l4LLjZ04WFccf2s7KPmLa0vXRHEs-253D-26amp-3Breserved-3D0&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=Te_hCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4&m=6VZR-vIePl47fHjWPfbEHD1HtA9LvXNXT8bwIhb3rMI&s=uh9Liav_AzD5B_d5nwC9ZToUY9obJUjVqBZkXde7zoE&e= 
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fmailman.jlab.org-2D252Fmailman-2D252Flistinfo-2D252Fhalld-2D2Doffline-2D26amp-2D3Bdata-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257C27fd9227d73c4da2c00a08d7ec41b3d3-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637237639251886232-2D26amp-2D3Bsdata-2D3DdoNOCLzRPLj59h9l4LLjZ04WFccf2s7KPmLa0vXRHEs-2D253D-2D26amp-2D3Breserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253DTe-5FhCR4EUlJ6iCDYLJ8Viv2aDOR7D9ZZMoBAvf2H0M4-2526m-253Ds1rydy2wtTm4oNwi0LnpZYCnXrxxs7xi6nYvhktvqrw-2526s-253D6e6-2DOfuqOOge3abxyj5Vicwdfy-5FvI-2Dt762D8-2DaTxNxY-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441242048-26sdata-3DjxKufYQaNk5Jd9YZLjMCTggr3LZRXRIwluyXXQYCClc-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=LKEB3iyEWlaHNZ2WdVnTrxtgWkci-BR1gEnz95l8TII&e=>
>>
>> _______________________________________________
>> Halld-offline mailing list
>> Halld-offline at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/halld-offline
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmailman.jlab.org-252Fmailman-252Flistinfo-252Fhalld-2Doffline-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C3224b389c3ea4042e01a08d7ec647081-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637237788441242048-26sdata-3DuEgzcanz3l1xBt3V2s2oUdfgc7k4P9leryuUxCwBjAw-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=i0p-C_T1RieVsCZl8ZnUyQ&m=XwmANgWW_lq5fapXyHRU8fUAPhJtMIopxTuSv8myGS8&s=6zJWBM8w_4PRho1iUz1wUzK3PGJUd7lFJTgmXmkzuR0&e=>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20200430/30b3bcfa/attachment-0002.html>