[Clas_offline] minutes from OLTWG meeting Nov 12, 2009
Stepan Stepanyan
stepanya at jlab.org
Mon Nov 16 20:28:33 EST 2009
We need "ec_time" under utilities. From packages -
sh
icf
There may be some others that I do not remember.
Stepan
Eugene Pasyuk wrote:
> There seems to be several programs for ec calibrations.
> What I see:
>
> ec_calib
> ecGammaCal
> ec_photnCal
>
> There is also ec_timing. I know for sure this one must be ported.
>
> I suppose EC experts out there should tell us which ones are to be ported.
>
> I think the last time pid_mon was used in g6c cooking.
>
> We also should copy ec1_calib. This is for LAC calibration.
>
> Perhaps IC folks have something for IC calibrations to be copied.
>
> The same for RTPC.
>
> I forgot to mention filter_tagr utilitie. It's part of gsim/gpp suite.
> We also need to port DC alignment utilities. They are in calib/dc_cal
>
> And DC efficiency/dead wire utilities. They are in pdu directory
>
> -Eugene
>
> Craig Bookwalter wrote:
>
>> Hi Eugene,
>> sc_mon was ready, I just forgot to add it to the repository. It's in
>> there now.
>>
>> I built ecGammaCal but not ec_calib, mostly because I don't clearly
>> understand ec_calib's role or source code structure--it also has its own
>> versions of libsc.a and libec.a. That might be a mistake but I wasn't
>> sure. I suppose ec_calib is used for electron running...?
>> Should I also include pid_mon?
>>
>> Of the unfamiliar programs you mention, the ones in camelCase are
>> related to ecGammaCal--I don't know how that program works but I guess
>> it has like 5 executables. `skim' and the `view*' programs are all
>> g12-specific, and can probably be relegated to on-demand builds. gif2C
>> is unfamiliar to me as well. part2gamp and vice-versa can easily be
>> relegated as well.
>>
>> Thanks for the quick feedback.
>>
>> --cb
>>
>> Eugene Pasyuk wrote:
>>
>>> Craig, Great progress!
>>>
>>> In addition to cc calibrations you mentioned also missing but
>>> essential for calibrations and must be in default build:
>>> sc_mon
>>> gpid_mon
>>> sync
>>> ec calibrations
>>>
>>> I agree, some of the packages we have may be left for building on
>>> demand but should be ported to svn. I'll take a look and let you know
>>> what appears to be necessary to port to svn.
>>>
>>> Also, some of the executables in your default build do not look familiar:
>>>
>>> checkBetaNeutrals
>>> computeCalibConsts
>>> countgamp
>>> filterGammas
>>> fitTimeDifference
>>> gamp2part
>>> getCalibConsts
>>> gif2C
>>> part2gamp
>>> partMaker
>>> putConstsInMap
>>> rejectGammas
>>> skim
>>> view-kaon
>>> view-st
>>> view-tag
>>> view-tracks
>>>
>>> What are they? Are they of general interest?
>>>
>>>
>>> One more thing just occurred to me. We have tools directory in CVS
>>> with collection of scripts for various things. Particularly important
>>> are scripts for caldb access. Are those ported to svn?
>>> The checked out version is in /group/clas/tools There is also
>>> environment CLAS_TOOLS pointing to this location.
>>>
>>>
>>>
>>> Craig Bookwalter wrote:
>>>
>>>
>>>> Folks,
>>>> The current status of the SVN repository is as follows.
>>>>
>>>> A default build following the instructions here:
>>>> http://clasweb.jlab.org/wiki/index.php/Building_CLAS_libraries_with_SCons_on_ifarm
>>>>
>>>>
>>>> produces these executables:
>>>> a1c
>>>> bosdump
>>>> ced
>>>> checkBetaNeutrals
>>>> computeCalibConsts
>>>> countbos
>>>> countgamp
>>>> dc3
>>>> filterGammas
>>>> fitTimeDifference
>>>> g12Anal
>>>> g12skim
>>>> gamecock
>>>> gamp2part
>>>> getCalibConsts
>>>> gflux
>>>> gif2C
>>>> gmean_cooked
>>>> gpp
>>>> gsim_bat
>>>> gsim_int
>>>> make_tw_histos
>>>> p2p_delay_el
>>>> part2gamp
>>>> partMaker
>>>> putConstsInMap
>>>> rejectGammas
>>>> skim
>>>> stn_calib
>>>> tagraw
>>>> tdc_calib
>>>> tof_calib
>>>> trk_mon
>>>> trk_mon_lite
>>>> user_ana
>>>> veff_calib
>>>> view-kaon
>>>> view-st
>>>> view-tag
>>>> view-tracks
>>>>
>>>> Notable exceptions are:
>>>> - the CC calibration program, which requires Qt. I'm going to defer
>>>> to Maurizio for this one, because it requires an optional SCons
>>>> package that I'm not familiar with.
>>>> - the TPhotTiming tool for doing TOF paddle-to-paddle alignments for
>>>> photon runs. I'm currently attempting to build ClasTool in order to
>>>> get TPhotTiming working, but the build of ClasTool is something of a
>>>> novelty under SCons at the moment. Are there any other standard CLAS
>>>> packages that use ClasTool?
>>>>
>>>> In addition, Eugene noted a number of programs that were not included
>>>> previously by default, and still are not included in the latest
>>>> build. In the past many of these programs were built on demand by the
>>>> user as they were needed, and this functionality still exists in
>>>> SCons. For example, in analysis/bos2ntuple live some ntuple makers
>>>> for a couple of very specific photoproduction channels. These should
>>>> not be built by default because they are not widely useful for our
>>>> collaborators. A user who wants to build such a program can just `cd'
>>>> to the appropriate directory and run `scons install' from there.
>>>>
>>>> Of course, there are still programs that may be necessary for
>>>> rungroups that do not exist yet in SVN, or exist in SVN but do not
>>>> have a functional sconstruct. I'd like to know what these programs
>>>> are so I can bring them into the fold and help smooth the process of
>>>> experimenting with (and hopefully transitioning to) the new build
>>>> system. I would especially like to incorporate missing programs in
>>>> demand by the most active rungroups, like eg6, e1-dvcs and eg1-dvcs,
>>>> and g9b.
>>>>
>>>> Comments and concerns are welcome.
>>>>
>>>> --cb
>>>>
>>>> Eugene Pasyuk wrote:
>>>>
>>>>
>>>>> I don't have statistics for svn for svn.
>>>>> I am sure it is possible, but i don't know yet how to get this
>>>>> information from svn. Our svn experts can easily get this
>>>>> statistics, I suppose.
>>>>> My wild guess wold be that there were more changes in svn over this
>>>>> period of time.
>>>>>
>>>>> -Eugene
>>>>>
>>>>>
>>>>> Stepan Stepanyan wrote:
>>>>>
>>>>>
>>>>>> Eugene,
>>>>>>
>>>>>> Interesting statistics. Do you have any thing similar for svn
>>>>>> for the past 2 months?
>>>>>>
>>>>>> Stepan
>>>>>>
>>>>>> Eugene Pasyuk wrote:
>>>>>>
>>>>>>
>>>>>>> Here are some statistics of cvs activity. I looked over the same
>>>>>>> period
>>>>>>> of time from September 16 till today. All the information is from
>>>>>>> commitlog file which keeps all commits done in repository since
>>>>>>> day one,
>>>>>>> May 12, 1997. It has 42186 records total.
>>>>>>>
>>>>>>> So, since September 16 2009:
>>>>>>>
>>>>>>> Total number of commits 66.
>>>>>>> The majority of the changes are for Current Run web page.
>>>>>>> There are 18 commits related to software.
>>>>>>> Out of these 12 are in packages ported to svn.
>>>>>>> Details for these 12:
>>>>>>>
>>>>>>> 1 commit in ClasTool. (Sep 28)
>>>>>>>
>>>>>>> 5 commits in packages/utilities/sync. (Oct 9) They were done in
>>>>>>> one day,
>>>>>>> almost simultaneously. Can count as 1 commit.
>>>>>>>
>>>>>>> 5 commits in packages/eloss Sep30 Oct2 Oct22 Oct 23 Oct 29
>>>>>>> There is ongoing development of this package for FROST target.
>>>>>>>
>>>>>>> 1 commit in packages/seb done yesterday.
>>>>>>>
>>>>>>> These are all changes over the last two months.
>>>>>>>
>>>>>>> -Eugene
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, 2009-11-13 at 09:03 +0100, Marco Ripani wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi Stepan,
>>>>>>>> yes, it is necessary to put more efforts to fix everything and
>>>>>>>> then take an action. More help is needed to port the missing
>>>>>>>> packages and test that they build and work fine (there are still
>>>>>>>> many, as we see from Eugene's new test).
>>>>>>>> But currently there is also the problem of keeping CVS and SVN in
>>>>>>>> sync for packages that have been ported (e.g. user_ana). Johann
>>>>>>>> is doing that using scripts but he also has to do some manual
>>>>>>>> adjustments. To avoid this, the sooner we start using packages in
>>>>>>>> the new system, the better.
>>>>>>>>
>>>>>>>> Marco
>>>>>>>>
>>>>>>>> Stepan Stepanyan wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi Marco,
>>>>>>>>>
>>>>>>>>> I am not sure how to read your email. What do you mean "freeze
>>>>>>>>> CVS asap, not today".
>>>>>>>>> At the software meeting couple of month ago I though we came to
>>>>>>>>> agreement that we
>>>>>>>>> will discuss cvs vs svn after some experience will be gained.
>>>>>>>>> Several groups expressed
>>>>>>>>> interest to test svn/scon (e1-dvcs, eg1-dvcs, frost ...). So far
>>>>>>>>> only Eugene made an attempt
>>>>>>>>> to use svn and scon and as I can see from his email he succeeded
>>>>>>>>> using the staff that
>>>>>>>>> was polished by g12 for about a year, but there are still things
>>>>>>>>> that either missing or not
>>>>>>>>> ready yet. Why not put more efforts to fix every thing and then
>>>>>>>>> take an action?
>>>>>>>>>
>>>>>>>>> Stepan
>>>>>>>>>
>>>>>>>>> Marco Ripani wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> need to freeze CVS but asap, not today. Need to first figure
>>>>>>>>>> out what the best time is, with minimum impact on the other run
>>>>>>>>>> groups, in particular due to the missing packages.
>>>>>>>>>>
>>>>>>>>>> Marco
>>>>>>>>>>
>>>>>>>>>> Johann Goetz wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Thursday, Nov 12, 2009: SCons Build System and CLAS 6 Software
>>>>>>>>>>> 12:00 pm Eastern
>>>>>>>>>>> Cebaf Center A-110
>>>>>>>>>>> Call in information: Dial 1-877-643-6951
>>>>>>>>>>> International access: 1-302-607-2017
>>>>>>>>>>> Passcode: 24630562#
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> RPMs, yum, and all that (Maurizio)
>>>>>>>>>>> * creating RPM (installable pre-compiled packages with
>>>>>>>>>>> dependency
>>>>>>>>>>> tree) of the CLAS software is a great idea.
>>>>>>>>>>> * these RPMs can served with a JLab yum server so that
>>>>>>>>>>> installation on a Fedora or Red Hat (or CentOS) system
>>>>>>>>>>> could be
>>>>>>>>>>> done with a line like:
>>>>>>>>>>> yum install clas6-reconstruction clas6-generators etc...
>>>>>>>>>>> * CLAS12 has already implemented this on several flavors
>>>>>>>>>>> of Linux
>>>>>>>>>>> *
>>>>>>>>>>> Eugene's Review of SCons Build System
>>>>>>>>>>> * Generally favourable review. Eugene points out that
>>>>>>>>>>> there are
>>>>>>>>>>> many packages missing that are needed.
>>>>>>>>>>> * Many packages will have to copied over directories
>>>>>>>>>>> from CVS. * sconstruct files will have to be made.
>>>>>>>>>>> * gamecock (TOF calib program) has some problems
>>>>>>>>>>> (ROOT/GUI
>>>>>>>>>>> related)
>>>>>>>>>>> * CLASTool should be fairly easy to get to work
>>>>>>>>>>> now that
>>>>>>>>>>> ROOT dictionaries can be built properly.
>>>>>>>>>>> From: Eugene Pasyuk <pasyuk at jlab.org>
>>>>>>>>>>> To: CLAS Offline <clas_offline at jlab.org>
>>>>>>>>>>> Subject: [Clas_offline] testing svn/scons
>>>>>>>>>>> Date: Wed, 16 Sep 2009 10:29:31 -0400 (07:29 PDT)
>>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I tried new software tools today on ifarm. Here are some first
>>>>>>>>>>> impressions about this experience.
>>>>>>>>>>>
>>>>>>>>>>> I followed prescription given on the Wiki:
>>>>>>>>>>> http://clasweb.jlab.org/wiki/index.php/Building_CLAS_libraries_with_SCons_on_ifarm
>>>>>>>>>>>
>>>>>>>>>>> It took 20 minutes get the entire source tree source from svn.
>>>>>>>>>>> Then I tried to build. The very first attempt failed. I
>>>>>>>>>>> figured out that
>>>>>>>>>>> the problem was in suggested environment setup file. I
>>>>>>>>>>> modified a couple
>>>>>>>>>>> of lines in it (2 minutes to figure out and fix). After that I
>>>>>>>>>>> made a
>>>>>>>>>>> second attempt to run scons. This time it worked. It took about 3
>>>>>>>>>>> minutes to build all clas libraries and a just few executables by
>>>>>>>>>>> default:
>>>>>>>>>>> a1c countbos g12Anal gsim_bat tagraw view-kaon
>>>>>>>>>>> view-tag
>>>>>>>>>>> bosdump g12skim skim user_ana view-st view-tracks
>>>>>>>>>>>
>>>>>>>>>>> Out of these only few (a1c, gsim_bat, countbos, bosdump,
>>>>>>>>>>> user_ana and
>>>>>>>>>>> tagraw) are of interest to general CLAS user. The rest seem to
>>>>>>>>>>> be g12
>>>>>>>>>>> specific.
>>>>>>>>>>> After doing some archeology in sconstruct files for about a
>>>>>>>>>>> half an hour
>>>>>>>>>>> and making some modifications to them here and there (I have zero
>>>>>>>>>>> knowledge of scons or python at the moment) I managed to build
>>>>>>>>>>> a few
>>>>>>>>>>> more utilities, particularly DC calibration tools trk_mon,
>>>>>>>>>>> trk_mon_lite
>>>>>>>>>>> and dc3. I failed to build start counter calibration code. I
>>>>>>>>>>> was able to
>>>>>>>>>>> build fdump. gflux would not build because of the compiler
>>>>>>>>>>> errors. I
>>>>>>>>>>> wonder how g12 managed to build it in new build scheme from
>>>>>>>>>>> the source
>>>>>>>>>>> in svn. ced would not build because of the compiler errors too.
>>>>>>>>>>> TOF calibration source code is in svn, but it was not modified
>>>>>>>>>>> for scons
>>>>>>>>>>> at all. It was imported as is from cvs with all Makefiles
>>>>>>>>>>> which will not
>>>>>>>>>>> work because of the different tree structure. sc_mon is
>>>>>>>>>>> missing. As was
>>>>>>>>>>> mentioned at the meeting ec timing calibration utilities are not
>>>>>>>>>>> included as several other general purpose utilities. For some
>>>>>>>>>>> reason
>>>>>>>>>>> gsim_int and gpp are not included.
>>>>>>>>>>>
>>>>>>>>>>> To summarize, I succeeded in building most of the libraries
>>>>>>>>>>> and a few
>>>>>>>>>>> executables. Quite a few tools are missing (gsim_int, gpp, some
>>>>>>>>>>> calibration tools). Some programs which build fine with make
>>>>>>>>>>> would not
>>>>>>>>>>> build with scons (gflux, ced ...?) I don't know if those
>>>>>>>>>>> executables I
>>>>>>>>>>> built are working, I had not had a chance to test them yet. I
>>>>>>>>>>> have not
>>>>>>>>>>> tried to build any of the event generators with scons.
>>>>>>>>>>> By the way, there are only three of them in svn, while we have
>>>>>>>>>>> more in
>>>>>>>>>>> cvs.
>>>>>>>>>>>
>>>>>>>>>>> All in all, svn works just fine, apart from having different
>>>>>>>>>>> directory
>>>>>>>>>>> structure. scons works for some pieces of the software and
>>>>>>>>>>> does not for
>>>>>>>>>>> some
>>>>>>>>>>> others. Perhaps it is not quite ready for the prime time yet.
>>>>>>>>>>> Some essential
>>>>>>>>>>> utilities are not working yet in new build. -Eugene
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> What We Should Do Now
>>>>>>>>>>> * Hire some undergraduates to move packages from CVS to
>>>>>>>>>>> SVN and
>>>>>>>>>>> add sconstruct files to build these packages.
>>>>>>>>>>> * Armand Fraiser was a big help initially with the move to
>>>>>>>>>>> SVN/SCons and he could continue to do so.
>>>>>>>>>>> * There is some detailed help information already, it
>>>>>>>>>>> probably
>>>>>>>>>>> needs a bit of reorganization on the wiki.
>>>>>>>>>>> * Should we have a list of programs/libraries that
>>>>>>>>>>> should be moved
>>>>>>>>>>> from CVS to SVN (Armand already created an spreadsheet
>>>>>>>>>>> that has
>>>>>>>>>>> some of this information. It will be sent to Craig and
>>>>>>>>>>> he will
>>>>>>>>>>> look into it.
>>>>>>>>>>> * We need to freeze CVS today.
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Clas_offline mailing list
>>>>>>>>>>> Clas_offline at jlab.org
>>>>>>>>>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Clas_offline mailing list
>>>>>>> Clas_offline at jlab.org
>>>>>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>>>>>
>>>>>>>
>>>>> _______________________________________________
>>>>> Clas_offline mailing list
>>>>> Clas_offline at jlab.org
>>>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Clas_offline mailing list
>>> Clas_offline at jlab.org
>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>
>>>
>>
> _______________________________________________
> Clas_offline mailing list
> Clas_offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/clas_offline
>
More information about the Clas_offline
mailing list