[Clas_offline] minutes from OLTWG meeting Nov 12, 2009
Eugene Pasyuk
pasyuk at jlab.org
Mon Nov 16 18:19:38 EST 2009
Craig, Great progress!
In addition to cc calibrations you mentioned also missing but essential
for calibrations and must be in default build:
sc_mon
gpid_mon
sync
ec calibrations
I agree, some of the packages we have may be left for building on demand
but should be ported to svn. I'll take a look and let you know what
appears to be necessary to port to svn.
Also, some of the executables in your default build do not look familiar:
checkBetaNeutrals
computeCalibConsts
countgamp
filterGammas
fitTimeDifference
gamp2part
getCalibConsts
gif2C
part2gamp
partMaker
putConstsInMap
rejectGammas
skim
view-kaon
view-st
view-tag
view-tracks
What are they? Are they of general interest?
One more thing just occurred to me. We have tools directory in CVS with
collection of scripts for various things. Particularly important are
scripts for caldb access. Are those ported to svn?
The checked out version is in /group/clas/tools There is also
environment CLAS_TOOLS pointing to this location.
Craig Bookwalter wrote:
> Folks,
> The current status of the SVN repository is as follows.
>
> A default build following the instructions here:
> http://clasweb.jlab.org/wiki/index.php/Building_CLAS_libraries_with_SCons_on_ifarm
>
> produces these executables:
> a1c
> bosdump
> ced
> checkBetaNeutrals
> computeCalibConsts
> countbos
> countgamp
> dc3
> filterGammas
> fitTimeDifference
> g12Anal
> g12skim
> gamecock
> gamp2part
> getCalibConsts
> gflux
> gif2C
> gmean_cooked
> gpp
> gsim_bat
> gsim_int
> make_tw_histos
> p2p_delay_el
> part2gamp
> partMaker
> putConstsInMap
> rejectGammas
> skim
> stn_calib
> tagraw
> tdc_calib
> tof_calib
> trk_mon
> trk_mon_lite
> user_ana
> veff_calib
> view-kaon
> view-st
> view-tag
> view-tracks
>
> Notable exceptions are:
> - the CC calibration program, which requires Qt. I'm going to defer to
> Maurizio for this one, because it requires an optional SCons package
> that I'm not familiar with.
> - the TPhotTiming tool for doing TOF paddle-to-paddle alignments for
> photon runs. I'm currently attempting to build ClasTool in order to get
> TPhotTiming working, but the build of ClasTool is something of a novelty
> under SCons at the moment. Are there any other standard CLAS packages
> that use ClasTool?
>
> In addition, Eugene noted a number of programs that were not included
> previously by default, and still are not included in the latest build.
> In the past many of these programs were built on demand by the user as
> they were needed, and this functionality still exists in SCons. For
> example, in analysis/bos2ntuple live some ntuple makers for a couple of
> very specific photoproduction channels. These should not be built by
> default because they are not widely useful for our collaborators. A user
> who wants to build such a program can just `cd' to the appropriate
> directory and run `scons install' from there.
>
> Of course, there are still programs that may be necessary for rungroups
> that do not exist yet in SVN, or exist in SVN but do not have a
> functional sconstruct. I'd like to know what these programs are so I can
> bring them into the fold and help smooth the process of experimenting
> with (and hopefully transitioning to) the new build system. I would
> especially like to incorporate missing programs in demand by the most
> active rungroups, like eg6, e1-dvcs and eg1-dvcs, and g9b.
>
> Comments and concerns are welcome.
>
> --cb
>
> Eugene Pasyuk wrote:
>> I don't have statistics for svn for svn.
>> I am sure it is possible, but i don't know yet how to get this
>> information from svn. Our svn experts can easily get this statistics, I
>> suppose.
>> My wild guess wold be that there were more changes in svn over this
>> period of time.
>>
>> -Eugene
>>
>>
>> Stepan Stepanyan wrote:
>>
>>> Eugene,
>>>
>>> Interesting statistics. Do you have any thing similar for svn
>>> for the past 2 months?
>>>
>>> Stepan
>>>
>>> Eugene Pasyuk wrote:
>>>
>>>> Here are some statistics of cvs activity. I looked over the same period
>>>> of time from September 16 till today. All the information is from
>>>> commitlog file which keeps all commits done in repository since day one,
>>>> May 12, 1997. It has 42186 records total.
>>>>
>>>> So, since September 16 2009:
>>>>
>>>> Total number of commits 66.
>>>> The majority of the changes are for Current Run web page.
>>>> There are 18 commits related to software.
>>>> Out of these 12 are in packages ported to svn.
>>>> Details for these 12:
>>>>
>>>> 1 commit in ClasTool. (Sep 28)
>>>>
>>>> 5 commits in packages/utilities/sync. (Oct 9) They were done in one day,
>>>> almost simultaneously. Can count as 1 commit.
>>>>
>>>> 5 commits in packages/eloss Sep30 Oct2 Oct22 Oct 23 Oct 29
>>>> There is ongoing development of this package for FROST target.
>>>>
>>>> 1 commit in packages/seb done yesterday.
>>>>
>>>> These are all changes over the last two months.
>>>>
>>>> -Eugene
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, 2009-11-13 at 09:03 +0100, Marco Ripani wrote:
>>>>
>>>>
>>>>> Hi Stepan,
>>>>> yes, it is necessary to put more efforts to fix everything and then
>>>>> take an action. More help is needed to port the missing packages and
>>>>> test that they build and work fine (there are still many, as we see
>>>>> from Eugene's new test).
>>>>> But currently there is also the problem of keeping CVS and SVN in
>>>>> sync for packages that have been ported (e.g. user_ana). Johann is
>>>>> doing that using scripts but he also has to do some manual
>>>>> adjustments. To avoid this, the sooner we start using packages in the
>>>>> new system, the better.
>>>>>
>>>>> Marco
>>>>>
>>>>> Stepan Stepanyan wrote:
>>>>>
>>>>>
>>>>>> Hi Marco,
>>>>>>
>>>>>> I am not sure how to read your email. What do you mean "freeze CVS
>>>>>> asap, not today".
>>>>>> At the software meeting couple of month ago I though we came to
>>>>>> agreement that we
>>>>>> will discuss cvs vs svn after some experience will be gained.
>>>>>> Several groups expressed
>>>>>> interest to test svn/scon (e1-dvcs, eg1-dvcs, frost ...). So far
>>>>>> only Eugene made an attempt
>>>>>> to use svn and scon and as I can see from his email he succeeded
>>>>>> using the staff that
>>>>>> was polished by g12 for about a year, but there are still things
>>>>>> that either missing or not
>>>>>> ready yet. Why not put more efforts to fix every thing and then take
>>>>>> an action?
>>>>>>
>>>>>> Stepan
>>>>>>
>>>>>> Marco Ripani wrote:
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>> need to freeze CVS but asap, not today. Need to first figure out
>>>>>>> what the best time is, with minimum impact on the other run groups,
>>>>>>> in particular due to the missing packages.
>>>>>>>
>>>>>>> Marco
>>>>>>>
>>>>>>> Johann Goetz wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thursday, Nov 12, 2009: SCons Build System and CLAS 6 Software
>>>>>>>> 12:00 pm Eastern
>>>>>>>> Cebaf Center A-110
>>>>>>>> Call in information: Dial 1-877-643-6951
>>>>>>>> International access: 1-302-607-2017
>>>>>>>> Passcode: 24630562#
>>>>>>>>
>>>>>>>>
>>>>>>>> RPMs, yum, and all that (Maurizio)
>>>>>>>> * creating RPM (installable pre-compiled packages with
>>>>>>>> dependency
>>>>>>>> tree) of the CLAS software is a great idea.
>>>>>>>> * these RPMs can served with a JLab yum server so that
>>>>>>>> installation on a Fedora or Red Hat (or CentOS) system
>>>>>>>> could be
>>>>>>>> done with a line like:
>>>>>>>> yum install clas6-reconstruction clas6-generators etc...
>>>>>>>> * CLAS12 has already implemented this on several flavors of
>>>>>>>> Linux
>>>>>>>> *
>>>>>>>> Eugene's Review of SCons Build System
>>>>>>>> * Generally favourable review. Eugene points out that there are
>>>>>>>> many packages missing that are needed.
>>>>>>>> * Many packages will have to copied over directories from
>>>>>>>> CVS. * sconstruct files will have to be made.
>>>>>>>> * gamecock (TOF calib program) has some problems
>>>>>>>> (ROOT/GUI
>>>>>>>> related)
>>>>>>>> * CLASTool should be fairly easy to get to work now
>>>>>>>> that
>>>>>>>> ROOT dictionaries can be built properly.
>>>>>>>> From: Eugene Pasyuk <pasyuk at jlab.org>
>>>>>>>> To: CLAS Offline <clas_offline at jlab.org>
>>>>>>>> Subject: [Clas_offline] testing svn/scons
>>>>>>>> Date: Wed, 16 Sep 2009 10:29:31 -0400 (07:29 PDT)
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I tried new software tools today on ifarm. Here are some first
>>>>>>>> impressions about this experience.
>>>>>>>>
>>>>>>>> I followed prescription given on the Wiki:
>>>>>>>> http://clasweb.jlab.org/wiki/index.php/Building_CLAS_libraries_with_SCons_on_ifarm
>>>>>>>>
>>>>>>>> It took 20 minutes get the entire source tree source from svn.
>>>>>>>> Then I tried to build. The very first attempt failed. I figured
>>>>>>>> out that
>>>>>>>> the problem was in suggested environment setup file. I modified a
>>>>>>>> couple
>>>>>>>> of lines in it (2 minutes to figure out and fix). After that I made a
>>>>>>>> second attempt to run scons. This time it worked. It took about 3
>>>>>>>> minutes to build all clas libraries and a just few executables by
>>>>>>>> default:
>>>>>>>> a1c countbos g12Anal gsim_bat tagraw view-kaon view-tag
>>>>>>>> bosdump g12skim skim user_ana view-st view-tracks
>>>>>>>>
>>>>>>>> Out of these only few (a1c, gsim_bat, countbos, bosdump, user_ana and
>>>>>>>> tagraw) are of interest to general CLAS user. The rest seem to be g12
>>>>>>>> specific.
>>>>>>>> After doing some archeology in sconstruct files for about a half
>>>>>>>> an hour
>>>>>>>> and making some modifications to them here and there (I have zero
>>>>>>>> knowledge of scons or python at the moment) I managed to build a few
>>>>>>>> more utilities, particularly DC calibration tools trk_mon,
>>>>>>>> trk_mon_lite
>>>>>>>> and dc3. I failed to build start counter calibration code. I was
>>>>>>>> able to
>>>>>>>> build fdump. gflux would not build because of the compiler errors. I
>>>>>>>> wonder how g12 managed to build it in new build scheme from the
>>>>>>>> source
>>>>>>>> in svn. ced would not build because of the compiler errors too.
>>>>>>>> TOF calibration source code is in svn, but it was not modified for
>>>>>>>> scons
>>>>>>>> at all. It was imported as is from cvs with all Makefiles which
>>>>>>>> will not
>>>>>>>> work because of the different tree structure. sc_mon is missing.
>>>>>>>> As was
>>>>>>>> mentioned at the meeting ec timing calibration utilities are not
>>>>>>>> included as several other general purpose utilities. For some reason
>>>>>>>> gsim_int and gpp are not included.
>>>>>>>>
>>>>>>>> To summarize, I succeeded in building most of the libraries and a few
>>>>>>>> executables. Quite a few tools are missing (gsim_int, gpp, some
>>>>>>>> calibration tools). Some programs which build fine with make
>>>>>>>> would not
>>>>>>>> build with scons (gflux, ced ...?) I don't know if those
>>>>>>>> executables I
>>>>>>>> built are working, I had not had a chance to test them yet. I have
>>>>>>>> not
>>>>>>>> tried to build any of the event generators with scons.
>>>>>>>> By the way, there are only three of them in svn, while we have
>>>>>>>> more in
>>>>>>>> cvs.
>>>>>>>>
>>>>>>>> All in all, svn works just fine, apart from having different
>>>>>>>> directory
>>>>>>>> structure. scons works for some pieces of the software and does
>>>>>>>> not for
>>>>>>>> some
>>>>>>>> others. Perhaps it is not quite ready for the prime time yet. Some
>>>>>>>> essential
>>>>>>>> utilities are not working yet in new build. -Eugene
>>>>>>>>
>>>>>>>>
>>>>>>>> What We Should Do Now
>>>>>>>> * Hire some undergraduates to move packages from CVS to SVN and
>>>>>>>> add sconstruct files to build these packages.
>>>>>>>> * Armand Fraiser was a big help initially with the move to
>>>>>>>> SVN/SCons and he could continue to do so.
>>>>>>>> * There is some detailed help information already, it probably
>>>>>>>> needs a bit of reorganization on the wiki.
>>>>>>>> * Should we have a list of programs/libraries that should be
>>>>>>>> moved
>>>>>>>> from CVS to SVN (Armand already created an spreadsheet
>>>>>>>> that has
>>>>>>>> some of this information. It will be sent to Craig and he
>>>>>>>> will
>>>>>>>> look into it.
>>>>>>>> * We need to freeze CVS today.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Clas_offline mailing list
>>>>>>>> Clas_offline at jlab.org
>>>>>>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>> _______________________________________________
>>>> Clas_offline mailing list
>>>> Clas_offline at jlab.org
>>>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>>>
>>>>
>> _______________________________________________
>> Clas_offline mailing list
>> Clas_offline at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/clas_offline
>>
>
>
More information about the Clas_offline
mailing list