[Halld-offline] Confusion about JANA build

[David Lawrence]

Hi Matt,

On 3/12/11 1:45 PM, Matthew Shepherd wrote:
> Hi David,
>
> I recently did svn update on my copy of jana.
Let me stop you right there. You should never, ever, ever, under any 
circumstances (other developing JANA code itself), update from the JANA 
repository. If you are compiling JANA to use for Hall-D software, then 
you should always be using a tagged release built from a tarball 
distributed from the JANA website.
>   Then I went into the src directory and typed "make clean; make".
Good. Whenever you do an update from the repository, it is almost always 
a good idea to do a make clean before doing anything else. The fact that 
we use an automatic build system (one that dynamically discovers files 
at the time make is invoked) means one should always regenerate 
dependency files whenever directory structure and/of file existence 
status has changed as often happens with and svn update.

>   The build started but then failed on one of the utilities.  I tracked the problem down to the fact that the build was using src/../include as the include path.  This is the location of the installed headers.  Apparently make does not install the headers before trying to build the executables so I was trying to build executables with old headers.  There was a function mismatch and the build failed with "no matching function" error.
I'm not sure this is correct. When invoking make at the top level, it 
invokes make in "src" which then invokes make in JANA. That Makefile in 
src/JANA includes Makefile.lib which has the first target as:

install: all
     mkdir -p $(JANA_BUILD_DIR)/lib
     mkdir -p $(JANA_BUILD_DIR)/include/$(MODULE_NAME)
     install -p $(LIBNAME) $(JANA_BUILD_DIR)/lib
     install -p $(wildcard *.h *.hh *.hxx) 
$(JANA_BUILD_DIR)/include/$(MODULE_NAME)

The "all" target actually builds the library, but the commands that make 
up the "install" target that are executed immediately after "all" 
install the libraries and headers at the same time into the build 
directory. This guarantees two things before attempting to build any 
utilities or plugins:

1.) That libJANA.a and the header files installed into the build 
directory are consistent
2.) The header files installed in the build directory are the same as 
those in the src/JANA directory

After the JANA library is built, make will then leave the JANA directory 
and invoke make in the utilities and plugins directories respectively.

> So I remember a conversation we had a long time ago about the difference between make and make install and then go back to src and type:  "make clean; make install"  This time the build works fine.
>
> I notice in both cases the include path passed to the compiler is the same and this include path is passed to both library compilation and executable compilation.
>
> g++ -fPIC -I. -I.. -I/Users/mashephe/gluex/svn/jana/prod/src/../include
>
> When one is building the JANA library itself, one is relying on the fact that all headers are accessible from . or .. (which is the case) to avoid pulling in old headers installed in src/../include.  However, when the build moves on to the utilities the paths . and .. and no longer are sufficient to get headers (One needs ../.. also.) and old stuff comes in through src/../include.
>
So, I guess there is something I'm missing here since the structure I 
describe above and observed just now when building JANA 0.6.3 is that 
you can't really have old stuff in src/../include by the time you get to 
src/utilities. Could you give a few more details on exactly the commands 
I would need to do to reproduce the behavior you're seeing?

> Luckily my build actually failed due to a missing function to alert me to the problem.  For example, if the function ordering was different in the header then make would work fine and I'd have an incompatibility between the memory layouts in my compiled code and the JANA library.
>
> This very thing has made me nervous in builds of the main tree many times.  I don't like the fact that when I do a make inside of the tree I see the build is looking for headers in the install locations.  I'm trusting that it will find the up to date headers first not the ones from my previously installed build.  Often may paranoia will lead me to go in and wipe out all previously built headers and libraries manually.
>
The Hall-D build system has been deviating from the JANA build system 
for some time.

> I wonder if we can modify the system to make using only the source tree.  For example, the jana tree should build by issuing a single make command without reference to install locations.  A second install target would install headers and libraries needed to compile other external packages that use jana.  (This seems to be the convention for most distributed packages that I know of.)  I believe this means that the jana executables need to know where to find their appropriate libraries and headers within the tree -- they can't resort to a common install location.

This is actually the direction that JANA has been moving and I thought 
we were pretty to close to that goal. One thing to keep in mind is that 
the "install" target defined in Makefile.lib (and described above) only 
installs the library and headers into the top of the build tree. This is 
not the same as installing into the final working area outside of the 
build tree where external projects like the Hall-D source will use it. 
That "install" is done only when the user explicitly requests it by 
invoking "make install". It may be that if one defines the final 
installation directory somewhere in the build directory via the 
--prefix=XXX argument to ./configure that you could confuse the system. 
Perhaps that needs to be checked for and somehow handled better.
> It would be nice to see something similar for the main halld tree.  This would guarantee that each build was using fresh headers and libraries, not previously installed.
>
I've been wanting to move the Hall-D source over to using autoconf and 
perhaps simultaneously tighten up the build system. The Hall-D source is 
a little different in nature from JANA though. With JANA, we can treat 
it as a purely 3rd party package like Xerces or ROOT. The Hall-D source 
is going to be more of a user malleable entity so may need a more 
flexible build system. We should add this to the agenda of the next 
offline meeting.


-Dave