[GenABEL-dev] m4_ax_prog_cxx_mpi.m4
L.C. Karssen
lennart at karssen.org
Thu Oct 30 20:41:42 CET 2014
Hi Alvaro,
I'm sorry that the whole Autotools concept makes life difficult for you.
Let me try to explain it and feel free to ask more questions if they
arise. If you'd like we can also schedule a call (with Jürgen if he is
interested) where I can explain it some more.
Let me first answer the question in this e-mail before diving into your
questions of the previous one.
On 30-10-14 16:43, Frank, Alvaro Jesus wrote:
> What I seem to understand is that distcheck includes openblas in the
> distribution tar file? So that is why I needed when creating it?
No, that is not the case. 'make dist' (which is also run when running
'make distcheck') creates a PACKAGENAME-VERSION.tar.gz file (where
PACKAGENAME and VERSION are defined in the AC_INIT command in configure.ac
The idea is that the tar.gz file contains all the code/files/documents
you wrote that are needed by a user/sysadmin to compile and install the
tool. The idea is that it is packaged in such a way that it works on any
POSIX or POSIX-like system (Linux, Solaris, OS X, etc).
Since we don't have any BLAS included in the OAnoMM source tree in SVN
it won't be packaged in the tar.gz file either. We also don't package
e.g. the stdlib, Boost, etc. We leave it up to the user to provide these
libraries. Since these are all well known libraries that are packaged by
most if not all Linux distro's this should not be a problem. In fact, it
is good design. If you want to know why, please ask.
Moreover, in the case of OAnoMM the code is written so close to the
hardware that you have always been a proponent of compilation of BLAS
libs by the user/sysadmin himself to make sure of the proper
architecture-dependent options.
Basically in our tar.gz file we only include code for which we are
responsible.
Note: the files to be included in the tar.gz file are not simply all the
files in SVN. Every file that needs to be included should be explicitly
mentioned in the Automake files (Makefile.am). See the variables ending
in _SOURCES and those starting with dist_.
This does mean the user should have a BLAS lib installed in a standard
location (e.g. /usr/include). This way ./configure can detect whatever
we need. In configure.ac we have a check that looks for an MPI compiler,
one to see if Boost's distribution.hpp can be found, but also whether
tools like mkdir -p function as expected, whether Lapack can be found, etc.
If one of these cannot be found some action is taken. In the case of MPI
we continue without it. In the case of Boost we allow the user to
specify an alternate location of the include files via the
--with-boost-include-path= option of ./configure.
We do check for Lapack or a BLAS lib, but haven't added the autoconf
logic (configure.ac) to specify an alternative path. I think we should
(see the CPPFLAGS discussion below), moreover, the Autoconf Archive has
a macro for BLAS and Lapack [5,6], so it shouldn't be too hard.
If all is in the default locations, like it is on my Ubuntu system
(Lapack, Boost, OpenBLAS installed via the package manager) it is enough
to run
./configure
make
sudo make install
to compile and install OAnoMM. I don't even have to look at the
howToCompile.txt file. That's how I like my software :-).
If libraries like BLAS are not in the default location the user can add
LDFLAGS options to the compiler like this:
./configure LDFLAGS=-L$HOME/path/to/my/own/boost/copy
A path for including headers can be specified in the same way, there is
one catch though: In lines 19-30 of configure.ac we fill in some default
CPPFLAGS and CXXFLAGS. These are only taken into account if these
variables don't already exist (e.g. by specifying them like above). So
it is important to understand that if the user runs
./configure CXXFLAGS=-I../some/path
our defaults (-O3 -std=c++11 etc.) will be overwritten! This is by
design, because an experienced user should be allowed to set all the
options as he sees best. See also the last few lines of ./configure
--help. More info at [1,2,3].
For example, this allows me to cross-compile ProbABEL for Windows like
this:
./configure --host=i686-w64-mingw32 --build=i686-linux-gnu
make CFLAGS+="-O2 -static-libgcc -static-libstdc++" \
CXXFLAGS+="-O2 -static-libgcc -static-libstdc++"
As a side note: there is a case where I don't want the user to be able
to override one part of the CPPFLAGS. That's in tests/Makefile.am where
I add the -DDATADIR macro to the CPPFLAGS specific to the test binary
(line 11). This should always be there (and has no effect on
optimisation or things like that). The same holds for the addition of
-DUSE_MPI to the filename_CPPFLAGS, as the user can disable that by
running ./configure --without-mpi.
By the way, I did notice now that in line 69 of configure.ac you added
the -I../libs/include -I./libs/include options to the CXXFLAGS. I guess
this has to do with your own setup where you have a specific BLAS in on
of those directories. These should be removed as they can only create
problems in case a user has other stuff in those directories. An
experienced user doesn't expect specific stuff like that in the default
compiler options. I removed them in r1879.
Moreover, in that line you also specified -std=c++11 and -O3 again (they
are already specified in line 22). I have removed them in r1878.
Thanks for asking this question! I've learned a lot again from rereading
the documentation, searching the web and testing a few things :-). For
example the change in r1880 [4].
[1]
http://stackoverflow.com/questions/3116645/default-compiler-flags-with-autotools
[2]
http://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html
[3]
http://stackoverflow.com/questions/7467910/with-autoconf-automake-how-do-i-specify-include-file-paths
[4]
http://stackoverflow.com/questions/20230827/how-to-set-include-paths-with-autotools
[5] http://www.gnu.org/software/autoconf-archive/ax_blas.html
[6] http://www.gnu.org/software/autoconf-archive/ax_lapack.html
>
> I thought that users of the program had to specify their own BLAS to use.
>
> -Alvaro
>
> ________________________________________
> From: genabel-devel-bounces at lists.r-forge.r-project.org [genabel-devel-bounces at lists.r-forge.r-project.org] on behalf of Frank, Alvaro Jesus [alvaro.frank at rwth-aachen.de]
> Sent: Thursday, October 30, 2014 3:31 PM
> To: L.C. Karssen; genabel-devel at lists.r-forge.r-project.org
> Cc: juergen.riegel at helmholtz-muenchen.de
> Subject: Re: [GenABEL-dev] m4_ax_prog_cxx_mpi.m4
>
> I am really not an autoconf user, but I am failing to do the make distcheck. I do have openblas.
> I am struggling as I am not the sysadmin. The end user IS the sysadmin.
>
> I am having trouble grasping this:
> They will choose the openblas,boost ,gcc,mpi,etc, path right?
>
> So cant they just checkout the src and compile and obtain the binary normally?
>
> My error:
>
> make distcheck
> { test ! -d "omicabelnomm-0.1.0" || { find "omicabelnomm-0.1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -fr "omicabelnomm-0.1.0"; }; }
> test -d "omicabelnomm-0.1.0" || mkdir "omicabelnomm-0.1.0"
> test -n "" \
> || find "omicabelnomm-0.1.0" -type d ! -perm -755 \
> -exec chmod u+rwx,go+rx {} \; -o \
> ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
> ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
> ! -type d ! -perm -444 -exec /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/install-sh -c -m a+r {} {} \; \
> || chmod -R a+r "omicabelnomm-0.1.0"
> tardir=omicabelnomm-0.1.0 && /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar chof - "$tardir" | GZIP=--best gzip -c >omicabelnomm-0.1.0.tar.gz
> { test ! -d "omicabelnomm-0.1.0" || { find "omicabelnomm-0.1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -fr "omicabelnomm-0.1.0"; }; }
> case 'omicabelnomm-0.1.0.tar.gz' in \
> *.tar.gz*) \
> GZIP=--best gzip -dc omicabelnomm-0.1.0.tar.gz | /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar xf - ;;\
> *.tar.bz2*) \
> bzip2 -dc omicabelnomm-0.1.0.tar.bz2 | /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar xf - ;;\
> *.tar.lzma*) \
> lzma -dc omicabelnomm-0.1.0.tar.lzma | /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar xf - ;;\
> *.tar.xz*) \
> xz -dc omicabelnomm-0.1.0.tar.xz | /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar xf - ;;\
> *.tar.Z*) \
> uncompress -c omicabelnomm-0.1.0.tar.Z | /bin/sh /home/alvaro.frank/omicabelnomm/OmicABELnoMM/missing --run tar xf - ;;\
> *.shar.gz*) \
> GZIP=--best gzip -dc omicabelnomm-0.1.0.shar.gz | unshar ;;\
> *.zip*) \
> unzip omicabelnomm-0.1.0.zip ;;\
> esac
> chmod -R a-w omicabelnomm-0.1.0; chmod a+w omicabelnomm-0.1.0
> mkdir omicabelnomm-0.1.0/_build
> mkdir omicabelnomm-0.1.0/_inst
> chmod a-w omicabelnomm-0.1.0
> test -d omicabelnomm-0.1.0/_build || exit 0; \
> dc_install_base=`CDPATH="${ZSH_VERSION+.}:" && cd omicabelnomm-0.1.0/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
> && dc_destdir="${TMPDIR-/tmp}/am-dc-$$/" \
> && am__cwd=`pwd` \
> && CDPATH="${ZSH_VERSION+.}:" && cd omicabelnomm-0.1.0/_build \
> && ../configure --srcdir=.. --prefix="$dc_install_base" \
> \
> && make \
> && make dvi \
> && make check \
> && make install \
> && make installcheck \
> && make uninstall \
> && make distuninstallcheck_dir="$dc_install_base" \
> distuninstallcheck \
> && chmod -R a-w "$dc_install_base" \
> && ({ \
> (cd ../.. && umask 077 && mkdir "$dc_destdir") \
> && make DESTDIR="$dc_destdir" install \
> && make DESTDIR="$dc_destdir" uninstall \
> && make DESTDIR="$dc_destdir" \
> distuninstallcheck_dir="$dc_destdir" distuninstallcheck; \
> } || { rm -rf "$dc_destdir"; exit 1; }) \
> && rm -rf "$dc_destdir" \
> && make dist \
> && rm -rf omicabelnomm-0.1.0.tar.gz \
> && make distcleancheck \
> && cd "$am__cwd" \
> || exit 1
> checking for a BSD-compatible install... /usr/bin/install -c
> checking whether build environment is sane... yes
> checking for a thread-safe mkdir -p... /bin/mkdir -p
> checking for gawk... gawk
> checking whether make sets $(MAKE)... yes
> checking whether to disable maintainer-specific portions of Makefiles... yes
> checking for mpic++... mpic++
> checking whether the C++ compiler works... yes
> checking for C++ compiler default output file name... a.out
> checking for suffix of executables...
> checking whether we are cross compiling... no
> checking for suffix of object files... o
> checking whether we are using the GNU C++ compiler... yes
> checking whether mpic++ accepts -g... yes
> checking for style of include used by make... GNU
> checking dependency style of mpic++... gcc3
> checking dependency style of mpic++... (cached) gcc3
> checking for gcc... gcc
> checking whether we are using the GNU C compiler... yes
> checking whether gcc accepts -g... yes
> checking for gcc option to accept ISO C89... none needed
> checking dependency style of gcc... gcc3
> checking dependency style of gcc... (cached) gcc3
> checking for gcc option to support OpenMP... -fopenmp
> checking for library containing pthread_mutex_init... none required
> checking for library containing __iso_c_binding_c_f_pointer_l4... -lgfortran
> checking for library containing cblas_sgemm... no
> configure: error: OpenBLAS library NOT found
> make: *** [distcheck] Error 1
>
> ________________________________________
> From: L.C. Karssen [lennart at karssen.org]
> Sent: Thursday, October 30, 2014 3:23 PM
> To: Frank, Alvaro Jesus; genabel-devel at lists.r-forge.r-project.org
> Subject: Re: m4_ax_prog_cxx_mpi.m4
>
> Hi Alvaro,
>
> On 30-10-14 15:16, Frank, Alvaro Jesus wrote:
>> Hi Lennart,
>>
>> Can we include m4_ax_prog_cxx_mpi.m4 on the root directory with our svn?
>
> We can. I'm not too much a fan of including code that is maintained
> elsewhere and packaged properly by Linux distributions, but if you think
> it might otherwise deter people from contributing, be my guest.
>
> But keep in mind that a user shouldn't need to have that file (because
> autoreconf will include it in the ./configure script, which is packaged
> in the tar.gz file). Only a developer/someone who checks out from SVN/a
> user who runs autoreconf after unpacking the source tar.gz file needs
> that file.
>
> As to the location: what do you mean with root directory? If you mean
> the root of the OAnoMM package (pkg/OmicABELnoMM/) than it's fine with me.
>
>
> Lennart.
>
>>
>> Best,
>>
>> -Alvaro
>
> --
> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
> L.C. Karssen
> Utrecht
> The Netherlands
>
> lennart at karssen.org
> http://blog.karssen.org
> GPG key ID: A88F554A
> -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
>
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands
lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20141030/dab3d91e/attachment-0001.sig>
More information about the genabel-devel
mailing list