[Rcpp-devel] Getting different build commands on Linux vs. Mac OS
Evan Biederstedt
evan.biederstedt at gmail.com
Thu May 3 22:19:20 CEST 2018
Hi @Keith
Thanks for the help and apologies for the delay. You're correct---when I
change the install name it does work, i.e. use `sudo install_name_tool -id
/usr/local/lib/libbamdb.dylib /usr/local/lib/libbamdb.dylib`
Thanks for this! I hadn't known about `@rpath` in Mac OS, and in retrospect
I should have done more work looking into the properties of these shared
libraries. And thanks for the list, as this wasn't an Rcpp issue at all.
This segues to the next step, which is Rcpp-related: compiling this
together into one R package. Combine bamdb + bambi into one R package.
@list
Working with Linux first, is it common to compile a C library (i.e. bamdb)
within /src, and then use extern? My understanding is that I will have to
change the R package's Makevars to compile the C library first, and then
link to this.
If there are any examples on github which you would be able to recommend,
that might be best. Then I can close this thread :)
Thank you for the help!
On Tue, May 1, 2018 at 1:26 PM, Keith O'Hara <kth.ohr at gmail.com> wrote:
> From my experience, R (on macOS) doesn't play well with libraries that
> have install names containing runtime search paths. By changing the install
> name, I mean changing the '@rpath' part here:
>
> $ otool -L /usr/local/lib/libbamdb.dylib
>
> /usr/local/lib/libbamdb.dylib:
> @rpath/libbamdb.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> /usr/local/lib/libck.0.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> /opt/local/lib/libhts.2.dylib (compatibility version 2.0.0,
> current version 1.8.0)
> /opt/local/lib/liblmdb.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> ...
>
> using:
>
> $ sudo install_name_tool -id /usr/local/lib/libbamdb.dylib
> /usr/local/lib/libbamdb.dylib
> $ otool -L /usr/local/lib/libbamdb.dylib
> /usr/local/lib/libbamdb.dylib:
> /usr/local/lib/libbamdb.dylib (compatibility version 0.0.0,
> current version 0.0.0)
> /usr/local/lib/libck.0.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> /opt/local/lib/libhts.2.dylib (compatibility version 2.0.0,
> current version 1.8.0)
> /opt/local/lib/liblmdb.dylib (compatibility version 0.0.0, current
> version 0.0.0)
> ...
>
> Then build the R package (R CMD INSTALL …). Otherwise you could hit an
> 'image not found' error. (But, to be clear, this error is not related to
> Rcpp.)
>
> Keith
>
> > On May 1, 2018, at 12:04 PM, Evan Biederstedt <
> evan.biederstedt at gmail.com> wrote:
> >
> > @Dirk
> >
> > > Could you possibly construct a smaller, self-contained example
> exhibiting the
> > same problem? That may be more efficient than requiring volunteer
> helpers to
> > install four other libraries.
> >
> > Apologies, this is on the to-do list.
> >
> > @Keith
> >
> > Thanks for trying this out. This is interesting.
> >
> > > Worked for me, after I modified the install name of libbamdb (from a
> runtime path to the installation directory, /usr/local/lib/libbamdb.dylib)
> .
> >
> > I want to be sure I understand---Could you clarify this? Which changes
> did you make?
> >
> > That is created with `sudo make install`, i.e. from above
> >
> > ```
> > $ sudo make install
> > Password:
> > [ 50%] Built target bamdb
> > [100%] Built target libbamdb
> > Install the project...
> > -- Install configuration: ""
> > -- Installing: /usr/local/bin/bamdb
> > -- Up-to-date: /usr/local/include/bamdb
> > -- Up-to-date: /usr/local/include/bamdb/bam_api.h
> > -- Up-to-date: /usr/local/include/bamdb/bam_lmdb.h
> > -- Up-to-date: /usr/local/include/bamdb/bamdb.h
> > -- Installing: /usr/local/lib/libbamdb.dylib
> > ```
> >
> >
> >
> >
> >
> > On Tue, May 1, 2018 at 11:42 AM, Keith O'Hara <kth.ohr at gmail.com> wrote:
> > Worked for me, after I modified the install name of libbamdb (from a
> runtime path to the installation directory, /usr/local/lib/libbamdb.dylib)
> .
> >
> > * installing to library ‘/Users/<blah blah>/R-devel/lib/R/library’
> > * installing *source* package ‘bambi’ ...
> > ** libs
> > clang++-mp-6.0 -std=gnu++11 -I"/Users/<blah blah>/R-devel/lib/R/include"
> -DNDEBUG -I../inst/include/ -I/usr/local/include/bamdb -I/opt/local/include
> -I"/Users/<blah blah>/R-devel/lib/R/library/Rcpp/include"
> -I/usr/local/include -fPIC -Wall -march=native -g -O3 -ffp-contract=fast
> -c RcppExports.cpp -o RcppExports.o
> > clang++-mp-6.0 -std=gnu++11 -I"/Users/<blah blah>/R-devel/lib/R/include"
> -DNDEBUG -I../inst/include/ -I/usr/local/include/bamdb -I/opt/local/include
> -I"/Users/<blah blah>/R-devel/lib/R/library/Rcpp/include"
> -I/usr/local/include -fPIC -Wall -march=native -g -O3 -ffp-contract=fast
> -c bambi.cpp -o bambi.o
> > clang++-mp-6.0 -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names
> -undefined dynamic_lookup -single_module -multiply_defined suppress
> -L/Users/<blah blah>/R-devel/lib/R/lib -L/usr/local/lib -o bambi.so
> RcppExports.o bambi.o -lbamdb -L/opt/local/lib -llmdb -lhts -L/Users/<blah
> blah>/R-devel/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
> > installing to /Users/<blah blah>/R-devel/lib/R/library/bambi/libs
> > ** R
> > ** byte-compile and prepare package for lazy loading
> > ** help
> > *** installing help indices
> > ** building package indices
> > ** testing if installed package can be loaded
> > * DONE (bambi)
> >
> > Try building from a new, clean directory.
> >
> > Keith
> >
> > > On May 1, 2018, at 10:24 AM, Evan Biederstedt <
> evan.biederstedt at gmail.com> wrote:
> > >
> > > @Keith
> > >
> > > > Unless I'm mistaken, neither are required to install binary
> packages. Can you check your *source* build setup; try:
> > >
> > > > curl -O https://cran.r-project.org/src/contrib/RcppArmadillo_0.8.500
> .0.tar.gz
> > > > R CMD INSTALL RcppArmadillo_0.8.500.0.tar.gz
> > >
> > > This looks successful
> > >
> > > ```
> > > $ R CMD INSTALL RcppArmadillo_0.8.500.0.tar.gz
> > >
> > > * installing to library ‘/Library/Frameworks/R.framewo
> rk/Versions/3.4/Resources/library’
> > > * installing *source* package ‘RcppArmadillo’ ...
> > > ** package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked
> > > checking whether the C++ compiler works... yes
> > > checking for C++ compiler default output file name... a.out
> > > checking for suffix of executables...
> > > checking whether we are cross compiling... no
> > > checking for suffix of object files... o
> > > checking whether we are using the GNU C++ compiler... yes
> > > checking whether clang++ accepts -g... yes
> > > checking how to run the C++ preprocessor... clang++ -E
> > > checking whether we are using the GNU C++ compiler... (cached) yes
> > > checking whether clang++ accepts -g... (cached) yes
> > > checking whether g++ version is sufficient... almost
> > > configure: WARNING: Compiler self-identifies as being compliant with
> GNUC extensions but is not g++.
> > > checking for macOS... found
> > > checking for macOS Apple compiler... found
> > > configure: WARNING: OpenMP unavailable and turned off.
> > > checking LAPACK_LIBS... R-supplied partial LAPACK found
> > > configure: WARNING: Some complex-valued LAPACK functions may not be
> available
> > > configure: creating ./config.status
> > > config.status: creating inst/include/RcppArmadilloConfigGenerated.h
> > > config.status: creating src/Makevars
> > > ** libs
> > > clang++ -std=gnu++11 -I/Library/Frameworks/R.framework/Resources/include
> -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcpp/include"
> -I/usr/local/include -I../inst/include -fPIC -Wall -g -O2 -c
> RcppArmadillo.cpp -o RcppArmadillo.o
> > > clang++ -std=gnu++11 -I/Library/Frameworks/R.framework/Resources/include
> -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcpp/include"
> -I/usr/local/include -I../inst/include -fPIC -Wall -g -O2 -c
> RcppExports.cpp -o RcppExports.o
> > > clang++ -std=gnu++11 -I/Library/Frameworks/R.framework/Resources/include
> -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcpp/include"
> -I/usr/local/include -I../inst/include -fPIC -Wall -g -O2 -c fastLm.cpp
> -o fastLm.o
> > > clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names
> -undefined dynamic_lookup -single_module -multiply_defined suppress
> -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o
> RcppArmadillo.so RcppArmadillo.o RcppExports.o fastLm.o
> -L/Library/Frameworks/R.framework/Resources/lib -lRlapack
> -L/Library/Frameworks/R.framework/Resources/lib -lRblas
> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0
> -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
> -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework
> -Wl,CoreFoundation
> > > ld: warning: directory not found for option
> '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
> > > ld: warning: directory not found for option '-L/usr/local/gfortran/lib'
> > > installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/
> library/RcppArmadillo/libs
> > > ** R
> > > ** inst
> > > ** preparing package for lazy loading
> > > ** help
> > > *** installing help indices
> > > ** building package indices
> > > ** installing vignettes
> > > ** testing if installed package can be loaded
> > > * DONE (RcppArmadillo)
> > >
> > > ```
> > >
> > > > Check your build directory for any .Rprofile or .Renviron files, and
> also check ~/.R/Makevars for errors.
> > >
> > > I'm using the command line. I do not seem to have anything in `$ cat
> ~/.R/Makevars` or `$cat ~/.Rprofile`. Perhaps this is the problem?
> > >
> > >
> > > > If you can list the required libraries for bamdb, I'm happy to try a
> build on my end. (I'm lazy to go through the CMake list :-))
> > >
> > > There are four dependencies:
> > >
> > > htslib/1.5 or higher: https://github.com/samtools/htslib
> > >
> > > LMDB key-value store: https://github.com/LMDB/lmdb
> > >
> > > Concurreny-Ck: https://github.com/concurrencykit/ck
> > >
> > > zlib: Standard compression library https://github.com/madler/zlib
> (probably already installed)
> > >
> > > Mac OS:
> > > ```
> > > brew install lmdb
> > > brew install concurrencykit
> > > brew install zlib // mac OS already provides this software
> > > brew install htslib // I *think* this is the latest htslib, but I
> would double-check
> > > ```
> > >
> > > Linux (for the record):
> > > ```
> > > yum install libhts-dev ## dev branch of latest htslib
> > > yum install liblmdb-dev ## LMDB dev
> > > yum install libck-dev ## Concurrency-ck
> > > yum install libz-dev ## zlib compresssion
> > > ```
> > >
> > > On Tue, May 1, 2018 at 3:45 AM, Keith O'Hara <kth.ohr at gmail.com>
> wrote:
> > > Unless I'm mistaken, neither are required to install binary packages.
> Can you check your *source* build setup; try:
> > >
> > > curl -O https://cran.r-project.org/src/contrib/RcppArmadillo_0.8.500
> .0.tar.gz
> > > R CMD INSTALL RcppArmadillo_0.8.500.0.tar.gz
> > >
> > > If that goes through without a hitch then it's probably a package
> issue. Are you using RStudio? Check your build directory for any .Rprofile
> or .Renviron files, and also check ~/.R/Makevars for errors.
> > >
> > > If you can list the required libraries for bamdb, I'm happy to try a
> build on my end. (I'm lazy to go through the CMake list :-))
> > >
> > > Keith
> > >
> > > > On May 1, 2018, at 3:07 AM, Evan Biederstedt <
> evan.biederstedt at gmail.com> wrote:
> > > >
> > > > I would add, I think clang and gfortran are working correctly, as
> these are required for `install.packages('RcppArmadillo')` to install
> properly.
> > > >
> > > > On Tue, May 1, 2018 at 3:06 AM, Evan Biederstedt <
> evan.biederstedt at gmail.com> wrote:
> > > > Maybe my Mac OS needs updating...possibly something with XCode, but
> it's not clear how...
> > > >
> > > > --- mac OS Sierra 10.12.6
> > > > --- CRAN R 3.4.3
> > > > ---clang & gfortran
> > > >
> > > > $ clang --version
> > > > Apple LLVM version 9.0.0 (clang-900.0.39.2)
> > > > Target: x86_64-apple-darwin16.7.0
> > > > Thread model: posix
> > > > InstalledDir: /Library/Developer/CommandLineTools/usr/bin
> > > >
> > > > $ gfortran --version
> > > > GNU Fortran (GCC) 7.1.0
> > > > Copyright (C) 2017 Free Software Foundation, Inc.
> > > > This is free software; see the source for copying conditions. There
> is NO
> > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
> > > >
> > > > I will update once I have access to and try this on several other
> mac boxes with different specs..
> > > >
> > > > Thank you to everyone for the help.
> > > >
> > > >
> > > > On Tue, May 1, 2018 at 1:44 AM, Ralf Stubner <
> ralf.stubner at r-institute.com> wrote:
> > > >
> > > > Am 30.04.2018 um 23:42 schrieb Evan Biederstedt <
> evan.biederstedt at gmail.com>:
> > > >
> > > >> @Ralf
> > > >>
> > > >> > I had not installed the necessary C library, so unsurprisingly
> the compilation step already failed. What I find interesting is that in my
> case "-std=gnu++11“ and "-I../inst/include/" are present in the command
> line options for clang++. These are also missing from what you quoted and
> are also a consequence of src/Makevars. It almost looks as if this file got
> lost on your macOS box …
> > > >>
> > > >> Interesting....I'm not sure what to make of this of course, but it
> would be interesting if it was only happening on this macbook. Perhaps OS
> version or brew could be to blame? I'm not sure what else would be
> "special" about this macbook....
> > > >
> > > > On my side that would be
> > > > • macOS high Sierra
> > > > • CRAN R 3.4.4
> > > > • clang and gfortran as recommended by CRAN
> > > > • brew is installed but I do not see how it enters the picture
> > > >
> > > > Greetings
> > > > Ralf
> > > >
> > > >
> > > > _______________________________________________
> > > > Rcpp-devel mailing list
> > > > Rcpp-devel at lists.r-forge.r-project.org
> > > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo
> /rcpp-devel
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180503/577b5386/attachment-0001.html>
More information about the Rcpp-devel
mailing list