[Phylobase-devl] Rcpp and OS X compiliation

Dirk Eddelbuettel edd at debian.org
Sun Jan 17 05:17:15 CET 2010 

Hi Mark,

On 16 January 2010 at 20:57, Mark Holder wrote:
| Hi again,
| 
| On Jan 15, 2010, at 11:16 AM, Ben Bolker wrote:
| 
| > Mark Holder wrote:
| >>
| >> I'm not sure how to fix it other than altering the user's env (which
| >> is obviously not ideal).
| >
| >  No, but it might be an interim hack: R has Sys.getenv() and
| > Sys.setenv() which might be usable ?  (although they might have to be
| > called too early in the package-loading process)
| 
| I am not having any success getting the library to load by modifying  
| DYLD_LIBRARY_PATH from within R.

Seconded. I played with that a looong time ago and concluded that these
environment variables are 'sealed' once you are running your process. You
cannot modify your own process. Which is probably why the (shell script)
/usr/bin/R sets them up before calling the underlying binary.

| I can augment the list of paths successfully (with  
| Sys.setenv(DYLD_LIBRARY_PATH="/Us..."),
| but that doesn't seem to help the loader.  I suspect that you are  
| correct that using setenv is simply too late.

Agreed.
 
| According to "Writing R extensions
| ${R_HOME}/etc/${R_ARCH}/ldpaths is sourced before the R process is  
| launched.

Exactly.
 
| On my machine this is:
| 	/Library/Frameworks/R.framework/Resources/etc/i386/ldpaths
| 
| This file adds ${R_HOME}/lib/${R_ARCH} to the dynamic load library  
| path list (one an intel-mac this is /Library/Frameworks/R.framework/ 
| Resources/lib/i386). This directory contains libR.dylib,  
| libRblas.dylib, and  libRlapack.dylib on my machine.
| 
| I can get phylobase to run if I do any one of the  following four  
| things:
| 	1. Add ~/Library/R/2.10/library/Rcpp/lib/i386 to the user's env  
| before calling R
| 	2. Add ~/Library/R/2.10/library/Rcpp/lib/i386 to DYLD_LIBRARY_PATH in  
| the /Library/Frameworks/R.framework/Resources/etc/i386/ldpaths file
| 	3. Copy ~/Library/R/2.10/library/Rcpp/lib/i386/libRcpp.dylib to  / 
| Library/Frameworks/R.framework/Resources/lib/i386
| 	4. Copy ~/Library/R/2.10/library/Rcpp/lib/i386/libRcpp.dylib to /usr/ 
| local/lib

Hm. None of 1 to 4 should be necessary if the 'rpath' option works correctly.
Does that exist / work on OS X?  Otherwise I have a problem with Rcpp that is
much larger than phylobase :-/ --- and I am surprised that has not come up
before.  As AFAIK the build bot on R-Forge doesn't use this either.

I would still like to disentangle things, if possible.  

Could you have a look at the baby package 'earthmovdist' from R-Forge?  I
attach a tarball for your persusal.  It simple uses Rcpp to get to a simple
C++ class.  No external library, but of course a dependency on Rcpp's
library.   On my system this builds as

edd at ron:~/svn/earthmovdist$ R CMD INSTALL earthmovdist_0.1.0.tar.gz
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘earthmovdist’ ...
** libs
g++ -I/usr/share/R/include -I/usr/local/lib/R/site-library/Rcpp/lib     -fpic  -g -O3 -Wall -pipe -c emdL1.cpp -o emdL1.o
g++ -I/usr/share/R/include -I/usr/local/lib/R/site-library/Rcpp/lib     -fpic  -g -O3 -Wall -pipe -c emdL1_R.cpp -o emdL1_R.o
g++ -shared -o earthmovdist.so emdL1.o emdL1_R.o -L/usr/local/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/local/lib/R/site-library/Rcpp/lib -L/usr/lib/R/lib -lR
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
* DONE (earthmovdist)
edd at ron:~/svn/earthmovdist$

and I can then say 

edd at ron:~/svn/earthmovdist$ Rscript -e 'library(earthmovdist); example(emdL1)'
Loading required package: Rcpp
Loading required package: methods

emdL1R>   set.seed(42)

emdL1R>   x <- rnorm(100)

emdL1R>   y <- x + rnorm(100)/10

emdL1R>   emdL1(x, y, verbose=TRUE)
1D - EmdL1(h1,h2)=68.420658
[1] 68.42
edd at ron:~/svn/earthmovdist$

as

edd at ron:~/svn/earthmovdist$ ldd /usr/local/lib/R/site-library/earthmovdist/libs/earthmovdist.so
        linux-gate.so.1 =>  (0xb7f67000)
        libRcpp.so => /usr/local/lib/R/site-library/Rcpp/lib/libRcpp.so (0xb7f2c000)
        libR.so => /usr/lib/R/lib/libR.so (0xb7bfb000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7b09000)
        libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7ae3000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7ac6000)
        libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb797f000)
        libblas.so.3gf => /usr/lib/atlas/libblas.so.3gf (0xb7604000)
        libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0xb753e000)
        libreadline.so.6 => /lib/libreadline.so.6 (0xb7509000)
        libpcre.so.3 => /lib/libpcre.so.3 (0xb74d9000)
        libbz2.so.1.0 => /lib/libbz2.so.1.0 (0xb74c9000)
        libz.so.1 => /usr/lib/libz.so.1 (0xb74b5000)
        libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb74b1000)
        /lib/ld-linux.so.2 (0xb7f68000)
        libncurses.so.5 => /lib/libncurses.so.5 (0xb7478000)
edd at ron:~/svn/earthmovdist$

That would be the same for phylobase (in the sense that it would know where
to find libRcpp.so), plus we still would have to deal with NCL.

Now, I just checked on R-Forge on earthmovdist did NOT build cleanly the last
time which has me puzzled.  This should have worked.

(Small) tarball for earthmovdist attached.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: earthmovdist_0.1.0.tar.gz
Type: application/octet-stream
Size: 15674 bytes
Desc: not available
Url : http://lists.r-forge.r-project.org/pipermail/phylobase-devl/attachments/20100116/b0ff6611/attachment-0001.obj 
-------------- next part --------------

This corresponds to the most recent revision r24.

| On Jan 15, 2010, at 11:56 AM, Dirk Eddelbuettel wrote:
| >  [snip successful attempt]
| > | >
| > | > So it appears to be an issue of the dynamic linker not finding the
| > | > right version of libRcpp.dylib
| > | >
| > | >
| > | > I'm not sure how to fix it other than altering the user's env  
| > (which
| > | > is obviously not ideal).
| >
| > The idea (originally suggested by Simon Urbanek who also supplied  
| > quite some
| > help in getting the dylib business right) was that from an R prompt  
| > (or shell
| > prompt via Rscript / littler) you could call Rcpp itself by using the
| > functions
| >
| > 	  Rcpp:::CxxFlags()
| > 	  Rcpp:::LdFlags()
| 
| Sorry for the confusion -- I meant to say the "loader not finding"  
| rather than "linker not finding".  It links, but the loading fails. So  
| I think that your Rcpp:::LdFlags() is working.

Then I may well need some OS X help on learning you guys instrument the
loader.  I thought Simon and I had that squared away.  

| [snip]
| >
| > ( Building RQuantLib munges a lot CPU because QuantLib so huge.  And  
| > you'd
| > need Quantlib (Debian/ Ubuntu do. So this may be easier to look at  
| > than to
| > try to run at your end. )
| 
| You were not kidding -- I've never seen a library as big as QuantLib!   
| I did get it installed.
| 
| I see the same errors with running RQuantLib from an unaltered env  
| that I do for phylobase:
| 
| ###################################
| $ R CMD check  pkg
| * checking for working pdflatex ... OK
| * using log directory '/Users/mholder/builds/rin/rquantlib/pkg.Rcheck'
| [snip]
| * checking R files for syntax errors ... OK
| * checking whether the package can be loaded ... ERROR
| 
| Loading required package: Rcpp
| Error in dyn.load(file, DLLpath = DLLpath, ...) :
|    unable to load shared library '/Users/mholder/builds/rin/rquantlib/ 
| pkg.Rcheck/RQuantLib/libs/i386/RQuantLib.so':
|    dlopen(/Users/mholder/builds/rin/rquantlib/pkg.Rcheck/RQuantLib/ 
| libs/i386/RQuantLib.so, 6): Library not loaded: /Builds/Rdev-web/QA/ 
| Simon/packages/tiger-universal/Rlib/2.10/Rcpp/lib/i386/libRcpp.dylib
|    Referenced from: /Users/mholder/builds/rin/rquantlib/pkg.Rcheck/ 
| RQuantLib/libs/i386/RQuantLib.so
|    Reason: image not found
| Error in library(RQuantLib) : .First.lib failed for 'RQuantLib'
| Execution halted
| 
| ###################################
| 
| If I can install and then I add the parent of libRcpp.dylib to my  
| DYLD_LIBRARY_PATH, then library(RQuantLib) succeeds.
| 
| 
| I looked at https://r-forge.r-project.org/R/?group_id=111&log=check_x86_64_mac&pkg=RQuantLib&flavor=patched
| which indicates no problems.

Right.  I do not know what R-Forge does but they don't cut any specials for
Rcpp as far as I know.

| Could it be that it is the
| "Additional arguments to R CMD check: --install=fake"
| line that explains why this build-bot is succeeding while phylobase's  
| is failing?

AFAIK --fake does less rather than more and I would expect it to fail earlier
/ more often.  R CMD INSTALL --help says:

 --fake            do minimal install for testing purposes


Dirk

| I tried adding that --install-args="--install=fake" but that never  
| seemed to result in a "Additional arguments to R CMD check: ..."  
| statement in the output.
| 
| 
| all the best,
| Mark
| 
| 
| 
| 
| 
| 
| Mark Holder
| 
| mtholder at ku.edu
| http://www.people.ku.edu/~mtholder/
| 
| ==============================================
| Department of Ecology and Evolutionary Biology
| University of Kansas
| 6031 Haworth Hall
| 1200 Sunnyside Avenue
| Lawrence, Kansas 66045
| 
| lab phone:  785.864.5789
| 
| fax (shared): 785.864.5860
| ==============================================
| 
| 
| 
| 
| 
| 
| 

-- 
Three out of two people have difficulties with fractions.

More information about the Phylobase-devl mailing list