[Rcpp-devel] Rcpp build trouble with strings

[jacob.a.russell.th at dartmouth.edu]

Hey Dirk,

I know that linking to a shared library is not enough, and that I need
to runtime load the actual libraries that have their shared code stored
separately. I wanted to do this in a way that when loaded by someone who
installs the package, the shared libraries that I am including are
available and the installer does the work for setting the ldconfig path
for the libraries themselves. I looked at the RcppGSL package, but it
doesn't actually include the library itself, but rather has you install
it from an external source. I would like to include the libraries
themselves, as they are not available from an external source. I settled
on trying to include just the object files themselves (extracted them
from the archives, they are all compiled with -fPIC). This got rid of
the error that it can't find my shared library (obviously) but still has
an error that it cannot find the symbols for one of the files that was
in one of the libraries. 

Running the default build I get an error:

Error in dyn.load....
...undefined symbol:_Z20parse_options_createv

This is a function in a file that I only have an object for and not a
cpp file (api.h and api.o)

Because I kept getting this error, I tried to add a target to Makevars
to see if I could override the build options for only the
SantosDartmouth.so file that I am trying to create. I created the build
target (Note: $OBJS is a variable that I hand-made with ls *.o):

SantosDartmouth.so: $(OBJECTS)
        $(CC) -shared $(PKG_LIBS) $(PKG_CPPFLAGS) $(OBJS) -Wl,--no-undefined -o SantosDartmouth.so

I got the same error, and the output of R CMD check SantosDartmouth was
the same except that it output that it overrode my target and showed
what it actually used to compile. So from the command line, I ran what I
actually wanted to run to build the target.

g++ -shared `Rscript -e 'Rcpp:::LdFlags()'` `Rscript -e
'Rcpp:::CxxFlags()'` analyze-linkage.o and.o api.o build-disjuncts.o
command-line.o constituents.o count.o DG_commonTypes.o
DGG_graphComponents.o DGG_heuristics.o DGG_strings.o DGG_templates.o
DGG_tree.o DG_objectTable.o diGraph.o DLL_commonTypes.o
documentGraphGenerator.o error.o extract-links.o fast-match.o
functions.o getDocGraph.o idiom.o linkset.o massage.o my_std.o parse.o
post-process.o pp_knowledge.o pp_lexer.o pp_linkset.o preparation.o
print.o print-util.o prune.o read-dict.o resources.o skip_exp.o
string-set.o stringTable.o tokenize.o utilities.o word-file.o -o
SantosDartmouth.so

Then I was able to load this in R with dyn.load("SantosDartmouth.so")
and sourcing my R file in the R directory of my package, I could run the
function. However, I would still like to be able to build the package
and not manually distribute this. How can I force the package to accept
my override for this target?

Thanks again, sorry to be so long winded.

Jacob
On Mon, Aug 19, 2013 at 06:58:06PM -0500, Dirk Eddelbuettel wrote:
> 
> Hi Jacob,
> 
> On 19 August 2013 at 13:10, jacob.a.russell.th at dartmouth.edu wrote:
> | Hey again,
> | 
> | I built the default Rcpp.package.skeleton rcpp_hello_world to take in
> | strings and it worked fine. My problems seem to be from the libraries
> | themselves. I recompiled the libraries that I could to be shared
> | libraries and now the package builds the .so file, but I still get an
> | error when checking the package.
> | 
> | Error in dyn.load(file, DLLpath = DLLpath, ...) : 
> |   unable to load shared object
> |   '/home/jrussell/src/C++/NLP/RDGGDriver/SantosDartmouth.Rcheck/SantosDartmouth/libs/SantosDartmouth.so':
> |     liblink_4_1b.so: cannot open shared object file: No such file or directory
> 
> Do you know about ldconfig and all that?  "Just linking" to a shared library
> is usually not enough.
> 
> If all that is way too confusing, then don't build a library but rather
> including all src/*.o directly into your package library -- R knows how to
> find that.
> | 
> | The library liblink_4_1b.so is link grammar with slight modifications to
> | work with C++ code instead of C code and doesn't have an R package. I
> | have placed all libs in the src directory of the package, and they can
> | all be found at build time, and the .so file seems to be correctly
> | built, but then when it is checked, it seems not to use the same
> | ldflags that it uses when building (PKG_LIBS). Is this something that I
> | need to set separately? I have tried ??Makevars and google searches for
> | "R Makevars" and "external libraries Rcpp" and not found much. Are there
> | any concrete examples of packaging external libraries inside of R
> 
> Of course. For example RcppGSL has done it for years. Here GSL is assumed to
> be a standard system library (but gsl-config --libs is queried to get the -L
> argument). 
> 
> There are now 130 packages on CRAN that deploy Rcpp. Maybe you can find one
> that suits your setup.
> 
> | packages? I have gone through all of the vignettes that I could find and
> | can't seem to find anything about including pre-built .so files in
> | packages. I suspect this is to keep source-only, but the libraries are
> | not all mine and I don't have this option available.
> | 
> | Thanks!
> | 
> | Jacob Russell
> | 
> | P.S. Time-To-Response on this mailing list is probably the best that I
> | have ever seen on a mailing list. Congrats!
> 
> :)
> 
> Had a busy day at work or else I would have replied earlier.
> 
> Dirk
> 
> | 
> | On Fri, Aug 16, 2013 at 12:52:37PM -0500, Dirk Eddelbuettel wrote:
> | > 
> | > On 16 August 2013 at 13:45, jacob.a.russell.th at dartmouth.edu wrote:
> | > | Hello,
> | > | 
> | > | I am trying to build a package but I have quite a few dependencies. As such, I
> | > | have modified Makevars to be (I don't like backticks because I find them too
> | > | similar to single quotes):
> | > | 
> | > | PKG_LIBS = $($(R_HOME)/bin/Rscript -e 'Rcpp:::LdFlags()') -L. -llink_4_1b -lDGG -lSTR -lDLL -lAF -lmy_std -lm
> | > | PKG_CPPFLAGS=-I. -g
> | > | 
> | > | Which seems to compile when my R wrapper is (incorrectly):
> | > | 
> | > | RcppExport SEXP getDocGraph(std::string input){
> | > |     
> | > |     std::string testStr = input;
> | > |     std::string output=getDG(testStr);
> | > |     return(Rcpp::wrap(output));
> | > | }
> | > | 
> | > | but then it gives me a segmentation fault because it cannot access the
> | > | underlying character array (or general object) passed in by R.
> | > | 
> | > | Then if I change the function protoype to:
> | > | 
> | > | RcppExport SEXP getDocGraph(SEXP input){
> | > |     
> | > |     std::string testStr = Rcpp::as<std::string>(input);
> | > |     std::string output=getDG(testStr);
> | > |     return(Rcpp::wrap(output));
> | > | }
> | > | 
> | > | it seems to compile, but when I go to check the package, the .Rcheck directory
> | > | gives me an error:
> | > | 
> | > | ** testing if installed package can be loaded
> | > | Error in dyn.load(file, DLLpath = DLLpath, ...) : 
> | > |   unable to load shared object ...<packagename>.so':
> | > | 
> | > | and the .so file is not in the tgz package, but no error was given during
> | > | compilation.
> | > | 
> | > | Any help is greatly appreciated.
> | > 
> | > Simplify. 
> | > 
> | > I would start by removing your six additional libraries and convince myself
> | > that accessing a string via Rcpp works (and as our unit tests do that a lot,
> | > you would have to be in a very unusual situation to not have it work).
> | > 
> | > Once that works, add your libraries. If and when things break again, examine
> | > the differences.
> | > 
> | > Dirk
> | > 
> | > -- 
> | > Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> 
> -- 
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com