[Rcpp-devel] Rcpp build trouble with strings
jacob.a.russell.th at dartmouth.edu
jacob.a.russell.th at dartmouth.edu
Thu Aug 22 01:53:21 CEST 2013
Hey Dirk,
Thanks for the help! I ended up reading a little more and found that I
could create a Makefile instead of using Makevars. I took this approach.
I am including the Makefile in case anyone else has a similar problem to
what I had but I don't recommend deviating from the Makevars approach
unless you have to (it's nice that it will automatically compile all of
the files for you). It's also nice that will automatically resolve the
location of the R and Rcpp header and libs for you (architecture
independent) and this will likely only work on some versions of linux.
PKG_LIBS = $(shell $(R_HOME)/bin/Rscript -e 'Rcpp:::LdFlags()') -L/usr/lib64/R/lib -lR
PKG_CPPFLAGS= $(shell $(R_HOME)/bin/Rscript -e 'Rcpp:::CxxFlags()') -I. -I/usr/include/R -fPIC -g
OBJECTS = analyze-linkage.o and.o api.o build-disjuncts.o command-line.o constituents.o count.o DG_commonTypes.o DGG_graphComponents.o DGG_heuristics.o DGG_strings.o DGG_templates.o DGG_tree.o DG_objectTable.o diGraph.o DLL_commonTypes.o documentGraphGenerator.o error.o extract-links.o fast-match.o functions.o getDocGraph.o idiom.o linkset.o massage.o my_std.o parse.o post-process.o pp_knowledge.o pp_lexer.o pp_linkset.o preparation.o print.o print-util.o prune.o read-dict.o resources.o skip_exp.o string-set.o stringTable.o tokenize.o utilities.o word-file.o
OBJS = DG_commonTypes.o DG_objectTable.o diGraph.o getDocGraph.o
LIBS = libAF.a libDGG.a libDLL.a liblink_4_1b.a libmy_std.a libSTR.a
.PHONY: clean xlibs
CC = g++
SantosDartmouth.so: $(OBJS) xlibs
$(CC) -shared -Wl,-O1,--sort-common,--as-needed,-z,relro $(PKG_LIBS) $(PKG_CPPFLAGS) $(OBJECTS) -o SantosDartmouth.so
xlibs:
$(shell for i in *.a;do ar -x $i;done)
%.o: %.cpp
$(CC) $(PKG_CPPFLAGS) $(PKG_LIBS) -c $< -o $@
clean:
rm SantosDartmouth.so *.o
Thanks again for the help!
Jacob
On Wed, Aug 21, 2013 at 06:35:15PM -0500, Dirk Eddelbuettel wrote:
>
> On 20 August 2013 at 15:09, jacob.a.russell.th at dartmouth.edu wrote:
> | Hey Dirk,
> |
> | I know that linking to a shared library is not enough, and that I need
> | to runtime load the actual libraries that have their shared code stored
> | separately. I wanted to do this in a way that when loaded by someone who
> | installs the package, the shared libraries that I am including are
> | available and the installer does the work for setting the ldconfig path
>
> The installer (for R) does nothing to ldconfig. That is a systems issue. I do
> not recommend mucking with that.
>
> | for the libraries themselves. I looked at the RcppGSL package, but it
> | doesn't actually include the library itself, but rather has you install
> | it from an external source. I would like to include the libraries
> | themselves, as they are not available from an external source. I settled
> | on trying to include just the object files themselves (extracted them
> | from the archives, they are all compiled with -fPIC). This got rid of
> | the error that it can't find my shared library (obviously) but still has
> | an error that it cannot find the symbols for one of the files that was
> | in one of the libraries.
> |
> | Running the default build I get an error:
> |
> | Error in dyn.load....
> | ...undefined symbol:_Z20parse_options_createv
> |
> | This is a function in a file that I only have an object for and not a
> | cpp file (api.h and api.o)
> |
> | Because I kept getting this error, I tried to add a target to Makevars
> | to see if I could override the build options for only the
> | SantosDartmouth.so file that I am trying to create. I created the build
> | target (Note: $OBJS is a variable that I hand-made with ls *.o):
> |
> | SantosDartmouth.so: $(OBJECTS)
> | $(CC) -shared $(PKG_LIBS) $(PKG_CPPFLAGS) $(OBJS) -Wl,--no-undefined -o SantosDartmouth.so
> |
> | I got the same error, and the output of R CMD check SantosDartmouth was
> | the same except that it output that it overrode my target and showed
> | what it actually used to compile. So from the command line, I ran what I
> | actually wanted to run to build the target.
> |
> | g++ -shared `Rscript -e 'Rcpp:::LdFlags()'` `Rscript -e
> | 'Rcpp:::CxxFlags()'` analyze-linkage.o and.o api.o build-disjuncts.o
> | command-line.o constituents.o count.o DG_commonTypes.o
> | DGG_graphComponents.o DGG_heuristics.o DGG_strings.o DGG_templates.o
> | DGG_tree.o DG_objectTable.o diGraph.o DLL_commonTypes.o
> | documentGraphGenerator.o error.o extract-links.o fast-match.o
> | functions.o getDocGraph.o idiom.o linkset.o massage.o my_std.o parse.o
> | post-process.o pp_knowledge.o pp_lexer.o pp_linkset.o preparation.o
> | print.o print-util.o prune.o read-dict.o resources.o skip_exp.o
> | string-set.o stringTable.o tokenize.o utilities.o word-file.o -o
> | SantosDartmouth.so
> |
> | Then I was able to load this in R with dyn.load("SantosDartmouth.so")
> | and sourcing my R file in the R directory of my package, I could run the
> | function. However, I would still like to be able to build the package
> | and not manually distribute this. How can I force the package to accept
> | my override for this target?
>
> I would look at any of the packages including a local copy of a library. I
> can't name a good one from the top of my head, but for example a little while
> back I helped the folks of the phylobase package to rationalize / rework
> their package build.
>
> Long story short, instead of mucking with local libraries (which, as I
> recall, invariably failed on one of the platforms) we just rearranged things
> so that all required sources are in src/ -- and the R knows how to do the
> right thing in that case.
>
> Otherwise, your long-road approach would be to learn how to build a library
> (outside of R), have it installed and use that.
>
> Dirk
>
> |
> | Thanks again, sorry to be so long winded.
> |
> | Jacob
> | On Mon, Aug 19, 2013 at 06:58:06PM -0500, Dirk Eddelbuettel wrote:
> | >
> | > Hi Jacob,
> | >
> | > On 19 August 2013 at 13:10, jacob.a.russell.th at dartmouth.edu wrote:
> | > | Hey again,
> | > |
> | > | I built the default Rcpp.package.skeleton rcpp_hello_world to take in
> | > | strings and it worked fine. My problems seem to be from the libraries
> | > | themselves. I recompiled the libraries that I could to be shared
> | > | libraries and now the package builds the .so file, but I still get an
> | > | error when checking the package.
> | > |
> | > | Error in dyn.load(file, DLLpath = DLLpath, ...) :
> | > | unable to load shared object
> | > | '/home/jrussell/src/C++/NLP/RDGGDriver/SantosDartmouth.Rcheck/SantosDartmouth/libs/SantosDartmouth.so':
> | > | liblink_4_1b.so: cannot open shared object file: No such file or directory
> | >
> | > Do you know about ldconfig and all that? "Just linking" to a shared library
> | > is usually not enough.
> | >
> | > If all that is way too confusing, then don't build a library but rather
> | > including all src/*.o directly into your package library -- R knows how to
> | > find that.
> | > |
> | > | The library liblink_4_1b.so is link grammar with slight modifications to
> | > | work with C++ code instead of C code and doesn't have an R package. I
> | > | have placed all libs in the src directory of the package, and they can
> | > | all be found at build time, and the .so file seems to be correctly
> | > | built, but then when it is checked, it seems not to use the same
> | > | ldflags that it uses when building (PKG_LIBS). Is this something that I
> | > | need to set separately? I have tried ??Makevars and google searches for
> | > | "R Makevars" and "external libraries Rcpp" and not found much. Are there
> | > | any concrete examples of packaging external libraries inside of R
> | >
> | > Of course. For example RcppGSL has done it for years. Here GSL is assumed to
> | > be a standard system library (but gsl-config --libs is queried to get the -L
> | > argument).
> | >
> | > There are now 130 packages on CRAN that deploy Rcpp. Maybe you can find one
> | > that suits your setup.
> | >
> | > | packages? I have gone through all of the vignettes that I could find and
> | > | can't seem to find anything about including pre-built .so files in
> | > | packages. I suspect this is to keep source-only, but the libraries are
> | > | not all mine and I don't have this option available.
> | > |
> | > | Thanks!
> | > |
> | > | Jacob Russell
> | > |
> | > | P.S. Time-To-Response on this mailing list is probably the best that I
> | > | have ever seen on a mailing list. Congrats!
> | >
> | > :)
> | >
> | > Had a busy day at work or else I would have replied earlier.
> | >
> | > Dirk
> | >
> | > |
> | > | On Fri, Aug 16, 2013 at 12:52:37PM -0500, Dirk Eddelbuettel wrote:
> | > | >
> | > | > On 16 August 2013 at 13:45, jacob.a.russell.th at dartmouth.edu wrote:
> | > | > | Hello,
> | > | > |
> | > | > | I am trying to build a package but I have quite a few dependencies. As such, I
> | > | > | have modified Makevars to be (I don't like backticks because I find them too
> | > | > | similar to single quotes):
> | > | > |
> | > | > | PKG_LIBS = $($(R_HOME)/bin/Rscript -e 'Rcpp:::LdFlags()') -L. -llink_4_1b -lDGG -lSTR -lDLL -lAF -lmy_std -lm
> | > | > | PKG_CPPFLAGS=-I. -g
> | > | > |
> | > | > | Which seems to compile when my R wrapper is (incorrectly):
> | > | > |
> | > | > | RcppExport SEXP getDocGraph(std::string input){
> | > | > |
> | > | > | std::string testStr = input;
> | > | > | std::string output=getDG(testStr);
> | > | > | return(Rcpp::wrap(output));
> | > | > | }
> | > | > |
> | > | > | but then it gives me a segmentation fault because it cannot access the
> | > | > | underlying character array (or general object) passed in by R.
> | > | > |
> | > | > | Then if I change the function protoype to:
> | > | > |
> | > | > | RcppExport SEXP getDocGraph(SEXP input){
> | > | > |
> | > | > | std::string testStr = Rcpp::as<std::string>(input);
> | > | > | std::string output=getDG(testStr);
> | > | > | return(Rcpp::wrap(output));
> | > | > | }
> | > | > |
> | > | > | it seems to compile, but when I go to check the package, the .Rcheck directory
> | > | > | gives me an error:
> | > | > |
> | > | > | ** testing if installed package can be loaded
> | > | > | Error in dyn.load(file, DLLpath = DLLpath, ...) :
> | > | > | unable to load shared object ...<packagename>.so':
> | > | > |
> | > | > | and the .so file is not in the tgz package, but no error was given during
> | > | > | compilation.
> | > | > |
> | > | > | Any help is greatly appreciated.
> | > | >
> | > | > Simplify.
> | > | >
> | > | > I would start by removing your six additional libraries and convince myself
> | > | > that accessing a string via Rcpp works (and as our unit tests do that a lot,
> | > | > you would have to be in a very unusual situation to not have it work).
> | > | >
> | > | > Once that works, add your libraries. If and when things break again, examine
> | > | > the differences.
> | > | >
> | > | > Dirk
> | > | >
> | > | > --
> | > | > Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> | >
> | > --
> | > Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> | _______________________________________________
> | Rcpp-devel mailing list
> | Rcpp-devel at lists.r-forge.r-project.org
> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
> --
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
More information about the Rcpp-devel
mailing list