[Rcpp-devel] Interfacing Rcpp with a C library: memory allocation

[Dirk Eddelbuettel]

On 20 November 2013 at 18:54, Alessandro Mammana wrote:
| Thanks a lot for your help, I'll look at the references. I saw Section
| 6.1.2 already. Does that mean that I should replace  all the calls to
| malloc() and to free() with those to Calloc() and Free() in the
| external C code (maybe with a macro?)? Or does it just mean that I
| would get the same memory errors, but this way they are handled by R?

The idea is not to get errors. 

Let's assume that the library you use is in itself correct.  You could then
use it and write wrapper functions. In the wrapper functions, you create Rcpp
type and assign to those. That would avoid having to alter the library. A
good defensive approach.

Depending on how well you know what you are doing, you can do something like
too where we assume we have  someFunction(double* p):

    int n;

    Rcpp::NumericVector x(n);      // vector of n element

    someFunction(x.begin());       // someFunction sees a double*

which is how I would interface a C library.  The devil is in the detail, and
you still have not provide any so we can't help you much more than via
generalities.

Pick one function from the library. Interface it. Get it work without a
segfault.  Apply what you learned on another function etc pp.

There are other approaches (Rcpp modules is one) but you may to make sure
you clear a few conceptual hurdles first.

Dirk



| 
| Thanks a lot!
| Ale
| 
| On Wed, Nov 20, 2013 at 6:35 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
| >
| > On 20 November 2013 at 18:27, Alessandro Mammana wrote:
| > | Dear all,
| > | I'm trying to write some efficient code for analyzing sequencing data
| > | in R. To do this I would like to use the C library samtools. I've
| > | created a package where the src directory looks like this:
| > |
| > | src
| > | |-- Makevars
| > | |-- RcppExports.cpp
| > | |-- mysourcecode.cpp
| > | `-- samtools
| > |     |-- all *.c and *.h files as well as an independent Makefile
| > |
| > | My Makevars file looks like this:
| > |
| > | PKG_CPPFLAGS = -Isamtools
| > | PKG_LIBS = `$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"`
| > | PKG_LIBS += -Lsamtools  -lbam -lz -lpthread
| > |
| > | $(SHLIB): samtools/libbam.a
| > |
| > | samtools/libbam.a:
| > |       @(cd samtools-0.1.19 && $(MAKE) libbam.a \
| > |           CC="$(CC)" CFLAGS="$(CFLAGS) $(CPICFLAGS)" AR="$(AR)"
| > | RANLIB="$(RANLIB)")
| > |
| > | Everything compiles and I get my cpp functions in R, however I am
| > | getting some weird segfaults, I think they are due to memory
| > | allocation, but it's hard for me to track them. Especially now, these
| > | errors are showing up not immediately, but at the second time that I
| > | call a Rcpp function.
| > |
| > | I wanted to ask the following:
| > | 1. Is it the right way of using external C libraries? I couldn't find
| > | much documentation around
| >
| > Sure.
| >
| > | 2. The C library uses malloc and free, and so do I (as little as
| > | possible, just to interface with the library), is this mechanism
| > | clashing against Rcpp/R memory management? Could it happen, for
| > | instance, that R tries to free allocated memory that I already
| > | manually freed myself?
| >
| > Yes.
| >
| > Please read the "Writing R Extensions" manual, section 6.1.2, on
| > User-controlled memory.
| >
| > And/or see the Rcpp documentation: if you actually use our data containers,
| > things "just work", ie
| >
| >   R> cppFunction("NumericMatrix foo(int n) { return NumericMatrix(n); }")
| >   R> foo(2)
| >        [,1] [,2]
| >   [1,]    0    0
| >   [2,]    0    0
| >   R> foo(4)
| >        [,1] [,2] [,3] [,4]
| >   [1,]    0    0    0    0
| >   [2,]    0    0    0    0
| >   [3,]    0    0    0    0
| >   [4,]    0    0    0    0
| >   R>
| >
| > will never create a segfault. Ditto for using proper C++ containers (eg
| > std::vector<double>).
| >
| > But you cannot randomly match the C approach (see Section 6.1.2, as mentioned
| > above) and the R/Rcpp approach.
| >
| > | In general I didn't understand much about memory allocation in Rcpp
| > | and I couldn't find many resources talking about it. Is there anything
| >
| > Well I could recommend a book to you...  There are also eight vignettes in
| > the package, and the more on other sites (the list archive, our blogs,
| > StackOverlow, Hadley's draft book).
| >
| > | R- or Rcpp-specific that I have to keep in mind or should I program as
| > | if I were programming in C/C++?
| >
| > You can provided you do it right.
| >
| > Hope this helps, if it was unclear please come back with more questions.
| > Also see the unit tests and examples.
| >
| > Cheers, Dirk
| >
| > --
| > Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
| 
| 
| 
| -- 
| Alessandro Mammana, PhD Student
| Max Planck Institute for Molecular Genetics
| Ihnestraße 63-73
| D-14195 Berlin, Germany

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com