[Rcpp-devel] Interfacing Rcpp with a C library: memory allocation

[Dirk Eddelbuettel]

On 20 November 2013 at 18:27, Alessandro Mammana wrote:
| Dear all,
| I'm trying to write some efficient code for analyzing sequencing data
| in R. To do this I would like to use the C library samtools. I've
| created a package where the src directory looks like this:
| 
| src
| |-- Makevars
| |-- RcppExports.cpp
| |-- mysourcecode.cpp
| `-- samtools
|     |-- all *.c and *.h files as well as an independent Makefile
| 
| My Makevars file looks like this:
| 
| PKG_CPPFLAGS = -Isamtools
| PKG_LIBS = `$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"`
| PKG_LIBS += -Lsamtools  -lbam -lz -lpthread
| 
| $(SHLIB): samtools/libbam.a
| 
| samtools/libbam.a:
|       @(cd samtools-0.1.19 && $(MAKE) libbam.a \
|           CC="$(CC)" CFLAGS="$(CFLAGS) $(CPICFLAGS)" AR="$(AR)"
| RANLIB="$(RANLIB)")
| 
| Everything compiles and I get my cpp functions in R, however I am
| getting some weird segfaults, I think they are due to memory
| allocation, but it's hard for me to track them. Especially now, these
| errors are showing up not immediately, but at the second time that I
| call a Rcpp function.
| 
| I wanted to ask the following:
| 1. Is it the right way of using external C libraries? I couldn't find
| much documentation around

Sure.

| 2. The C library uses malloc and free, and so do I (as little as
| possible, just to interface with the library), is this mechanism
| clashing against Rcpp/R memory management? Could it happen, for
| instance, that R tries to free allocated memory that I already
| manually freed myself?

Yes. 

Please read the "Writing R Extensions" manual, section 6.1.2, on
User-controlled memory. 

And/or see the Rcpp documentation: if you actually use our data containers,
things "just work", ie 

  R> cppFunction("NumericMatrix foo(int n) { return NumericMatrix(n); }")
  R> foo(2)
       [,1] [,2]
  [1,]    0    0
  [2,]    0    0
  R> foo(4)
       [,1] [,2] [,3] [,4]
  [1,]    0    0    0    0
  [2,]    0    0    0    0
  [3,]    0    0    0    0
  [4,]    0    0    0    0
  R> 

will never create a segfault. Ditto for using proper C++ containers (eg
std::vector<double>).

But you cannot randomly match the C approach (see Section 6.1.2, as mentioned
above) and the R/Rcpp approach.
 
| In general I didn't understand much about memory allocation in Rcpp
| and I couldn't find many resources talking about it. Is there anything

Well I could recommend a book to you...  There are also eight vignettes in
the package, and the more on other sites (the list archive, our blogs,
StackOverlow, Hadley's draft book).

| R- or Rcpp-specific that I have to keep in mind or should I program as
| if I were programming in C/C++?

You can provided you do it right.  

Hope this helps, if it was unclear please come back with more questions.
Also see the unit tests and examples.

Cheers, Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com