[Rcpp-devel] Loading a package using Rcpp Modules results in memory corruption

[Dirk Eddelbuettel]

On 9 January 2011 at 12:12, Dominick Samperi wrote:
| Actually, the problem seems to be pretty transparent, and the
| solution is the same (see below). This gets you through Evaluator,
| but you fail in SlotProxy in the call to "$<-". The fact that these
| problems only appear when gctorture() is turned on suggests
| that they all of the same explanation: unprotected SEXP's.
| 
| same:
| //#define UNSAFE_CODE
| #ifdef UNSAFE_CODE
|     int error = LOGICAL( Rf_eval( Rf_lang1( Rf_install("errorOccured") ),
| RCPP ) )[0];
| #else
|     SEXP cmd = PROTECT(Rf_lang1( Rf_install("errorOccured")));
|     int error = LOGICAL(Rf_eval( cmd, RCPP ))[0];
|     UNPROTECT(1);
| #endif
|     Rprintf("Got error = %d\n", error);
|  

There may be an impedance mismatch going on.  'Writing R Extension' says

    4.3.1 Using gctorture
    ---------------------
    
    We can help to detect memory problems earlier by running garbage
    collection as often as possible.  This is achieved by
    `gctorture(TRUE)', which as described on its help page
    
         Provokes garbage collection on (nearly) every memory allocation.
         Intended to ferret out memory protection bugs.  Also makes R run
         _very_ slowly, unfortunately.
    
    The reference to `memory protection' is to missing C-level calls to
    `PROTECT'/`UNPROTECT' (*note Garbage Collection::) which if missing
    allow R objects to be garbage-collected when they are still in use.
    But it can also help with other memory-related errors.
    
       Normally running under `gctorture(TRUE)' will just produce a crash
    earlier in the R program, hopefully close to the actual cause. See the
    next section for how to decipher such crashes.
    
       It is possible to run all the examples, tests and vignettes covered
    by `R CMD check' under `gctorture(TRUE)' by using the option
    `--use-gct'.

I do not see this as compatible with the C++ Design principle we use whereby
protection / unprotection occurs relative to the end of scope -- and not
after every single assignment or allocation. 

In other words, gctorture() may well be a fine test for the C API and its
PROTECT and UNPROTECT at every step but possibly not quite as much for Rcpp.

Dirk

PS Here is slight generalization of your examples. Cases 2 and 3 do not bomb,
and Case 3 is using almost all Rcpp idioms (but for the final Rf_eval call).

-----------------------------------------------------------------------------
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; tab-width: 8 -*-

#include "Rcpp.h"

RcppExport SEXP testfft(SEXP vec, SEXP env, SEXP choiceS) {

    int choice = Rcpp::as<int>(choiceS);
    SEXP res = R_NilValue;

    if (choice == 1) {
	Rprintf("In case 1\n");
	// may not survive gctorture(), sometimes!
	SEXP call = ::Rf_lcons(Rf_install("fft"),
			       Rf_cons(vec , R_NilValue));
	res = PROTECT(Rf_eval(call, env));
	UNPROTECT(1);

    } else if (choice == 2) {
	Rprintf("In case 2\n");
	// always survives gctorture()
	SEXP call = PROTECT(::Rf_lcons(Rf_install("fft"),
				       Rf_cons(vec , R_NilValue)));
	res = PROTECT(Rf_eval(call, env));
	UNPROTECT(2);

    } else if (choice == 3) {
	Rprintf("In case 3\n");
	Rcpp::Function fft("fft");
	Rcpp::Environment env_(env);
	Rcpp::Language call(fft);
	call.push_back(vec);
	res = Rf_eval(call, env_);
    }
    return res;
}
-----------------------------------------------------------------------------

and

-----------------------------------------------------------------------------
# Tests Rcpp Evaluator code with PROTECTION fix and gctorture ON.
# Should get a complex vector of zeros.
library(Rcpp)
gctorture()
dyn.load("testfft.so")
for (i in 2:3) {  # bombs on 1 but not 2 or 3
    print(.Call('testfft', 1:20, getNamespace("stats"), i) - fft(1:20))
}
cat("still here\n")
-----------------------------------------------------------------------------


edd at max:/tmp/samperi-gc$ r -lRcpp -e'Rcpp:::SHLIB("testfft.cpp")'
ccache g++ -I/usr/share/R/include -I/usr/local/lib/R/site-library/Rcpp/include     -fpic  -g -O3 -Wall -pipe -pedantic -Wno-variadic-macros -c testfft.cpp -o testfft.o
g++ -shared -o testfft.so testfft.o -L/usr/local/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/local/lib/R/site-library/Rcpp/lib -L/usr/lib64/R/lib -lR
edd at max:/tmp/samperi-gc$ r torture.R
Loading required package: methods
In case 2
 [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
In case 3
 [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
still here
edd at max:/tmp/samperi-gc$ 

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com