[Rcpp-devel] Loading a package using Rcpp Modules results in memory corruption

Douglas Bates bates at stat.wisc.edu
Sun Jan 9 19:30:07 CET 2011


On Sun, Jan 9, 2011 at 12:07 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>
> On 9 January 2011 at 12:12, Dominick Samperi wrote:
> | Actually, the problem seems to be pretty transparent, and the
> | solution is the same (see below). This gets you through Evaluator,
> | but you fail in SlotProxy in the call to "$<-". The fact that these
> | problems only appear when gctorture() is turned on suggests
> | that they all of the same explanation: unprotected SEXP's.
> |
> | same:
> | //#define UNSAFE_CODE
> | #ifdef UNSAFE_CODE
> |     int error = LOGICAL( Rf_eval( Rf_lang1( Rf_install("errorOccured") ),
> | RCPP ) )[0];
> | #else
> |     SEXP cmd = PROTECT(Rf_lang1( Rf_install("errorOccured")));
> |     int error = LOGICAL(Rf_eval( cmd, RCPP ))[0];
> |     UNPROTECT(1);
> | #endif
> |     Rprintf("Got error = %d\n", error);
> |
>
> There may be an impedance mismatch going on.  'Writing R Extension' says
>
>    4.3.1 Using gctorture
>    ---------------------
>
>    We can help to detect memory problems earlier by running garbage
>    collection as often as possible.  This is achieved by
>    `gctorture(TRUE)', which as described on its help page
>
>         Provokes garbage collection on (nearly) every memory allocation.
>         Intended to ferret out memory protection bugs.  Also makes R run
>         _very_ slowly, unfortunately.
>
>    The reference to `memory protection' is to missing C-level calls to
>    `PROTECT'/`UNPROTECT' (*note Garbage Collection::) which if missing
>    allow R objects to be garbage-collected when they are still in use.
>    But it can also help with other memory-related errors.
>
>       Normally running under `gctorture(TRUE)' will just produce a crash
>    earlier in the R program, hopefully close to the actual cause. See the
>    next section for how to decipher such crashes.
>
>       It is possible to run all the examples, tests and vignettes covered
>    by `R CMD check' under `gctorture(TRUE)' by using the option
>    `--use-gct'.
>
> I do not see this as compatible with the C++ Design principle we use whereby
> protection / unprotection occurs relative to the end of scope -- and not
> after every single assignment or allocation.
>
> In other words, gctorture() may well be a fine test for the C API and its
> PROTECT and UNPROTECT at every step but possibly not quite as much for Rcpp.

I don't think so.  In the situation that Dominick is describing the C
API is being used (calls to Rf_install, Rf_lang1, Rf_eval, ...) and
you have to play by the rules of the C API.  Essentially every time
that you call a function in the C API that can cause a memory
allocation you are open yourself to the possibility of having the
garbage collector run and getting unprotected SEXPs blown away.  And,
naturally, Rf_eval can cause memory allocation.

I understood Dominick to be saying that in the code related to Modules
and the evaluator and all that which we have been trying to debug
there are such cases of the use of the C API that are unsafe.

> Dirk
>
> PS Here is slight generalization of your examples. Cases 2 and 3 do not bomb,
> and Case 3 is using almost all Rcpp idioms (but for the final Rf_eval call).
>
> -----------------------------------------------------------------------------
> // -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; tab-width: 8 -*-
>
> #include "Rcpp.h"
>
> RcppExport SEXP testfft(SEXP vec, SEXP env, SEXP choiceS) {
>
>    int choice = Rcpp::as<int>(choiceS);
>    SEXP res = R_NilValue;
>
>    if (choice == 1) {
>        Rprintf("In case 1\n");
>        // may not survive gctorture(), sometimes!
>        SEXP call = ::Rf_lcons(Rf_install("fft"),
>                               Rf_cons(vec , R_NilValue));
>        res = PROTECT(Rf_eval(call, env));
>        UNPROTECT(1);
>
>    } else if (choice == 2) {
>        Rprintf("In case 2\n");
>        // always survives gctorture()
>        SEXP call = PROTECT(::Rf_lcons(Rf_install("fft"),
>                                       Rf_cons(vec , R_NilValue)));
>        res = PROTECT(Rf_eval(call, env));
>        UNPROTECT(2);
>
>    } else if (choice == 3) {
>        Rprintf("In case 3\n");
>        Rcpp::Function fft("fft");
>        Rcpp::Environment env_(env);
>        Rcpp::Language call(fft);
>        call.push_back(vec);
>        res = Rf_eval(call, env_);
>    }
>    return res;
> }
> -----------------------------------------------------------------------------
>
> and
>
> -----------------------------------------------------------------------------
> # Tests Rcpp Evaluator code with PROTECTION fix and gctorture ON.
> # Should get a complex vector of zeros.
> library(Rcpp)
> gctorture()
> dyn.load("testfft.so")
> for (i in 2:3) {  # bombs on 1 but not 2 or 3
>    print(.Call('testfft', 1:20, getNamespace("stats"), i) - fft(1:20))
> }
> cat("still here\n")
> -----------------------------------------------------------------------------
>
>
> edd at max:/tmp/samperi-gc$ r -lRcpp -e'Rcpp:::SHLIB("testfft.cpp")'
> ccache g++ -I/usr/share/R/include -I/usr/local/lib/R/site-library/Rcpp/include     -fpic  -g -O3 -Wall -pipe -pedantic -Wno-variadic-macros -c testfft.cpp -o testfft.o
> g++ -shared -o testfft.so testfft.o -L/usr/local/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/local/lib/R/site-library/Rcpp/lib -L/usr/lib64/R/lib -lR
> edd at max:/tmp/samperi-gc$ r torture.R
> Loading required package: methods
> In case 2
>  [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> In case 3
>  [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> still here
> edd at max:/tmp/samperi-gc$
>
> --
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
>


More information about the Rcpp-devel mailing list