[Rcpp-devel] Loading a package using Rcpp Modules results in memory corruption

[Dominick Samperi]

On Sun, Jan 9, 2011 at 1:07 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

>
> On 9 January 2011 at 12:12, Dominick Samperi wrote:
> | Actually, the problem seems to be pretty transparent, and the
> | solution is the same (see below). This gets you through Evaluator,
> | but you fail in SlotProxy in the call to "$<-". The fact that these
> | problems only appear when gctorture() is turned on suggests
> | that they all of the same explanation: unprotected SEXP's.
> |
> | same:
> | //#define UNSAFE_CODE
> | #ifdef UNSAFE_CODE
> |     int error = LOGICAL( Rf_eval( Rf_lang1( Rf_install("errorOccured") ),
> | RCPP ) )[0];
> | #else
> |     SEXP cmd = PROTECT(Rf_lang1( Rf_install("errorOccured")));
> |     int error = LOGICAL(Rf_eval( cmd, RCPP ))[0];
> |     UNPROTECT(1);
> | #endif
> |     Rprintf("Got error = %d\n", error);
> |
>
> There may be an impedance mismatch going on.  'Writing R Extension' says
>
>    4.3.1 Using gctorture
>    ---------------------
>
>    We can help to detect memory problems earlier by running garbage
>    collection as often as possible.  This is achieved by
>    `gctorture(TRUE)', which as described on its help page
>
>         Provokes garbage collection on (nearly) every memory allocation.
>         Intended to ferret out memory protection bugs.  Also makes R run
>         _very_ slowly, unfortunately.
>
>    The reference to `memory protection' is to missing C-level calls to
>    `PROTECT'/`UNPROTECT' (*note Garbage Collection::) which if missing
>    allow R objects to be garbage-collected when they are still in use.
>    But it can also help with other memory-related errors.
>
>       Normally running under `gctorture(TRUE)' will just produce a crash
>    earlier in the R program, hopefully close to the actual cause. See the
>    next section for how to decipher such crashes.
>
>       It is possible to run all the examples, tests and vignettes covered
>    by `R CMD check' under `gctorture(TRUE)' by using the option
>    `--use-gct'.
>
> I do not see this as compatible with the C++ Design principle we use
> whereby
> protection / unprotection occurs relative to the end of scope -- and not
> after every single assignment or allocation.
>

Thanks, but note that Doug's original report was not about gctorture(). We
are using this tool to try to fix the memory corruption problems, and this
tool seems to be pretty helpful for this purpose, even if it might be too
demanding in some situations.

Dominick


> In other words, gctorture() may well be a fine test for the C API and its
> PROTECT and UNPROTECT at every step but possibly not quite as much for
> Rcpp.
>
> Dirk
>
> PS Here is slight generalization of your examples. Cases 2 and 3 do not
> bomb,
> and Case 3 is using almost all Rcpp idioms (but for the final Rf_eval
> call).
>
>
> -----------------------------------------------------------------------------
> // -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; tab-width: 8 -*-
>
> #include "Rcpp.h"
>
> RcppExport SEXP testfft(SEXP vec, SEXP env, SEXP choiceS) {
>
>    int choice = Rcpp::as<int>(choiceS);
>    SEXP res = R_NilValue;
>
>    if (choice == 1) {
>        Rprintf("In case 1\n");
>        // may not survive gctorture(), sometimes!
>        SEXP call = ::Rf_lcons(Rf_install("fft"),
>                               Rf_cons(vec , R_NilValue));
>        res = PROTECT(Rf_eval(call, env));
>        UNPROTECT(1);
>
>    } else if (choice == 2) {
>        Rprintf("In case 2\n");
>        // always survives gctorture()
>        SEXP call = PROTECT(::Rf_lcons(Rf_install("fft"),
>                                       Rf_cons(vec , R_NilValue)));
>        res = PROTECT(Rf_eval(call, env));
>        UNPROTECT(2);
>
>    } else if (choice == 3) {
>        Rprintf("In case 3\n");
>        Rcpp::Function fft("fft");
>        Rcpp::Environment env_(env);
>        Rcpp::Language call(fft);
>        call.push_back(vec);
>        res = Rf_eval(call, env_);
>    }
>    return res;
> }
>
> -----------------------------------------------------------------------------
>
> and
>
>
> -----------------------------------------------------------------------------
> # Tests Rcpp Evaluator code with PROTECTION fix and gctorture ON.
> # Should get a complex vector of zeros.
> library(Rcpp)
> gctorture()
> dyn.load("testfft.so")
> for (i in 2:3) {  # bombs on 1 but not 2 or 3
>    print(.Call('testfft', 1:20, getNamespace("stats"), i) - fft(1:20))
> }
> cat("still here\n")
>
> -----------------------------------------------------------------------------
>
>
> edd at max:/tmp/samperi-gc$ r -lRcpp -e'Rcpp:::SHLIB("testfft.cpp")'
> ccache g++ -I/usr/share/R/include
> -I/usr/local/lib/R/site-library/Rcpp/include     -fpic  -g -O3 -Wall -pipe
> -pedantic -Wno-variadic-macros -c testfft.cpp -o testfft.o
> g++ -shared -o testfft.so testfft.o
> -L/usr/local/lib/R/site-library/Rcpp/lib -lRcpp
> -Wl,-rpath,/usr/local/lib/R/site-library/Rcpp/lib -L/usr/lib64/R/lib -lR
> edd at max:/tmp/samperi-gc$ r torture.R
> Loading required package: methods
> In case 2
>  [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> In case 3
>  [1] 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> 0+0i 0+0i 0+0i 0+0i 0+0i 0+0i
> still here
> edd at max:/tmp/samperi-gc$
>
> --
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20110109/e066d647/attachment.htm>