[Rcpp-devel] Possible unprotected memory problems

[Steve Lianoglou]

Hi,

On Thu, Jul 21, 2011 at 2:04 PM, Douglas Bates <bates at stat.wisc.edu> wrote:
> On Thu, Jul 21, 2011 at 12:52 PM, Douglas Bates <bates at stat.wisc.edu> wrote:
>> In testing this new version of lme4 based on RcppEigen, I have
>> encountered what looks to be a memory protection problem.  Naturally
>> it only occurs on the large examples but I may be able to provoke it
>> with gctorture even for a small example.
>>
>> I'm using Rcpp Modules to expose some classes in C++.  The symptom is
>> that suddenly, probably as the result of a garbage collection, the
>> value of one of the methods jumps.  In this case the residuals
>> suddenly appear to be zero.
>>
>> I'll start trying to track this but may end up running out of time as
>> I have some presentations for the next few weeks to prepare.
>
> This is probably my fault.  I was being careful to avoid copying some
> large structures.  I used the Eigen::Map capability to share R's
> storage but, unlike a case where the object in the C structure would
> be a NumericVector or NumericMatrix, the contents of the R object were
> not being protected because it would go out of scope.  I'll need to
> rewrite that part of the logic.  Sorry for the false alarm.

I'm glad you brought this up!

I've been meaning to ask if there is a way to do this successfully, or
is it impossible?

Where "it" is the ability to not copy the contents of (a potentially
large) numeric vector that we pass into a C/++ function, but rather
just "pass the pointer/data off" to the C side of the equation, and
let that worry about GC'ing the data when appropriate.

In theory, I guess it would be like having an unbalanced
PROTECT/UNPROTECT going on.

The "hand off" of the data/pointer to a C library would be like
calling PROTECT. After your C function returns control back to R, it
would still claim ownership/usage of the data. Things would hum along
"as usual", but the data in that part of memory wouldn't be GC'd by R
until your C library decides to call its UNPROTECT on that some point
later, at which point the normal R GC functionality would happen when
it happens.

Is that even possible?

Thanks,
-steve

-- 
Steve Lianoglou
Graduate Student: Computational Systems Biology
 | Memorial Sloan-Kettering Cancer Center
 | Weill Medical College of Cornell University
Contact Info: http://cbio.mskcc.org/~lianos/contact