[Rcpp-devel] Read-only property in Rcpp module, can it be a Vector& or must it be a Vector

[Dominick Samperi]

On Sun, Jan 30, 2011 at 4:47 PM, Douglas Bates <bates at stat.wisc.edu> wrote:
> As the Grateful Dead said, "What a long, strange trip it's been."  I
> have spent over a month trying to debug some code in the lme4a package
> that uses Rcpp modules.  Partly I was getting the wrong numerical
> answer, which was due to a dumb cut-and-paste error on my part - now
> fixed - but more importantly I was getting memory protection errors.
>
> Luke Tierney has added new facilities to the memory allocation and
> garbage collection in R-devel to make it easier to detect when
> unprotected memory is being accessed.  You compile R-devel with
> -DTESTING_WRITE_BARRIER to enable this checking.
>
> I found that I was consistently getting an indication of unprotected
> memory being accessed in code that looks like
>
>  obj1$method1(obj2$method2)
>
> but only after many evaluations of this sequence, even with
> gctorture(1) on.  obj2$method2 returns a numeric vector, which is the
> argument to obj1$method1.  The declaration within the module is as a
> read-only property
>
>    .property("cu",        &reModule::cu,
>              "intermediate solution for u")
>
> and the accessor function is defined as returning a Rcpp::NumericVector &
>
>        const Rcpp::NumericVector    &cu() const {return d_cu;}
>
> This is starting to seem like the situation that Dominick mentioned
> where a reference to an object goes out of scope in the C++ code once
> the value is passed to R but R is retaining the SEXP somehow.  This is
> still pretty murky to me because the underlying SEXP is part of the
> Rcpp::NumericVector structure and is protected but somehow the
> reference gets stale?

Returning a reference looks dangerous, but as you point out later removing
the reference doesn't help. Nevertheless, there is a strong temptation to
pass vectors back to R by reference to limit copying, or to use the opposite
approach which is to work on R vectors that R passes to C/C++ directly
instead of making a copy.

I have done this, and it can lead to big
performance gains, but it comes at the cost of violating the Bob Dylan rule
(you have to be honest to live outside the law), so one has to be very careful.

Recall that this happens when R's assumption that called functions will
not modify the vectors passed to them is violated, as in:

x = rep(0,1000)
y = x # R typically points x and y to the same memory instead of copying here
.Call('nastyfunc', x)

If 'nastyfunc' modifies x, then y will be modified too, and this is probably not
what the R user expects to happen.

Could aliasing problems of this kind arise in the module code and lead to
a clobbered external pointer? I think this is unlikely because most R objects
are created from the C++ side and are protected. On the other hand, the
module code is quite intricate and it is possible that something
that needs protecting is missed.

It might be helpful to rewrite the problematic code to use reference classes and
external pointers explicitly instead of using modules. This is
tedious, but it will
reveal whether or not the problem is in the modules code or elsewhere.

Dominick

>
> Anyway I am going to change the accessor method to
>
>        const Rcpp::NumericVector          cu() const {return d_cu;}
>
> or, perhaps
>
>        Rcpp::NumericVector                  cu() const {return clone(d_cu);}
>
> which might cause unnecessary copying but at least will guarantee that
> there are no references involved.  The reason I would clone d_cu is
> because I want to ensure that d_cu cannot be modified at the R level.
>
> So my questions are: Should I avoid returning a reference and always
> return the vector itself?  Do I need to clone the object can I use
> const Rcpp::NumericVector cu() const {return d_cu;}?
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>