[Rcpp-devel] Read-only property in Rcpp module, can it be a Vector& or must it be a Vector

[Douglas Bates]

On Sun, Jan 30, 2011 at 3:47 PM, Douglas Bates <bates at stat.wisc.edu> wrote:
> As the Grateful Dead said, "What a long, strange trip it's been."  I
> have spent over a month trying to debug some code in the lme4a package
> that uses Rcpp modules.  Partly I was getting the wrong numerical
> answer, which was due to a dumb cut-and-paste error on my part - now
> fixed - but more importantly I was getting memory protection errors.
>
> Luke Tierney has added new facilities to the memory allocation and
> garbage collection in R-devel to make it easier to detect when
> unprotected memory is being accessed.  You compile R-devel with
> -DTESTING_WRITE_BARRIER to enable this checking.
>
> I found that I was consistently getting an indication of unprotected
> memory being accessed in code that looks like
>
>  obj1$method1(obj2$method2)
>
> but only after many evaluations of this sequence, even with
> gctorture(1) on.  obj2$method2 returns a numeric vector, which is the
> argument to obj1$method1.  The declaration within the module is as a
> read-only property
>
>    .property("cu",        &reModule::cu,
>              "intermediate solution for u")
>
> and the accessor function is defined as returning a Rcpp::NumericVector &
>
>        const Rcpp::NumericVector    &cu() const {return d_cu;}
>
> This is starting to seem like the situation that Dominick mentioned
> where a reference to an object goes out of scope in the C++ code once
> the value is passed to R but R is retaining the SEXP somehow.  This is
> still pretty murky to me because the underlying SEXP is part of the
> Rcpp::NumericVector structure and is protected but somehow the
> reference gets stale?
>
> Anyway I am going to change the accessor method to
>
>        const Rcpp::NumericVector          cu() const {return d_cu;}
>
> or, perhaps
>
>        Rcpp::NumericVector                  cu() const {return clone(d_cu);}
>
> which might cause unnecessary copying but at least will guarantee that
> there are no references involved.  The reason I would clone d_cu is
> because I want to ensure that d_cu cannot be modified at the R level.
>
> So my questions are: Should I avoid returning a reference and always
> return the vector itself?  Do I need to clone the object can I use
> const Rcpp::NumericVector cu() const {return d_cu;}?

Returning a reference doesn't seem to be the problem.  I changed the
code to return an Rcpp::NumericVector and the same thing happened.
Also, it doesn't appear to be the argument vector that has gone is
unprotected.  It is the external pointer to the method being invoked.

What is shown below is the result of running R-devel (with Luke's
checking turned on) under gdb and setting a breakpoint at Rf_errorcall
which is going to be called in the event of an error.  The initial
information about iterations shows that there are 8 successful
evaluations before the evaluation that showed unprotected memory being
used

(gdb) b Rf_errorcall
Breakpoint 1 at 0x7ffff7933898: file
/home/bates/src/R-devel/src/main/errors.c, line 617.
(gdb) sig 0
Continuing with no signal.

> lmer2(Yield ~ 1|Batch, Dyestuff, REML=FALSE, verbose=3L)
npt = 3 , n =  1
rhobeg =  0.2 , rhoend =  2e-07
  1:     327.76702:  1.00000
  2:     328.51213:  1.20000
  3:     327.34688: 0.800000
  4:     327.45504: 0.641430
   0.020:   4:      327.347;0.800000
  5:     327.32839: 0.764659
  6:     327.33628: 0.784659
  0.0020:   6:      327.328;0.764659
  7:     327.32711: 0.754808
  8:     327.32706: 0.752808
  9:     327.32709: 0.750808
 0.00020:   9:      327.327;0.752808

Breakpoint 1, Rf_errorcall (call=0x5e43868, format=0x7ffff7ad53de
"%s") at /home/bates/src/R-devel/src/main/errors.c:617
617	{
(gdb) up 5
#5  0x00007ffff038fee7 in Rcpp::class_<mer::deFeMod>::invoke_notvoid
(this=0x192a770, method_xp=0x3b7c128, object=0x5eeac58,
    args=0x7fffffff43c0, nargs=1) at
/home/bates/R/x86_64-unknown-linux-gnu-library/2.13/Rcpp/include/Rcpp/Module.h:465
465			return m->operator()( XP(object), args ) ;
(gdb) p LENGTH(*args)
$1 = 6
(gdb) p *REAL(*args)@6
$2 = {2893.0333687576567, 2937.2458388449832, 3006.4479659381896,
2879.5773996006442, 3075.650093031396, 2825.7535229725954}
(gdb) down 1
#4  0x00007ffff03a2056 in Rcpp::XPtr<mer::deFeMod>::operator
mer::deFeMod* (this=0x7fffffff4330)
    at /home/bates/R/x86_64-unknown-linux-gnu-library/2.13/Rcpp/include/Rcpp/XPtr.h:85
85	    inline operator T*(){ return (T*)(EXTPTR_PTR(m_sexp)) ; }
(gdb) p m_sexp
$3 = (SEXP) 0x5eeac58
(gdb) p *m_sexp
$4 = {sxpinfo = {type = 22, obj = 0, named = 2, gp = 0, mark = 1,
debug = 0, trace = 0, spare = 0, gcgen = 0, gccls = 0}, attrib =
0x605ab8,
  gengc_next_node = 0x5eeb5a0, gengc_prev_node = 0x5e7cc70, u =
{primsxp = {offset = 88700640}, symsxp = {pname = 0x54976e0, value =
0x605ab8,
      internal = 0x605ab8}, listsxp = {carval = 0x54976e0, cdrval =
0x605ab8, tagval = 0x605ab8}, envsxp = {frame = 0x54976e0,
      enclos = 0x605ab8, hashtab = 0x605ab8}, closxp = {formals =
0x54976e0, body = 0x605ab8, env = 0x605ab8}, promsxp = {value =
0x54976e0,
      expr = 0x605ab8, env = 0x605ab8}}}
(gdb) p CAR(m_sexp)

Breakpoint 1, Rf_errorcall (call=0x5e43868, format=0x7ffff7ad53de
"%s") at /home/bates/src/R-devel/src/main/errors.c:617
617	{

The error message is

  unprotected object (0x54976e0) encountered (was <unknown>)