[Rcpp-devel] Range constructor and memory allocation

Gregor Kastner gregor.kastner at wu.ac.at
Wed Feb 19 09:53:41 CET 2014


Hi Kevin, hi Dirk,

K> All R data internally is a SEXPREC, ie, a C struct full of different
K> bits of information. The data part of a SEXPREC is forced to a
K> specific offset from the C structure, e.g. from Rinternals.h:
K> 
K> /* Under the generational allocator the data for vector nodes comes
K>    immediately after the node structure, so the data address is a
K>    known offset from the node SEXP. */
K> #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
K> 
K> Since Rcpp objects are just proxy wrappers over R objects, for Rcpp to
K> be able to reuse memory, we would have to construct the SEXPREC in the
K> correct memory location so that the resulting DATAPTR call happens to
K> fall where the data is. However, I'm not sure how we could reserve
K> that space for the SEXPREC.
K> 
K> Other data structures that just contain a pointer to their data,
K> rather than requiring that they lie a specified offset away, don't
K> have this problem (I imagine this is why e.g. Armadillo can reuse
K> memory)

All right, that helps me a lot understanding why every conversion from an
Armadillo object to an Rcpp object always seems to allocate fresh memory.

If I understand correctly, that also implies that returning (potentially
very large) RcppArmadillo objects to R requires a full copy of the data,
while this is not the case for pure Rcpp objects. (And then I guess that's
also the case in RcppEigen, so running there is not gonna save me... :))

D> To use an advanced constructor for Armadillo object requires Armadillo to
D> provide such a constructor in the first place. See 
D>
D>        http://arma.sourceforge.net/docs.html
D>
D> and I can only spot them for outright matrix, rowvec, colvec objects, but
D> not for ranges.

I guess all I was trying to understand is that it's possible to create e.g.
arma::mat objects with the range constructor while reusing memory. Because of
the reasons Kevin pointed out, this is difficult/impossible for e.g.
Rcpp::NumericMatrix objects.

D> For Rcpp constructor, things are different too as Kevin kindly pointed out
D> already. 

To sum up:

R ----(reuse)----> Rcpp ----(reuse)----> RcppArmadillo

but

RcppArmadillo ----(fresh)----> Rcpp ----(reuse)----> R

Correct?

Cheers,
Gregor


More information about the Rcpp-devel mailing list