<div dir="ltr"><div class="markdown-here-wrapper" style><p style="margin:1.2em 0px!important">Hello RCPP,</p>
<p style="margin:1.2em 0px!important">There is a puzzle that I have been wrestling with for a while now. This is in the context of a package for numerical optimization. The underlying implementation is, of course, in C++. I’d like to let the user pass in both R functions and external pointers to C++ for function evaluation, much like in the RcppDE package. I also want the user to be able to pass in an environment in which to evaluate the function. So far so good. No issue doing that with the R functions, and for the C++ pointers I define a class after the RcppDE model:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important;display:block;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,255)">class EvalCompiled : public EvalBase {
public:
EvalCompiled(Rcpp::XPtr<funcPtr> xptr, SEXP __env) {
funptr = *(xptr);
env = __env;
};
EvalCompiled(SEXP xps, SEXP __env) {
Rcpp::XPtr<funcPtr> xptr(xps);
funptr = *(xptr);
env = __env;
};
Rcpp::NumericVector eval(SEXP par) {
neval++;
return funptr(par, env);
}
private:
funcPtr funptr;
SEXP env;
};
</code></pre><p style="margin:1.2em 0px!important">The user has to create the external pointer using inline in R. The following is an example using a logit regression setup with ridge penalty. The extra environment is required to pass in the data:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important;display:block;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,255)">lhoodL2.inc <- 'Rcpp::NumericVector lhood(SEXP xs, SEXP env){
arma::vec par = Rcpp::as<arma::vec>(xs);
Rcpp::Environment e = Rcpp::as<Rcpp::Environment>(env);
arma::mat X = Rcpp::as<arma::mat>(e["X"]);
arma::vec y = Rcpp::as<arma::vec>(e["y"]);
double prec = Rcpp::as<double>(e["prec"]);
arma::mat Xbeta = X * par;
double sum1 = sum(y % Xbeta - log(1 + exp(Xbeta)));
arma::mat sum2 = sum(pow(par, 2 * prec));
arma::vec out = -(sum1 - 0.5 * sum2);
Rcpp::NumericVector ret = Rcpp::as<Rcpp::NumericVector>(wrap(out));
return ret;
}
'
lhoodL2.body <- '
typedef Rcpp::NumericVector (*funcPtr)(SEXP, SEXP);
return(XPtr<funcPtr>(new funcPtr(&lhood)));
'
lhoodL2.CPP <- cxxfunction(signature(), body=lhoodL2.body,
inc=lhoodL2.inc, plugin="RcppArmadillo")
# Passing in some data
env <- new.env()
env[["X"]] <- X
env[["y"]] <- y
env[["prec"]] <- prec
output <- myNumericalOptimizationFun(lhoodL2.CPP(), c(-1.2,1), environment=env)
</code></pre><p style="margin:1.2em 0px!important">The external pointer is passed into the numerical optimization routine, and then the function is evaluated repeatedly until the optimization algorithm reaches convergence. Now this works, but there is a big inefficiency here, which is presumably slowing down things a lot. Namely, the data in the environment is copied in memory at each iteration of the function, even though this is not actually necessary. One would only need to copy the objects in memory the first time the function is evaluated, and then reference them with a pointer for the next iterations. Can you envision any way of doing this within the context of the RCPP framework?</p>
<p style="margin:1.2em 0px!important">Best wishes,</p>
<p style="margin:1.2em 0px!important">Antonio Coppola<br>Harvard University</p>
<div title="MDH:SGVsbG8gUkNQUCw8ZGl2Pjxicj48L2Rpdj48ZGl2PlRoZXJlIGlzIGEgcHV6emxlIHRoYXQgSSBo
YXZlIGJlZW4gd3Jlc3RsaW5nIHdpdGggZm9yIGEgd2hpbGUgbm93LiBUaGlzIGlzIGluIHRoZSBj
b250ZXh0IG9mIGEgcGFja2FnZSBmb3IgbnVtZXJpY2FsIG9wdGltaXphdGlvbi4gVGhlIHVuZGVy
bHlpbmcgaW1wbGVtZW50YXRpb24gaXMsIG9mIGNvdXJzZSwgaW4gQysrLiBJJ2QgbGlrZSB0byBs
ZXQgdGhlIHVzZXIgcGFzcyBpbiBib3RoIFIgZnVuY3Rpb25zIGFuZCBleHRlcm5hbCBwb2ludGVy
cyB0byBDKysgZm9yIGZ1bmN0aW9uIGV2YWx1YXRpb24sIG11Y2ggbGlrZSBpbiB0aGUgUmNwcERF
IHBhY2thZ2UuIEkgYWxzbyB3YW50IHRoZSB1c2VyIHRvIGJlIGFibGUgdG8gcGFzcyBpbiBhbiBl
bnZpcm9ubWVudCBpbiB3aGljaCB0byBldmFsdWF0ZSB0aGUgZnVuY3Rpb24uIFNvIGZhciBzbyBn
b29kLiBObyBpc3N1ZSBkb2luZyB0aGF0IHdpdGggdGhlIFIgZnVuY3Rpb25zLCBhbmQgZm9yIHRo
ZSBDKysgcG9pbnRlcnMgSSBkZWZpbmUgYSBjbGFzcyBhZnRlciB0aGUgUmNwcERFIG1vZGVsOjwv
ZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+YGBgPC9kaXY+PGRpdj48ZGl2PmNsYXNzIEV2YWxDb21w
aWxlZCA6IHB1YmxpYyBFdmFsQmFzZSB7PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0id2hpdGUtc3Bh
Y2U6IHByZS13cmFwOyI+CQk8L3NwYW4+cHVibGljOjwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9Indo
aXRlLXNwYWNlOiBwcmUtd3JhcDsiPgkJPC9zcGFuPiAmbmJzcDsgJm5ic3A7RXZhbENvbXBpbGVk
KFJjcHA6OlhQdHImbHQ7PHdicj5mdW5jUHRyJmd0OyB4cHRyLCBTRVhQIF9fZW52KSB7PC9kaXY+
PGRpdj48c3BhbiBzdHlsZT0id2hpdGUtc3BhY2U6IHByZS13cmFwOyI+CQkJCTwvc3Bhbj5mdW5w
dHIgPSAqKHhwdHIpOzwvZGl2Pgo8ZGl2PjxzcGFuIHN0eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdy
YXA7Ij4JCQkJPC9zcGFuPmVudiA9IF9fZW52OzwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9IndoaXRl
LXNwYWNlOiBwcmUtd3JhcDsiPgkJPC9zcGFuPiAmbmJzcDsgJm5ic3A7fTs8L2Rpdj48ZGl2Pjxz
cGFuIHN0eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdyYXA7Ij4JCTwvc3Bhbj4gJm5ic3A7ICZuYnNw
O0V2YWxDb21waWxlZChTRVhQIHhwcywgU0VYUCBfX2VudikgezwvZGl2PjxkaXY+PHNwYW4gc3R5
bGU9IndoaXRlLXNwYWNlOiBwcmUtd3JhcDsiPgkJCQk8L3NwYW4+UmNwcDo6WFB0ciZsdDtmdW5j
UHRyJmd0OyB4cHRyKHhwcyk7PC9kaXY+CjxkaXY+PHNwYW4gc3R5bGU9IndoaXRlLXNwYWNlOiBw
cmUtd3JhcDsiPgkJCQk8L3NwYW4+ZnVucHRyID0gKih4cHRyKTs8L2Rpdj48ZGl2PjxzcGFuIHN0
eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdyYXA7Ij4JCQkJPC9zcGFuPmVudiA9IF9fZW52OzwvZGl2
PjxkaXY+PHNwYW4gc3R5bGU9IndoaXRlLXNwYWNlOiBwcmUtd3JhcDsiPgkJPC9zcGFuPiAmbmJz
cDsgJm5ic3A7fTs8L2Rpdj48ZGl2Pgo8c3BhbiBzdHlsZT0id2hpdGUtc3BhY2U6IHByZS13cmFw
OyI+CQk8L3NwYW4+ICZuYnNwOyAmbmJzcDtSY3BwOjpOdW1lcmljVmVjdG9yIGV2YWwoU0VYUCBw
YXIpIHs8L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdyYXA7Ij4JCQkJ
PC9zcGFuPm5ldmFsKys7PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0id2hpdGUtc3BhY2U6IHByZS13
cmFwOyI+CQkJCTwvc3Bhbj5yZXR1cm4gZnVucHRyKHBhciwgZW52KTs8L2Rpdj4KPGRpdj48c3Bh
biBzdHlsZT0id2hpdGUtc3BhY2U6IHByZS13cmFwOyI+CQk8L3NwYW4+ICZuYnNwOyAmbmJzcDt9
PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0id2hpdGUtc3BhY2U6IHByZS13cmFwOyI+CQk8L3NwYW4+
cHJpdmF0ZTo8L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdyYXA7Ij4J
CTwvc3Bhbj4gJm5ic3A7ICZuYnNwO2Z1bmNQdHIgZnVucHRyOzxicj48L2Rpdj48ZGl2PjxzcGFu
IHN0eWxlPSJ3aGl0ZS1zcGFjZTogcHJlLXdyYXA7Ij4JCTwvc3Bhbj4gJm5ic3A7ICZuYnNwO1NF
WFAgZW52OzwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9IndoaXRlLXNwYWNlOiBwcmUtd3JhcDsiPgkJ
PC9zcGFuPn07PC9kaXY+PC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PlRo
ZSB1c2VyIGhhcyB0byBjcmVhdGUgdGhlIGV4dGVybmFsIHBvaW50ZXIgdXNpbmcgaW5saW5lIGlu
IFIuIFRoZSBmb2xsb3dpbmcgaXMgYW4gZXhhbXBsZSB1c2luZyBhIGxvZ2l0IHJlZ3Jlc3Npb24g
c2V0dXAgd2l0aCByaWRnZSBwZW5hbHR5LiBUaGUgZXh0cmEgZW52aXJvbm1lbnQgaXMgcmVxdWly
ZWQgdG8gcGFzcyBpbiB0aGUgZGF0YTo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PmBgYDwvZGl2
PjxkaXY+PGRpdj5saG9vZEwyLmluYyAmbHQ7LSAnUmNwcDo6TnVtZXJpY1ZlY3RvciBsaG9vZChT
RVhQIHhzLCBTRVhQIGVudil7PC9kaXY+PGRpdj4mbmJzcDsgYXJtYTo6dmVjIHBhciA9IFJjcHA6
OmFzJmx0O2FybWE6OnZlYyZndDsoeHMpOzwvZGl2PjxkaXY+Jm5ic3A7IFJjcHA6OkVudmlyb25t
ZW50IGUgPSBSY3BwOjphcyZsdDtSY3BwOjpFbnZpcm9ubWVudCZndDsoZW52KTs8L2Rpdj48ZGl2
PiZuYnNwOyBhcm1hOjptYXQgWCA9IFJjcHA6OmFzJmx0O2FybWE6Om1hdCZndDsoZVsiWCJdKTs8
L2Rpdj48ZGl2PiZuYnNwOyBhcm1hOjp2ZWMgeSA9IFJjcHA6OmFzJmx0O2FybWE6OnZlYyZndDso
ZVsieSJdKTs8L2Rpdj48ZGl2PiZuYnNwOyBkb3VibGUgcHJlYyA9IFJjcHA6OmFzJmx0O2RvdWJs
ZSZndDsoZVsicHJlYyJdKTs8L2Rpdj48ZGl2PiZuYnNwOyBhcm1hOjptYXQgWGJldGEgPSBYICog
cGFyOzwvZGl2PjxkaXY+Jm5ic3A7IGRvdWJsZSBzdW0xID0gc3VtKHkgJSBYYmV0YSAtIGxvZygx
ICsgZXhwKFhiZXRhKSkpOzwvZGl2PjxkaXY+Jm5ic3A7IGFybWE6Om1hdCBzdW0yID0gc3VtKHBv
dyhwYXIsIDIgKiBwcmVjKSk7PC9kaXY+PGRpdj4mbmJzcDsgYXJtYTo6dmVjIG91dCA9IC0oc3Vt
MSAtIDAuNSAqIHN1bTIpOzwvZGl2PjxkaXY+Jm5ic3A7IFJjcHA6Ok51bWVyaWNWZWN0b3IgcmV0
ID0gUmNwcDo6YXMmbHQ7UmNwcDo6TnVtZXJpY1ZlY3RvciZndDsod3JhcChvdXQpKTs8L2Rpdj48
ZGl2PiZuYnNwOyByZXR1cm4gcmV0OzwvZGl2PjxkaXY+fTwvZGl2PjxkaXY+JzwvZGl2PjwvZGl2
PjxkaXY+PGJyPjwvZGl2PjxkaXY+PGRpdj5saG9vZEwyLmJvZHkgJmx0Oy0gJzwvZGl2PjxkaXY+
Jm5ic3A7ICZuYnNwOyAmbmJzcDt0eXBlZGVmIFJjcHA6Ok51bWVyaWNWZWN0b3IgKCpmdW5jUHRy
KShTRVhQLCBTRVhQKTs8L2Rpdj48ZGl2PiZuYnNwOyAmbmJzcDsgJm5ic3A7cmV0dXJuKFhQdHIm
bHQ7ZnVuY1B0ciZndDsobmV3IGZ1bmNQdHIoJmFtcDtsaG9vZCkpKTs8L2Rpdj48ZGl2PiZuYnNw
OyAmbmJzcDsgJm5ic3A7JzwvZGl2PjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PGRpdj5saG9v
ZEwyLkNQUCAmbHQ7LSBjeHhmdW5jdGlvbihzaWduYXR1cmUoKSwgYm9keT1saG9vZEwyLmJvZHks
Jm5ic3A7PC9kaXY+PGRpdj4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNw
OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgaW5jPWxo
b29kTDIuaW5jLCBwbHVnaW49IlJjcHBBcm1hZGlsbG8iKTwvZGl2PjwvZGl2PjxkaXY+PGJyPjwv
ZGl2PjxkaXY+IyBQYXNzaW5nIGluIHNvbWUgZGF0YTwvZGl2PjxkaXY+PGRpdj5lbnYgJmx0Oy0g
bmV3LmVudigpPC9kaXY+PGRpdj5lbnZbWyJYIl1dICZsdDstIFg8L2Rpdj48ZGl2PmVudltbInki
XV0gJmx0Oy0geTwvZGl2PjxkaXY+ZW52W1sicHJlYyJdXSAmbHQ7LSBwcmVjPC9kaXY+PC9kaXY+
PGRpdj48YnI+PC9kaXY+PGRpdj48ZGl2Pm91dHB1dCAmbHQ7LSBteU51bWVyaWNhbE9wdGltaXph
dGlvbkZ1bihsaG9vZEwyLkNQUCgpLCBjKC0xLjIsMSksIGVudmlyb25tZW50PWVudik8L2Rpdj48
L2Rpdj48ZGl2PmBgYDwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhlIGV4dGVybmFsIHBvaW50
ZXIgaXMgcGFzc2VkIGludG8gdGhlIG51bWVyaWNhbCBvcHRpbWl6YXRpb24gcm91dGluZSwgYW5k
IHRoZW4gdGhlIGZ1bmN0aW9uIGlzIGV2YWx1YXRlZCByZXBlYXRlZGx5IHVudGlsIHRoZSBvcHRp
bWl6YXRpb24gYWxnb3JpdGhtIHJlYWNoZXMgY29udmVyZ2VuY2UuIE5vdyB0aGlzIHdvcmtzLCBi
dXQgdGhlcmUgaXMgYSBiaWcgaW5lZmZpY2llbmN5IGhlcmUsIHdoaWNoIGlzIHByZXN1bWFibHkg
c2xvd2luZyBkb3duIHRoaW5ncyBhIGxvdC4gTmFtZWx5LCB0aGUgZGF0YSBpbiB0aGUgZW52aXJv
bm1lbnQgaXMgY29waWVkIGluIG1lbW9yeSBhdCBlYWNoIGl0ZXJhdGlvbiBvZiB0aGUgZnVuY3Rp
b24sIGV2ZW4gdGhvdWdoIHRoaXMgaXMgbm90IGFjdHVhbGx5IG5lY2Vzc2FyeS4gT25lIHdvdWxk
IG9ubHkgbmVlZCB0byBjb3B5IHRoZSBvYmplY3RzIGluIG1lbW9yeSB0aGUgZmlyc3QgdGltZSB0
aGUgZnVuY3Rpb24gaXMgZXZhbHVhdGVkLCBhbmQgdGhlbiByZWZlcmVuY2UgdGhlbSB3aXRoIGEg
cG9pbnRlciBmb3IgdGhlIG5leHQgaXRlcmF0aW9ucy4gQ2FuIHlvdSBlbnZpc2lvbiBhbnkgd2F5
IG9mIGRvaW5nIHRoaXMgd2l0aGluIHRoZSBjb250ZXh0IG9mIHRoZSBSQ1BQIGZyYW1ld29yaz88
L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkJlc3Qgd2lzaGVzLDwvZGl2PjxkaXY+PGJyPjwvZGl2
PjxkaXY+QW50b25pbyBDb3Bwb2xhPC9kaXY+PGRpdj5IYXJ2YXJkIFVuaXZlcnNpdHk8L2Rpdj48
ZGl2Pjxicj48L2Rpdj4K" style="height:0;font-size:0em;padding:0;margin:0"></div></div></div>