[Rcpp-devel] Repeatedly evaluating external C++ function in the same environment

[Dirk Eddelbuettel]

Hi Antonio,

On 5 August 2014 at 11:03, Antonio Coppola wrote:
| Hello RCPP,
| 
| There is a puzzle that I have been wrestling with for a while now. This is in
| the context of a package for numerical optimization. The underlying
| implementation is, of course, in C++. I’d like to let the user pass in both R
| functions and external pointers to C++ for function evaluation, much like in
| the RcppDE package. I also want the user to be able to pass in an environment
| in which to evaluate the function. So far so good. No issue doing that with the
| R functions, and for the C++ pointers I define a class after the RcppDE model:
| 
| class EvalCompiled : public EvalBase {
|         public:
|             EvalCompiled(Rcpp::XPtr<funcPtr> xptr, SEXP __env) {
|                 funptr = *(xptr);
|                 env = __env;
|             };
|             EvalCompiled(SEXP xps, SEXP __env) {
|                 Rcpp::XPtr<funcPtr> xptr(xps);
|                 funptr = *(xptr);
|                 env = __env;
|             };
|              Rcpp::NumericVector eval(SEXP par) {
|                 neval++;
|                 return funptr(par, env);
|             }
|         private:
|             funcPtr funptr;
| 
|             SEXP env;
|         };
| 
| The user has to create the external pointer using inline in R. The following is
| an example using a logit regression setup with ridge penalty. The extra
| environment is required to pass in the data:
| 
| lhoodL2.inc <- 'Rcpp::NumericVector lhood(SEXP xs, SEXP env){
|   arma::vec par = Rcpp::as<arma::vec>(xs);
|   Rcpp::Environment e = Rcpp::as<Rcpp::Environment>(env);
|   arma::mat X = Rcpp::as<arma::mat>(e["X"]);
|   arma::vec y = Rcpp::as<arma::vec>(e["y"]);
|   double prec = Rcpp::as<double>(e["prec"]);
|   arma::mat Xbeta = X * par;
|   double sum1 = sum(y % Xbeta - log(1 + exp(Xbeta)));
|   arma::mat sum2 = sum(pow(par, 2 * prec));
|   arma::vec out = -(sum1 - 0.5 * sum2);
|   Rcpp::NumericVector ret = Rcpp::as<Rcpp::NumericVector>(wrap(out));
|   return ret;
| }
| '
| 
| lhoodL2.body <- '
|      typedef Rcpp::NumericVector (*funcPtr)(SEXP, SEXP);
|      return(XPtr<funcPtr>(new funcPtr(&lhood)));
|      '
| 
| lhoodL2.CPP <- cxxfunction(signature(), body=lhoodL2.body,
|                           inc=lhoodL2.inc, plugin="RcppArmadillo")
| 
| # Passing in some data
| env <- new.env()
| env[["X"]] <- X
| env[["y"]] <- y
| env[["prec"]] <- prec
| 
| output <- myNumericalOptimizationFun(lhoodL2.CPP(), c(-1.2,1), environment=env)
| 
| The external pointer is passed into the numerical optimization routine, and
| then the function is evaluated repeatedly until the optimization algorithm
| reaches convergence. Now this works, but there is a big inefficiency here,
| which is presumably slowing down things a lot. Namely, the data in the
| environment is copied in memory at each iteration of the function, even though
| this is not actually necessary. One would only need to copy the objects in
| memory the first time the function is evaluated, and then reference them with a
| pointer for the next iterations. Can you envision any way of doing this within
| the context of the RCPP framework?

Hm. Excellent question, and I quite like the XPtr use -- and I mentioned this
very trick for this very use in talks / workshops at U Chicago and U Kansas.

The environment should not get copied if it isn't altered.  You could try 
the  Rcpp::Environment class.  

Or you could try constructing an outer object holding the data, ensuring
there is only one and then supplying an accessor object.  I know I am being
brief here but does that makes sense to you?

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org