[Rcpp-devel] RcppArmadillo matrix with writeable auxiliary memory

[Dirk Eddelbuettel]

Hi Steven

On 25 November 2013 at 23:23, Steven Varga wrote:
| Thank you Dirk for the quick response;
| and for noting the memory handling.
| 
| I am aware of it and there are other ways of obtaining a pointer to some memory
| location. I used malloc/free to put focus on arma::mat copying data during

In short: don't use malloc/free _within and R context_. Use the replacement
functions provided by R, see "Writing R Extensions", Section 6.1 Memory
allocation and specifically Section 6.1.2. If you want to control it.

| assignment and/or not able to use reference as class member.
| Leaving plenty of room for that I am doing it wrong and there is a way to
| initialize a class member arma::mat with external ptr.
| 
| The documentation tells about arma::mat behaviour when setting ..., false, 
| strict=true); in my interpretation the last argument prevents changing size of
| arma::mat resulting in a 'copy free' instance. The provided c++ only example
| demonstrates that; it works as documented.

I may be missing something but as I understand your idea, you want to still
alter the memory content or size. That seems to be in conflict with strict=true.

As I mentioned before, I would start with a short standalone C++ test program
as a proof of concept.
 
| What I am looking for is setter on arma::mat to flexibly point to some memory
| location or a different '=' operator in terms of handling arma::mat objects
| when initialized with external pointer.
| 
| It is an armadillo question but came accross your post on stackoverflow where
| you suggested someone with similar question to post here.
| 
| Bigmemory looks great;  but the point here is to able to use matlab like BLAS
| level 3 operations on some large vector; small segements at a time.

That should not be an issue per se. We initialize Armadillo object with
externally provided memory (ie R objects) all the time.  

You can even allocate one big Armadillo object at first, and then
subset. Armadillo has a fairly rich API and a lot of subset and view
operators / functions.

But I don't have a precedent for you so you may need to work this out. We'll
try to help as well can.
 
| We use HDF5 for large datastores with flexible STL container like interface.

Sure. There is no reason why you couldn't reuse a pointer here.

Cheers,  Dirk

| Thanks again,
| Steven
| 
| On Nov 25, 2013 10:28 PM, "Dirk Eddelbuettel" <edd at debian.org> wrote:
| 
| 
|     Steven,
| 
|     In a case like this I would try to set up a fist minimal working example
|     using just plain C++, and no R. Once that works, try it with R.
| 
|     For Armadillo, the docs at http://arma.sourceforge.net/docs.html#Mat are
|     your
|     best shot.  I presume you saw what it says about strict=true. Also, in
| 
|        double* p = static_cast<double*>( malloc( 30*sizeof(double) ));
|        arma::mat M(p, 3,10, false, true ); // this works nicely
| 
|     you are going against the "Writing R Extensions" rule of mixing system
|     malloc/free with R.  That is generally a bad idea.
| 
|     If your intent is to keep memory 'fixed' and outside of R, you can use
|     Bigmemory objects via external pointers (Rcpp::XPtr). This may be of
|     interest: http://gallery.rcpp.org/articles/using-bigmemory-with-rcpp/
| 
|     That should extend to wrapping Armadillo around such memory. After all,
|     Armadillo behaves just _great_ with the R-allocated objects we pass back
|     and
|     forth.
| 
|     Hope this helps,  Dirk
| 
|     --
|     Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
| 
| 
| ----------------------------------------------------------------------
| _______________________________________________
| Rcpp-devel mailing list
| Rcpp-devel at lists.r-forge.r-project.org
| https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com