[Rcpp-devel] RcppArmadillo matrix with writeable auxiliary memory

[Steven Varga]

Thanks a lot again;

I will look at the views and see how that works out. The matrix size is not
modified inside of a layer but the content only (gradient) while the
optimizer outside of the instances modifies the weight matrix accoding to
gradients.

Storing all layers gradients and weights at one spot gives the adventage of
distributed updates.

My bad: I should have used R memory allocators or just a big preallocated
matrix. And thanks again pointing out. I am aware of R memory
pooling/handling.

Thanks again,
Steven
On Nov 26, 2013 7:46 AM, "Dirk Eddelbuettel" <edd at debian.org> wrote:

>
> Hi Steven
>
> On 25 November 2013 at 23:23, Steven Varga wrote:
> | Thank you Dirk for the quick response;
> | and for noting the memory handling.
> |
> | I am aware of it and there are other ways of obtaining a pointer to some
> memory
> | location. I used malloc/free to put focus on arma::mat copying data
> during
>
> In short: don't use malloc/free _within and R context_. Use the replacement
> functions provided by R, see "Writing R Extensions", Section 6.1 Memory
> allocation and specifically Section 6.1.2. If you want to control it.
>
> | assignment and/or not able to use reference as class member.
> | Leaving plenty of room for that I am doing it wrong and there is a way to
> | initialize a class member arma::mat with external ptr.
> |
> | The documentation tells about arma::mat behaviour when setting ...,
> false,
> | strict=true); in my interpretation the last argument prevents changing
> size of
> | arma::mat resulting in a 'copy free' instance. The provided c++ only
> example
> | demonstrates that; it works as documented.
>
> I may be missing something but as I understand your idea, you want to still
> alter the memory content or size. That seems to be in conflict with
> strict=true.
>
> As I mentioned before, I would start with a short standalone C++ test
> program
> as a proof of concept.
>
> | What I am looking for is setter on arma::mat to flexibly point to some
> memory
> | location or a different '=' operator in terms of handling arma::mat
> objects
> | when initialized with external pointer.
> |
> | It is an armadillo question but came accross your post on stackoverflow
> where
> | you suggested someone with similar question to post here.
> |
> | Bigmemory looks great;  but the point here is to able to use matlab like
> BLAS
> | level 3 operations on some large vector; small segements at a time.
>
> That should not be an issue per se. We initialize Armadillo object with
> externally provided memory (ie R objects) all the time.
>
> You can even allocate one big Armadillo object at first, and then
> subset. Armadillo has a fairly rich API and a lot of subset and view
> operators / functions.
>
> But I don't have a precedent for you so you may need to work this out.
> We'll
> try to help as well can.
>
> | We use HDF5 for large datastores with flexible STL container like
> interface.
>
> Sure. There is no reason why you couldn't reuse a pointer here.
>
> Cheers,  Dirk
>
> | Thanks again,
> | Steven
> |
> | On Nov 25, 2013 10:28 PM, "Dirk Eddelbuettel" <edd at debian.org> wrote:
> |
> |
> |     Steven,
> |
> |     In a case like this I would try to set up a fist minimal working
> example
> |     using just plain C++, and no R. Once that works, try it with R.
> |
> |     For Armadillo, the docs at http://arma.sourceforge.net/docs.html#Matare
> |     your
> |     best shot.  I presume you saw what it says about strict=true. Also,
> in
> |
> |        double* p = static_cast<double*>( malloc( 30*sizeof(double) ));
> |        arma::mat M(p, 3,10, false, true ); // this works nicely
> |
> |     you are going against the "Writing R Extensions" rule of mixing
> system
> |     malloc/free with R.  That is generally a bad idea.
> |
> |     If your intent is to keep memory 'fixed' and outside of R, you can
> use
> |     Bigmemory objects via external pointers (Rcpp::XPtr). This may be of
> |     interest:
> http://gallery.rcpp.org/articles/using-bigmemory-with-rcpp/
> |
> |     That should extend to wrapping Armadillo around such memory. After
> all,
> |     Armadillo behaves just _great_ with the R-allocated objects we pass
> back
> |     and
> |     forth.
> |
> |     Hope this helps,  Dirk
> |
> |     --
> |     Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> |
> |
> | ----------------------------------------------------------------------
> | _______________________________________________
> | Rcpp-devel mailing list
> | Rcpp-devel at lists.r-forge.r-project.org
> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
> --
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20131126/0bb40b88/attachment.html>