<p>Thanks a lot again;</p>
<p> I will look at the views and see how that works out. The matrix size is not modified inside of a layer but the content only (gradient) while the optimizer outside of the instances modifies the weight matrix accoding to gradients.</p>
<p>Storing all layers gradients and weights at one spot gives the adventage of distributed updates.</p>
<p>My bad: I should have used R memory allocators or just a big preallocated matrix. And thanks again pointing out. I am aware of R memory pooling/handling.</p>
<p>Thanks again, <br>
Steven</p>
<div class="gmail_quote">On Nov 26, 2013 7:46 AM, "Dirk Eddelbuettel" <<a href="mailto:edd@debian.org">edd@debian.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Hi Steven<br>
<br>
On 25 November 2013 at 23:23, Steven Varga wrote:<br>
| Thank you Dirk for the quick response;<br>
| and for noting the memory handling.<br>
|<br>
| I am aware of it and there are other ways of obtaining a pointer to some memory<br>
| location. I used malloc/free to put focus on arma::mat copying data during<br>
<br>
In short: don't use malloc/free _within and R context_. Use the replacement<br>
functions provided by R, see "Writing R Extensions", Section 6.1 Memory<br>
allocation and specifically Section 6.1.2. If you want to control it.<br>
<br>
| assignment and/or not able to use reference as class member.<br>
| Leaving plenty of room for that I am doing it wrong and there is a way to<br>
| initialize a class member arma::mat with external ptr.<br>
|<br>
| The documentation tells about arma::mat behaviour when setting ..., false, <br>
| strict=true); in my interpretation the last argument prevents changing size of<br>
| arma::mat resulting in a 'copy free' instance. The provided c++ only example<br>
| demonstrates that; it works as documented.<br>
<br>
I may be missing something but as I understand your idea, you want to still<br>
alter the memory content or size. That seems to be in conflict with strict=true.<br>
<br>
As I mentioned before, I would start with a short standalone C++ test program<br>
as a proof of concept.<br>
<br>
| What I am looking for is setter on arma::mat to flexibly point to some memory<br>
| location or a different '=' operator in terms of handling arma::mat objects<br>
| when initialized with external pointer.<br>
|<br>
| It is an armadillo question but came accross your post on stackoverflow where<br>
| you suggested someone with similar question to post here.<br>
|<br>
| Bigmemory looks great; but the point here is to able to use matlab like BLAS<br>
| level 3 operations on some large vector; small segements at a time.<br>
<br>
That should not be an issue per se. We initialize Armadillo object with<br>
externally provided memory (ie R objects) all the time.<br>
<br>
You can even allocate one big Armadillo object at first, and then<br>
subset. Armadillo has a fairly rich API and a lot of subset and view<br>
operators / functions.<br>
<br>
But I don't have a precedent for you so you may need to work this out. We'll<br>
try to help as well can.<br>
<br>
| We use HDF5 for large datastores with flexible STL container like interface.<br>
<br>
Sure. There is no reason why you couldn't reuse a pointer here.<br>
<br>
Cheers, Dirk<br>
<br>
| Thanks again,<br>
| Steven<br>
|<br>
| On Nov 25, 2013 10:28 PM, "Dirk Eddelbuettel" <<a href="mailto:edd@debian.org">edd@debian.org</a>> wrote:<br>
|<br>
|<br>
| Steven,<br>
|<br>
| In a case like this I would try to set up a fist minimal working example<br>
| using just plain C++, and no R. Once that works, try it with R.<br>
|<br>
| For Armadillo, the docs at <a href="http://arma.sourceforge.net/docs.html#Mat" target="_blank">http://arma.sourceforge.net/docs.html#Mat</a> are<br>
| your<br>
| best shot. I presume you saw what it says about strict=true. Also, in<br>
|<br>
| double* p = static_cast<double*>( malloc( 30*sizeof(double) ));<br>
| arma::mat M(p, 3,10, false, true ); // this works nicely<br>
|<br>
| you are going against the "Writing R Extensions" rule of mixing system<br>
| malloc/free with R. That is generally a bad idea.<br>
|<br>
| If your intent is to keep memory 'fixed' and outside of R, you can use<br>
| Bigmemory objects via external pointers (Rcpp::XPtr). This may be of<br>
| interest: <a href="http://gallery.rcpp.org/articles/using-bigmemory-with-rcpp/" target="_blank">http://gallery.rcpp.org/articles/using-bigmemory-with-rcpp/</a><br>
|<br>
| That should extend to wrapping Armadillo around such memory. After all,<br>
| Armadillo behaves just _great_ with the R-allocated objects we pass back<br>
| and<br>
| forth.<br>
|<br>
| Hope this helps, Dirk<br>
|<br>
| --<br>
| Dirk Eddelbuettel | <a href="mailto:edd@debian.org">edd@debian.org</a> | <a href="http://dirk.eddelbuettel.com" target="_blank">http://dirk.eddelbuettel.com</a><br>
|<br>
|<br>
| ----------------------------------------------------------------------<br>
| _______________________________________________<br>
| Rcpp-devel mailing list<br>
| <a href="mailto:Rcpp-devel@lists.r-forge.r-project.org">Rcpp-devel@lists.r-forge.r-project.org</a><br>
| <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel</a><br>
--<br>
Dirk Eddelbuettel | <a href="mailto:edd@debian.org">edd@debian.org</a> | <a href="http://dirk.eddelbuettel.com" target="_blank">http://dirk.eddelbuettel.com</a><br>
</blockquote></div>