<div dir="ltr">Hi Dirk,<div><br></div><div>It seems that something's going on with element modification of arma::sp_mat objects. While in most of the cases I use batch insertion to create arma::sp_mat objects, in some others I have no other way of doing so but by looping through a list of elements to be added(modified). I've created a minimal example in which I try to create an adjacency matrix (an square matrix of size N) by filling its cells according to an edgelist (a matrix with 2 columns providing coordinates). I ran the code using Valgrind and no memory leaks were detected, but the code is not behaving as I would expect. Here is the code:</div><div><br></div><div>---------------- edgelist_to_adjmat_minimal.cpp --------------------------</div><div><br></div><div><div>#include <RcppArmadillo.h></div><div>using namespace Rcpp;</div><div><br></div><div>// EXAMPLE 1: Element access with -at- and in-place addition</div><div>// [[Rcpp::export]]</div><div>arma::sp_mat edgelist_to_adjmat_minimal(<wbr>IntegerMatrix edgelist, int n) {</div><div><br></div><div> arma::sp_mat adjmat(n,n);</div><div> for (unsigned int i = 0u; i < edgelist.nrow(); i++)</div><div> <a href="http://adjmat.at" target="_blank">adjmat.at</a>(<a href="http://edgelist.at" target="_blank">edgelist.at</a>(i,0u), <a href="http://edgelist.at" target="_blank">edgelist.at</a>(i, 1u)) += 1.0;</div><div><br></div><div> return adjmat;</div><div><br></div><div>}</div><div><br></div><div>// EXAMPLE 2: Element access with -at- and addition.</div><div>// [[Rcpp::export]]</div><div>arma::sp_mat edgelist_to_adjmat_minimal2(<wbr>IntegerMatrix edgelist, int n) {</div><div><br></div><div> arma::sp_mat adjmat(n,n);</div><div> for (unsigned int i = 0u; i < edgelist.nrow(); i++)</div><div> <a href="http://adjmat.at" target="_blank">adjmat.at</a>(<a href="http://edgelist.at" target="_blank">edgelist.at</a>(i,0u), <a href="http://edgelist.at" target="_blank">edgelist.at</a>(i, 1u)) =</div><div> <a href="http://adjmat.at" target="_blank">adjmat.at</a>(<a href="http://edgelist.at" target="_blank">edgelist.at</a>(i,0u), <a href="http://edgelist.at" target="_blank">edgelist.at</a>(i, 1u)) + 1.0;</div><div><br></div><div> return adjmat;</div><div><br></div><div>}</div><div><br></div><div>// EXAMPLE 3: Element access with -()- and addition</div><div>// [[Rcpp::export]]</div><div>arma::sp_mat edgelist_to_adjmat_minimal3(<wbr>IntegerMatrix edgelist, int n) {</div><div><br></div><div> arma::sp_mat adjmat(n,n);</div><div> for (unsigned int i = 0u; i < edgelist.nrow(); i++)</div><div> adjmat(<a href="http://edgelist.at" target="_blank">edgelist.at</a>(i,0u), <a href="http://edgelist.at" target="_blank">edgelist.at</a>(i, 1u)) =</div><div> adjmat(<a href="http://edgelist.at" target="_blank">edgelist.at</a>(i,0u), <a href="http://edgelist.at" target="_blank">edgelist.at</a>(i, 1u)) + 1.0;</div><div><br></div><div> return adjmat;</div><div><br></div><div>}</div><div><br></div><div>/***R</div><div>library(Matrix)</div><div><br></div><div># Creating a random edgelist with n vertices</div><div>n <- 5</div><div>set.seed(1)</div><div>edgelist <- unique(matrix(<a href="http://sample.int" target="_blank">sample.int</a>(n, 10, TRUE), ncol=2) - 1L)</div><div><br></div><div># Coercing to adjacency matrix</div><div>edgelist_to_adjmat_minimal(<wbr>edgelist, n)</div><div>edgelist_to_adjmat_minimal2(<wbr>edgelist, n)</div><div>edgelist_to_adjmat_minimal3(<wbr>edgelist, n)</div><div><br></div><div># This is what we should get</div><div>adjmat_ans <- matrix(0, ncol=n, nrow=n)</div><div>adjmat_ans[edgelist] <- 1</div><div>methods::as(adjmat_ans, "dgCMatrix")</div><div><br></div><div>sessionInfo()</div><div>*/</div></div><div><br></div><div>---------------- edgelist_to_adjmat_minimal.cpp --------------------------<br></div><div><br></div><div><br></div><div>And here is what I get</div><div><br></div><div><br></div><div>----------------- begin of output ------------------------------<wbr>--------------</div><div><div>> Rcpp::sourceCpp("playground/<wbr>edgelist_to_adjmat_minimal.<wbr>cpp")</div><div><br></div><div>> library(Matrix)</div><div><br></div><div>> # Creating a random edgelist with n vertices</div><div>> n <- 5</div><div><br></div><div>> set.seed(1)</div><div><br></div><div>> edgelist <- unique(matrix(<a href="http://sample.int" target="_blank">sample.int</a>(n, 10, TRUE), ncol=2) - 1L)</div><div><br></div><div>> # Coercing to adjacency matrix</div><div>> edgelist_to_adjmat_minimal(<wbr>edgelist, n)</div><div>5 x 5 sparse Matrix of class "dgCMatrix"</div><div> </div><div>[1,] . . . . .</div><div>[2,] . . . . .</div><div>[3,] . . . . .</div><div>[4,] . . . . .</div><div>[5,] . . . . .</div><div><br></div><div>> edgelist_to_adjmat_minimal2(<wbr>edgelist, n)</div><div>5 x 5 sparse Matrix of class "dgCMatrix"</div><div> </div><div>[1,] . . . . .</div><div>[2,] . . . . .</div><div>[3,] . . . . .</div><div>[4,] . . . . .</div><div>[5,] . . . . .</div><div><br></div><div>> edgelist_to_adjmat_minimal3(<wbr>edgelist, n)</div><div>5 x 5 sparse Matrix of class "dgCMatrix"</div><div> </div><div>[1,] . . . . .</div><div>[2,] . . . . .</div><div>[3,] . . . . .</div><div>[4,] . . . . .</div><div>[5,] . . . . .</div><div><br></div><div>> # This is what we should get</div><div>> adjmat_ans <- matrix(0, ncol=n, nrow=n)</div><div><br></div><div>> adjmat_ans[edgelist] <- 1</div><div><br></div><div>> methods::as(adjmat_ans, "dgCMatrix")</div><div>5 x 5 sparse Matrix of class "dgCMatrix"</div><div> </div><div>[1,] . . . 1 .</div><div>[2,] . . 1 . .</div><div>[3,] . . . . .</div><div>[4,] . . 1 . .</div><div>[5,] . . . . .</div><div><br></div><div>> sessionInfo()</div><div>R version 3.4.2 (2017-09-28)</div><div>Platform: x86_64-pc-linux-gnu (64-bit)</div><div>Running under: Ubuntu 14.04.5 LTS</div><div><br></div><div>Matrix products: default</div><div>BLAS: /usr/lib/libblas/libblas.so.3.<wbr>0</div><div>LAPACK: /usr/lib/lapack/liblapack.so.<wbr>3.0</div><div><br></div><div>locale:</div><div> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C </div><div> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 </div><div> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 </div><div> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C </div><div> [9] LC_ADDRESS=C LC_TELEPHONE=C </div><div>[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C </div><div><br></div><div>attached base packages:</div><div>[1] stats graphics grDevices utils datasets methods base </div><div><br></div><div>other attached packages:</div><div>[1] Matrix_1.2-11</div><div><br></div><div>loaded via a namespace (and not attached):</div><div>[1] compiler_3.4.2 tools_3.4.2 </div><div>[3] RcppArmadillo_0.8.100.1.0 Rcpp_0.12.13 </div><div>[5] grid_3.4.2 lattice_0.20-35 </div></div><div><br></div><div>------------------------- end of output ------------------------------<wbr>-----------</div><div><br></div><div>I use three different examples thinking that perhaps some element accessing method was no longer valid, or even the in-place addition was no longer working.</div><div><br></div><div>Anything looks wrong in what I've written? From what I've seen in my test results, all of the errors that I've checked so far have to do with this.</div><div><br></div><div>Best,</div><div><br></div><div class="gmail_extra"><br clear="all"><div><div class="m_7561838855691988328gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>George G. Vega Yon<br><a href="tel:(626)%20381-8171" value="+16263818171" target="_blank">+1 (626) 381 8171</a><br><a href="http://cana.usc.edu/vegayon" target="_blank">http://cana.usc.edu/vegayon</a></div></div></div></div></div></div>
<br><div class="gmail_quote">On Sat, Oct 7, 2017 at 3:15 PM, Dirk Eddelbuettel <span dir="ltr"><<a href="mailto:edd@debian.org" target="_blank">edd@debian.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
RcppArmadillo 0.8.100.1.0 is available in the drat repo for the RcppCore org.<br>
See the README.md at <a href="https://github.com/RcppCore/drat" rel="noreferrer" target="_blank">https://github.com/RcppCore/dr<wbr>at</a> which has this exammple:<br>
<br>
# first add the repo<br>
drat:::add("RcppCore")<br>
# either install just one or more given packages<br>
install.packages("RcppArmadil<wbr>lo")<br>
# or update already installed packages<br>
update.packages()<br>
<br>
You can also add the repo URL by hand to options("repos"), or supply it to<br>
install.packages(), or ... I happen to like drat. The files NEWS.Rd and<br>
ChangeLog have the goods, I make a fuller announcement if and when it makes<br>
it to CRAN.<br>
<br>
There are a rathre fair number of upstream changes in here -- making it the<br>
first Armadillo release by Conrad with an 8.* number. It also has further<br>
sparse matrix improvements from our end thanks to Serguei and Binxiang.<br>
<br>
I had uploaded this to CRAN based on a reverse depends check ... where the<br>
previous version was still in the loadpath. Ooops. So I overlooked a few<br>
build or test errors for which I need a bit of help from the maintainers. In<br>
particular, packages<br>
<br>
biglasso<br>
bigstatsr<br>
HSAR<br>
netdiffuseR<br>
repolr<br>
<br>
now come up as failing their tests. I am BCCing the maintainers and kindly<br>
ask if they could take a look too --- I don't always have all suggested<br>
packages installed.<br>
<br>
In the case of HSAR, I found that a one life of code needs a changed imposed<br>
by Conrad as the sparse-to-dense conversion no longer works with with a<br>
copy. This patch covers it:<br>
<br>
diff -ru HSAR.orig/src/diagnostics.cpp HSAR/src/diagnostics.cpp<br>
--- HSAR.orig/src/diagnostics.cpp 2016-05-24 13:58:45.000000000 -0500<br>
+++ HSAR/src/diagnostics.cpp 2017-10-07 13:24:31.471883248 -0500<br>
@@ -10,7 +10,7 @@<br>
<br>
sp_mat SW = I_sp+rho*W+pow(rho,2)*(W*W)+po<wbr>w(rho,3)*(W*W*W)+pow(rho,4)*(<wbr>W*W*W*W)+pow(rho,5)*(W*W*W*W*W<wbr>);<br>
<br>
- vec d = SW.diag();<br>
+ vec d(SW.diag());<br>
<br>
direct = sum( d )/n * betas ;<br>
total = accu( SW )/n * betas ;<br>
<br>
<br>
For the other packages, I had less luck as the failures are generally in the<br>
tests and often require packages I have not installed (here at home, and for<br>
technical I don't currently have access to the machine where I usually run<br>
the tests).<br>
<br>
Sp I would be grateful if the maintainers of packages<br>
<br>
biglasso, bigstatsr, netdiffuseR, repolr<br>
<br>
(or also any interested volunteers) could take a peek. It seems of the now<br>
over 400 CRAN package using RcppArmadillo, all others are fine too -- and I<br>
suspect these four also need only small and simple changes. Happy to discuss<br>
here.<br>
<br>
<br>
Cheers, Dirk<br>
<span class="m_7561838855691988328HOEnZb"><font color="#888888"><br>
<br>
--<br>
<a href="http://dirk.eddelbuettel.com" rel="noreferrer" target="_blank">http://dirk.eddelbuettel.com</a> | @eddelbuettel | <a href="mailto:edd@debian.org" target="_blank">edd@debian.org</a><br>
</font></span></blockquote></div><br></div></div>