[Rcpp-devel] C++ issue only spotted by CRAN and not locally or via rhub

[Luca Scrucca]

Thanks Martyn to spot this. I was so concentrated on C++ code that I overlooked the wrapper. 
However, it seems that simply using 
  rep(as.double(0), length.out = ncol(x)) 
should also be ok (followed by your suggested check for conforming arguments).

For the ASAN I will have to further investigate the issue, including moving from Rcpp to RcppArmadillo as suggested by Sokol and Dirk.

Best,

Luca

> On 5 Dec 2022, at 17:34, Martyn Plummer <martyn.plummer at gmail.com> wrote:
> 
> I have good news and bad news.
> 
> I found the source of the error. In your wrapper function
> 
> logsumexp <- function(x, a = NULL)
> {
>   x <- as.matrix(x)
>   a <- if(is.null(a)) rep(double(0), ncol(x)) else as.vector(a)
>   logsumexp_Rcpp(x,a)
> }
> 
> the expression rep(double(0), ncol(x)) is not doing what you think it is doing. It returns a numeric vector of length 0, whereas you actually want a vector of zeros of length ncol(x). Note that the same issue affects your softmax function.
> 
> I don't know what Rcpp does when you try add a numeric vector of length 0, but at R level it is quite a destructive operation
> 
> > c(1,2) + numeric(0)
> numeric(0)
> 
> This is not the behaviour you want. Here is a modified wrapper that makes the ASAN error message go away and adds some checks that the arguments are conforming.
> 
> logsumexp <- function(x, a = NULL)
> {
>   x <- as.matrix(x)
>   a <- if(is.null(a)) rep(0.0, ncol(x)) else as.vector(a)
>   if (length(a) != ncol(x)) {
>       stop("Non-conforming arguments in logsumexp")
>   }
>   logsumexp_Rcpp(x,a)
> }
> 
> That's the good news. The bad news is that you have a lot of memory leaks, so this is still not going to pass R CMD check with ASAN. You're going to need Dirk's container.
> 
> Martyn
> 
> On Mon, 5 Dec 2022 at 09:07, Luca Scrucca <luca.scrucca at unipg.it> wrote:
> Dear all,
> 
> the "mclustAddons" R package contains some C++ functions included via Rcpp. 
> It's not the first time I've done this and, although I'm not a C++ expert, I've been able to get it right thanks to the wonderful Rcpp facilities. 
> 
> Before submitting to CRAN the new version, I tested the package both locally (on MacOS - Intel machine) and via rhub on many platforms (debian, fedora, ubuntu, windows) with different versions of R (release, develop), including memory sanitizers/valgrind.
> The "mclustAddons" package gives me no problems on any tested platform/version.
> Unfortunately when I submit it to CRAN the problems start to arise. Below are the links to the log files:
> 
> - clang-ASAN <https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-ASAN/mclustAddons>
> - gcc-ASAN <https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/mclustAddons>
> 
> After several e-mails with CRAN maintainers they suggest me to set up an environment to safely reproduce the ASAN reports:
> See https://www.stats.ox.ac.uk/pub/bdr/memtests/README.txt
> This of course would be the next thing to do, and I have already asked for help to colleagues at the Physics Dept. 
> 
> The code causing the problem is reported below and the offending line is the one with "xa = x.row(i) + a;". 
> Since I can't replicate the issue and test it, I can't resubmit the package to CRAN.
> Perhaps a C++ expert will spot the problem at first sight...
> 
> However, the main problem I have (with CRAN) is that, despite all the efforts that one can reasonably made, it is impossible for a maintainer of a package to be sure that the package will be tested correctly on CRAN. 
> 
> Thanks in advance for your help.
> 
> Luca
> 
> -- 
> --------------------------------------
> Luca Scrucca, PhD
> Associate Professor of Statistics
> Department of Economics 
> University of Perugia
> Via A. Pascoli, 20
> 06123 Perugia (Italy)
> Tel +39-075-5855231
> Fax +39-075-5855950
> E-mail luca.scrucca at unipg.it
> Web page http://www.stat.unipg.it/luca
> --------------------------------------
> 
> 
> 
> -- logsumexp.cpp ------------------------------------------------------
> #include <Rcpp.h>
> using namespace Rcpp;
> 
> // [[Rcpp::export]]
> NumericVector logsumexp_Rcpp(NumericMatrix x, NumericVector a)
> {
> // Efficiently computes log-sum-exp(x+a)
> // x = matrix (n x d)
> // a = vector (d)
> 
>  double n = x.nrow();
>  double d = x.ncol();
>  NumericVector lse(n);
>  NumericVector xa(d);
>  double m = 0.0;
>  // 
>  for (int i = 0; i < n; i++)
>  {
>    xa = x.row(i) + a;
>    m = max(xa);
>    lse[i] = m + log(sum(exp(xa-m)));
>  }
>  return lse;
> }
> 
> /*** R
> 
> x = structure(c(-4.19768936334846, -23.3334911845962, -5.36851445858848, 
> -15.2896460085004, -3.06018772423303, -13.2857737610833, -3.69968442181734, 
> -5.33468420156765, -22.954092839643, -3.03420360101199, -23.3405056884397, 
> -2.6395810621981, -16.4338853853632, -3.23305725595493, -50.7400647615373, 
> -7.76487486677727, -57.9522847161203, -26.7659048640944, -2.41249310267583, 
> -44.9591733534474), dim = c(10L, 2L))
> pro = c(0.644072589572232, 0.355927410427768)
> 
> logsumexp_Rcpp(x, log(pro))
> */
> 
> -----------------------------------------------------------------------
> 
> 
> 
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

-- 
--------------------------------------
Luca Scrucca, PhD
Associate Professor of Statistics
Department of Economics 
University of Perugia
Via A. Pascoli, 20
06123 Perugia (Italy)
Tel +39-075-5855231
Fax +39-075-5855950
E-mail luca.scrucca at unipg.it
Web page http://www.stat.unipg.it/luca
--------------------------------------