[Rcpp-devel] C++ issue only spotted by CRAN and not locally or via rhub

[Martyn Plummer]

I have good news and bad news.

I found the source of the error. In your wrapper function

logsumexp <- function(x, a = NULL)
{
  x <- as.matrix(x)
  a <- if(is.null(a)) rep(double(0), ncol(x)) else as.vector(a)
  logsumexp_Rcpp(x,a)
}

the expression rep(double(0), ncol(x)) is not doing what you think it is
doing. It returns a numeric vector of length 0, whereas you actually want a
vector of zeros of length ncol(x). Note that the same issue affects your
softmax function.

I don't know what Rcpp does when you try add a numeric vector of length 0,
but at R level it is quite a destructive operation

> c(1,2) + numeric(0)
numeric(0)

This is not the behaviour you want. Here is a modified wrapper that makes
the ASAN error message go away and adds some checks that the arguments are
conforming.

logsumexp <- function(x, a = NULL)
{
  x <- as.matrix(x)
  a <- if(is.null(a)) rep(0.0, ncol(x)) else as.vector(a)
  if (length(a) != ncol(x)) {
      stop("Non-conforming arguments in logsumexp")
  }
  logsumexp_Rcpp(x,a)
}

That's the good news. The bad news is that you have a lot of memory leaks,
so this is still not going to pass R CMD check with ASAN. You're going to
need Dirk's container.

Martyn

On Mon, 5 Dec 2022 at 09:07, Luca Scrucca <luca.scrucca at unipg.it> wrote:

> Dear all,
>
> the "mclustAddons" R package contains some C++ functions included via
> Rcpp.
> It's not the first time I've done this and, although I'm not a C++ expert,
> I've been able to get it right thanks to the wonderful Rcpp facilities.
>
> Before submitting to CRAN the new version, I tested the package both
> locally (on MacOS - Intel machine) and via rhub on many platforms (debian,
> fedora, ubuntu, windows) with different versions of R (release, develop),
> including memory sanitizers/valgrind.
> The "mclustAddons" package gives me no problems on any tested
> platform/version.
> Unfortunately when I submit it to CRAN the problems start to arise. Below
> are the links to the log files:
>
> - clang-ASAN <
> https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-ASAN/mclustAddons>
> - gcc-ASAN <
> https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/mclustAddons>
>
> After several e-mails with CRAN maintainers they suggest me to set up an
> environment to safely reproduce the ASAN reports:
> See https://www.stats.ox.ac.uk/pub/bdr/memtests/README.txt
> This of course would be the next thing to do, and I have already asked for
> help to colleagues at the Physics Dept.
>
> The code causing the problem is reported below and the offending line is
> the one with "xa = x.row(i) + a;".
> Since I can't replicate the issue and test it, I can't resubmit the
> package to CRAN.
> Perhaps a C++ expert will spot the problem at first sight...
>
> However, the main problem I have (with CRAN) is that, despite all the
> efforts that one can reasonably made, it is impossible for a maintainer of
> a package to be sure that the package will be tested correctly on CRAN.
>
> Thanks in advance for your help.
>
> Luca
>
> --
> --------------------------------------
> Luca Scrucca, PhD
> Associate Professor of Statistics
> Department of Economics
> University of Perugia
> Via A. Pascoli, 20
> 06123 Perugia (Italy)
> Tel +39-075-5855231
> Fax +39-075-5855950
> E-mail luca.scrucca at unipg.it
> Web page http://www.stat.unipg.it/luca
> --------------------------------------
>
>
>
> -- logsumexp.cpp ------------------------------------------------------
> #include <Rcpp.h>
> using namespace Rcpp;
>
> // [[Rcpp::export]]
> NumericVector logsumexp_Rcpp(NumericMatrix x, NumericVector a)
> {
> // Efficiently computes log-sum-exp(x+a)
> // x = matrix (n x d)
> // a = vector (d)
>
>  double n = x.nrow();
>  double d = x.ncol();
>  NumericVector lse(n);
>  NumericVector xa(d);
>  double m = 0.0;
>  //
>  for (int i = 0; i < n; i++)
>  {
>    xa = x.row(i) + a;
>    m = max(xa);
>    lse[i] = m + log(sum(exp(xa-m)));
>  }
>  return lse;
> }
>
> /*** R
>
> x = structure(c(-4.19768936334846, -23.3334911845962, -5.36851445858848,
> -15.2896460085004, -3.06018772423303, -13.2857737610833,
> -3.69968442181734,
> -5.33468420156765, -22.954092839643, -3.03420360101199, -23.3405056884397,
> -2.6395810621981, -16.4338853853632, -3.23305725595493, -50.7400647615373,
> -7.76487486677727, -57.9522847161203, -26.7659048640944,
> -2.41249310267583,
> -44.9591733534474), dim = c(10L, 2L))
> pro = c(0.644072589572232, 0.355927410427768)
>
> logsumexp_Rcpp(x, log(pro))
> */
>
> -----------------------------------------------------------------------
>
>
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20221205/4bd71fa2/attachment.html>