[Rcpp-devel] Fwd: CRAN submission growfunctions 0.1
Dirk Eddelbuettel
edd at debian.org
Sun Aug 10 03:37:48 CEST 2014
Hi Terrance,
On 9 August 2014 at 20:05, terrance savitsky wrote:
| darn. i R CMD check'd --as-cran under R-devel and also loaded the minimal
| package, sparseList_1.1, the example and test run without issue
| [icon_10_ge] sparseList_1.1.tar.gz
Read the section "4.3 Checking memory access" of Writing R Extensions.
You need a version of R compiled with "-fsanitize=address" as described there.
My Docker image provides that, hence my links in the first email.
| . So I'm stumped why the test is failing for you and not me. I attach the
| file, again, just in I included the wrong version. Sorry I wasn't able to get
| you a link that you could wget. I work in the U.S. federal system and am
| limited by security protocols in what I'm able to do. Thanks for being
| patient.
|
| Since I include the data object (which requires both the spam and Matrix
| packages), I can/will remove the dependencies on those packages. Since I have
| an example written into the documentation that runs on check, I will remove the
| test I added (and, therefore, testthat). I will do this in the next day or
| so, assuming what I attached still doesn't work.
The minimal example we asked to provided is as follows
library(sparseList) ## just to get the data set
data(C) ## gets a list, you only need the first element
## function to invoke as<spmat>() on supplied argument
cppFunction("arma::sp_mat asSpMat(SEXP S) { return as<arma::sp_mat>(S); }", depends="RcppArmadillo")
## call it: boom (if using -fsanitize=address)
asSpMat(C[[1]])
The open question now is whether your dgCMatrix is valid (likely) in which
case our code may be wrong (yet someone who actually __used ans understands
sparse matrix storage conventions__ needs to look at it). Our code survived
the tests we threw at it, as well as use by other people using sparse matrix.
Or you created a faulty dgCMatrix, in which case the bug is yours.
Your didn't show how you create 'C'. So we can't really tell.
And this point it would be good if you could do some digging. You're closest
to your issue.
Dirk
| Stepping back, the code works in both the minimal package and growfunctions -
| it inputs a list of dgCMatrix objects, assigns the elements to a field<sp_mat>
| object and then writes it back out to a List object. The elements of the
| outputted list object at the R layer are, indeed, dgCMatrix objects. So
| 'yes', the script you referenced does create a valid dgCMatrix object and i've
| verified such.
|
| As an aside, the C++ file embeds an R script that Kevin asked for if that is
| useful.
You need to run it under an executable instrumented for memory debugging to
trigger the bug.
Dirk
|
| terrance.
|
|
| On Sat, Aug 9, 2014 at 7:14 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
|
|
| Terrance,
|
| On 9 August 2014 at 18:23, terrance savitsky wrote:
| | Hi Dirk, The revised version of this toy package, sparseList, does
| exactly what
| | you request. I've included a data set that is of the same construction
| (of
| | dgCMatrix elements from R::Matrix wrapped into a list object) and have
| added
| | both an example and test that call this data. The package is minimal,
| with a
| | simple .cpp function (that reads in the list of dgCMatrix objects and
| | writes the elements to an arma::field<arma:sp_mat>) to isolate the
| source of
| | the error in growfunctions.
| |
| | I attach the tarball, which you may also access through the following
| URL:
| |
| | https://drive.google.com/file/d/
| | 0B0DIDcaf6yMwZDRDZUNLd0lBbmZ3OHlHSzIyeV9hZ3RxZ0xr/edit?usp=sharing
|
| Did not work via wget, required a webpage visit and download.
|
| Package has extraneous dependencies (you do not need testthat to call a
| function).
|
| Moreover, testthat seems to pivot over to /usr/bin/R rather than Rdevel
| (not
| your issues, but you're not helping as it is not a minimally reproducible
| example).
|
| You also still import / depend on spam. Why?
|
| | I hope this gives you what you need and thanks very much for looking into
| this.
|
| I does blow up when I do
|
| > library(sparseList)
| Loading required package: Rcpp
| > data(C)
| > sparse_list <- manip_sparseList(C)
|
| and it similarly blows up when R CMD check runs the examples.
|
|
|
| But the question is whether your code which does
|
| ## ...
|
| ## convert to dcgmatrices (under Matrix package) required for sparse
| representation
| ## as sp_mats in RcppArmadillo
| C[[k]] <- as(C[[k]], "dgCMatrix")
|
|
| creates a valid dgCMatrix. I do not know.
|
| RcppArmadillo is used by 77 packages on CRAN. Prof Ripley checks all of
| those via valgrind and ASAN. Only yours threw an error.
|
| I am not yet convinced this is our issue. I could of course be wrong.
|
| Anyway, I won't be able to look at this for few days. Moreover, I do not
| use
| sparse matrices -- so you may need to rely on someone else to debug this
| for
| you, or do it yourself.
|
| Sorry, Dirk
|
| |
| | Terrance
| |
| | [icon_10_ge] sparseList_1.1.tar.gz
| |
| |
| |
| | On Sat, Aug 9, 2014 at 5:25 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
| |
| |
| | Hi Terrance,
| |
| | On 9 August 2014 at 16:58, terrance savitsky wrote:
| | | Hi Guys,
| | |
| | | I embed a data list object of dgCMatrix elements (produced in the
| same
| | manner
| | | as used in package, growfunctions) in an .RData file located in the
| | sparseList
| | | \data folder. I'm not sure what script and data ASAN uses to
| conduct the
| | memory
| | | check. I'm guessing the package examples are run. So I added an
| example
| | that
| | | loads the .RData object embedded with this toy package,
| sparseList.
| | Please
| | | pardon me if my understanding is off. I also embedd an R script
| in the
| | single
| |
| | It's not complicated.
| |
| | 'R CMD check' needs a code path that does this, which can be as
| simple as
| | an
| | example section in the manual page (mayne you meant this by 'added an
| | example'). Or something tickled via a test script in tests/.
| |
| | You submitted a CRAN package. It has testable code. It failed. We now
| need
| | a
| | _minimal reproducible example_, ideally without a long tail of
| | dependencies,
| | to examine it.
| |
| | | toy package .cpp file that generates the same data (list object of
| | dgCMatrix
| | | objects created using R::Matrix) and runs an example.
| | |
| | | Terrance
| | |
| | |
| | | [icon_10_ge] sparseList_1.1.tar.gz
| |
| | Can you update to a version which loads data and passes down to the
| code
| | which was seen to tickle this?
| |
| | And then place it somewhere where we can get it with a simple wget or
| curl
| | call.
| |
| | Thanks, Dirk
| |
| |
| | |
| | | On Sat, Aug 9, 2014 at 3:02 PM, Kevin Ushey <kevinushey at gmail.com>
| wrote:
| | |
| | | Hi Terrance,
| | |
| | | Can you please try to construct a reproducible example based on
| the
| | | (tagged, potentially offending) line of code -- that is,
| | |
| | | #4 0x7f79b6541df0 in IGMRFDPMIX
| | | /data/gannet/ripley/R/packages/tests-49x/growfunctions/src/
| | dpmix.cpp:63
| | |
| | | which has the line:
| | |
| | | C(k,0) = as<sp_mat>(Cr[k]);
| | |
| | | Can you construct a small, reproducible example using objects
| 'C',
| | | 'Cr' as you construct in the package, and then see if that
| gives a
| | | similar error?
| | |
| | | Preferably you can construct this using Rcpp Attributes, so an
| | example
| | | of the form
| | |
| | | // [[Rcpp::depends(RcppArmadillo)]]
| | |
| | | #include <RcppArmadillo.h>
| | | using namespace Rcpp;
| | |
| | | // [[Rcpp::export]]
| | | SEXP test(SEXP C, SEXP Cr) {
| | | ... use C, Cr as above...
| | | }
| | |
| | | /*** R
| | | ... construct C, Cr on the R side as necessary ...
| | | */
| | |
| | | That way, we can 'sourceCpp' it and immediately investigate.
| | |
| | | Thanks,
| | | Kevin
| | |
| | | On Sat, Aug 9, 2014 at 11:31 AM, terrance savitsky <
| tds151 at gmail.com>
| | | wrote:
| | | > Dirk, Thanks much for taking the time to investigate this
| issue.
| | I've
| | | > created a package (sparseList_1.0.tar.gz) with a minimal
| example of
| | what
| | | > writing from a list of dgCMatrix matrices at the R layer to a
| field
| | | <sp_mat>.
| | | > I had intended to install boot2docker and use load your
| R-devel
| | set-up to
| | | a
| | | > container so that I could replicate the ASAN-generated error.
| Its
| | taking
| | | me
| | | > longer than I'd hoped, however, so - for now - I include the
| | package
| | | tarball
| | | > with the minimal example and also an R script
| | (sparse_list_to_field.R) so
| | | > that you may see what the function does.
| | | > Terrance
| | | > sparse_list_to_field.R
| | | >
| | | > sparseList_1.0.tar.gz
| | | >
| | | >
| | | >
| | | > On Sat, Aug 9, 2014 at 11:50 AM, Dirk Eddelbuettel <
| edd at debian.org>
| | | wrote:
| | | >>
| | | >>
| | | >> Terrance,
| | | >>
| | | >> So I took a look with one of the prebuilt versions of R
| using the
| | Docker
| | | >> container I prepare (see the blog / sanitizers write-up I
| linked
| | to) as
| | | I
| | | >> realized that we had a Rcpp Gallery post similar to what
| that as<>
| | | >> conversion
| | | >> does.
| | | >>
| | | >> And no issue with that code.
| | | >>
| | | >> From a brief look at your package it seems like you are
| pushing
| | the
| | | >> envelope
| | | >> a lot harder with fields and list of sparse matrices ( in
| | dpmix.cpp,
| | | where
| | | >> another ASAN log pointed to ) so you may have to debug
| around
| | there.
| | | >>
| | | >> A small reproducible example would surely help us. So far I
| don't
| | have
| | | one
| | | >> for as<sp_mat>.
| | | >>
| | | >> Dirk
| | | >>
| | | >> --
| | | >> http://dirk.eddelbuettel.com | @eddelbuettel |
| edd at debian.org
| | | >
| | | >
| | | >
| | | >
| | | > --
| | | > Thank you, Terrance Savitsky
| | | >
| | | > _______________________________________________
| | | > Rcpp-devel mailing list
| | | > Rcpp-devel at lists.r-forge.r-project.org
| | | > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/
| | rcpp-devel
| | |
| | |
| | |
| | |
| | | --
| | | Thank you, Terrance Savitsky
| |
| | --
| | http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| |
| |
| |
| |
| | --
| | Thank you, Terrance Savitsky
|
| --
| http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
|
|
|
|
| --
| Thank you, Terrance Savitsky
--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
More information about the Rcpp-devel
mailing list