[Rcpp-devel] Overriding 'ARMA_64BIT_WORD 1' in RcppArmadillo

[George Vega Yon]

Hey!

Since it seems to be the case that we don't know much about what happens,
I'm making an experiment which consists on an R package to stress
RcppArmadillo here: https://github.com/gvegayon/arma64bit. Following Dirk's
advice, I've included the #define ARMA_64BIT_WORD before #include
<RcppArmadillo.h> (as seen here
https://github.com/gvegayon/arma64bit/blob/c10b5d4a80b93088da730e45ee8c583feeff2ab6/src/misc.cpp#L2-L3)
and included CXX_STD = CXX11 in Makevars. After compiling the package and
asking for a matrix of size 1e5 x 1e5 I get an error, here (so it doesn't
seem to be working):

> x <- sp_runif(1e5,1e5,.01)

error: SpMat::init(): requested size is too large
Error: SpMat::init(): requested size is too large

My guess is that #define ARMA_32BIT_WORD is overriding #define
ARMA_64BIT_WORD. I'm an R user that jumped to Rcpp a while ago so I still
don't fully understand how things work under the hood with R + Rcpp. Any
clues?

Best,

sessionInfo()

R version 3.2.5 (2016-04-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
 LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C

[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] arma64bit_0.1.0.9000 Matrix_1.2-5

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.4     lattice_0.20-33 digest_0.6.9    withr_1.0.1
grid_3.2.5
 [6] covr_2.0.1      git2r_0.14.0    magrittr_1.5    lazyeval_0.1.10
rstudioapi_0.5
[11] whisker_0.3-2   rex_1.1.1       rmarkdown_0.9.5 devtools_1.11.1
tools_3.2.5
[16] yaml_2.1.13     htmltools_0.3.5 memoise_1.0.0

George G. Vega Yon
+1 (626) 381 8171
http://www.its.caltech.edu/~gvegayon/

On Thu, Mar 31, 2016 at 12:37 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

>
> On 29 March 2016 at 15:55, George Vega Yon wrote:
> | Hi there,
> |
> | A couple of days ago I included this comment on a Stackoverflow question
> about
> | "too large" matrices in RcppArmadillo
> | I wonder if we can use this in a package. After trying to compile the
> package
> | with and without PKG_CXXFLAGS=-DARMA_64BIT_WORD I'm still not able to
> create
> | sparse matrices bigger than 65,535 x 65,535. I've realized that the
> | RcppArmadilloConfig.h (version 0.6.600.4.0) file has ARMA_32BIT_WORD 1
> so it
> | actually prevents using ARMA_64BIT_WORD (I think?). Any way I can solve
> this,
> | or I just have to deal with R (3.2.4 Revised (2016-03-16 r70336)) not
> been able
> | to handle 64INT?
> |
> | http://stackoverflow.com/questions/16159174/
> | large-spmat-object-with-rcpparmadillo
> |
> | For which Dirk answered:
> |
> | Try locally setting ARMA_64BIT_WORD (check how) -- I can't at the
> package level
> | as it breaks existing code. It will likely help you. If you set it
> before the #
> | include <RcppArmadillo.h> things may just work. That at least is the
> pln. (And
> | you should have asked on rcpp-devel to get more than two eyeballs on
> this...)
> | And yes the other answer basically says the same thing. – Dirk
> Eddelbuettel 2
> | days ago
> |
> | Now I'm asking again but on rcpp-devel =). The issue is that I'm
> concerned on
> | the part that Dirk says that "things may just work". I was reading about
> R not
> | been able to handle int64 and its implications for big matrices, and so
> I'm a
> | little concerned that if passing ARMA_64BIT_WORD works I may loose some
> | information on the way. For example, if I allow a users creating and
> | manipulating 1e6 x 1e6 sparse matrices, is there any chance for them to
> loose
> | information?
>
> It is entirely possibly that nobody stresses RcppArmadillo this way and
> that
> setting ARMA_64BIT_WORD gets unset.   I only know that I cannot set it
> globally now as some packages (still) depend on ARMA_32BIT_WORD.
>
> So I encourage you to experiment and to try locally.  Maybe something can
> (and even should) be improved.
>
> Dirk
>
>
> |
> | Thanks,
> |
> | George G. Vega Yon
> | +1 (626) 381 8171
> | http://www.its.caltech.edu/~gvegayon/
> | _______________________________________________
> | Rcpp-devel mailing list
> | Rcpp-devel at lists.r-forge.r-project.org
> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20160428/8543a09b/attachment.html>