[Rcpp-devel] Overriding 'ARMA_64BIT_WORD 1' in RcppArmadillo
Dirk Eddelbuettel
edd at debian.org
Tue May 3 19:48:29 CEST 2016
Hi George,
On 3 May 2016 at 10:17, George Vega Yon wrote:
| I think I've found a solution for this. I've submitted a pull request here
| https://github.com/RcppCore/RcppArmadillo/pull/88. It basically now allows the
| users to pass -DARMA_64BIT_WORD and compile with it (so ARMA_32BIT_WORD is not
| defined but it is still the default behavior of RcppArmadillo). Will work on
| tests on this repo https://github.com/gvegayon/arma64bit.
The pull request looks good. I know this issue had been bugging you for a
while (and I am really glad we seem to be having a solution for you !!) but
do you recall if I posted here _which_ packages failed when Conrad started to
default to ARMA_64BIT_WORD under C++11 ?
Dirk
|
| Best,
|
| George G. Vega Yon
| +1 (626) 381 8171
| http://www.its.caltech.edu/~gvegayon/
|
| On Thu, Apr 28, 2016 at 6:07 PM, George Vega Yon <g.vegayon at gmail.com> wrote:
|
| Hey!
|
| Since it seems to be the case that we don't know much about what happens,
| I'm making an experiment which consists on an R package to stress
| RcppArmadillo here: https://github.com/gvegayon/arma64bit. Following Dirk's
| advice, I've included the #define ARMA_64BIT_WORD before #include
| <RcppArmadillo.h> (as seen here https://github.com/gvegayon/arma64bit/blob/
| c10b5d4a80b93088da730e45ee8c583feeff2ab6/src/misc.cpp#L2-L3) and
| included CXX_STD = CXX11 in Makevars. After compiling the package and
| asking for a matrix of size 1e5 x 1e5 I get an error, here (so it doesn't
| seem to be working):
|
| > x <- sp_runif(1e5,1e5,.01)
|
| error: SpMat::init(): requested size is too large
| Error: SpMat::init(): requested size is too large
|
| My guess is that #define ARMA_32BIT_WORD is overriding #define
| ARMA_64BIT_WORD. I'm an R user that jumped to Rcpp a while ago so I still
| don't fully understand how things work under the hood with R + Rcpp. Any
| clues?
|
| Best,
|
| sessionInfo()
|
| R version 3.2.5 (2016-04-14)
| Platform: x86_64-pc-linux-gnu (64-bit)
| Running under: Ubuntu 14.04.4 LTS
|
| locale:
| [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=
| en_US.UTF-8
| [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=
| en_US.UTF-8
| [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
|
| [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8
| LC_IDENTIFICATION=C
|
| attached base packages:
| [1] stats graphics grDevices utils datasets methods base
|
| other attached packages:
| [1] arma64bit_0.1.0.9000 Matrix_1.2-5
|
| loaded via a namespace (and not attached):
| [1] Rcpp_0.12.4 lattice_0.20-33 digest_0.6.9 withr_1.0.1
| grid_3.2.5
| [6] covr_2.0.1 git2r_0.14.0 magrittr_1.5 lazyeval_0.1.10
| rstudioapi_0.5
| [11] whisker_0.3-2 rex_1.1.1 rmarkdown_0.9.5 devtools_1.11.1
| tools_3.2.5
| [16] yaml_2.1.13 htmltools_0.3.5 memoise_1.0.0
|
| George G. Vega Yon
| +1 (626) 381 8171
| http://www.its.caltech.edu/~gvegayon/
|
| On Thu, Mar 31, 2016 at 12:37 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
|
|
| On 29 March 2016 at 15:55, George Vega Yon wrote:
| | Hi there,
| |
| | A couple of days ago I included this comment on a Stackoverflow
| question about
| | "too large" matrices in RcppArmadillo
| | I wonder if we can use this in a package. After trying to compile the
| package
| | with and without PKG_CXXFLAGS=-DARMA_64BIT_WORD I'm still not able to
| create
| | sparse matrices bigger than 65,535 x 65,535. I've realized that the
| | RcppArmadilloConfig.h (version 0.6.600.4.0) file has ARMA_32BIT_WORD
| 1 so it
| | actually prevents using ARMA_64BIT_WORD (I think?). Any way I can
| solve this,
| | or I just have to deal with R (3.2.4 Revised (2016-03-16 r70336)) not
| been able
| | to handle 64INT?
| |
| | http://stackoverflow.com/questions/16159174/
| | large-spmat-object-with-rcpparmadillo
| |
| | For which Dirk answered:
| |
| | Try locally setting ARMA_64BIT_WORD (check how) -- I can't at the
| package level
| | as it breaks existing code. It will likely help you. If you set it
| before the #
| | include <RcppArmadillo.h> things may just work. That at least is the
| pln. (And
| | you should have asked on rcpp-devel to get more than two eyeballs on
| this...)
| | And yes the other answer basically says the same thing. – Dirk
| Eddelbuettel 2
| | days ago
| |
| | Now I'm asking again but on rcpp-devel =). The issue is that I'm
| concerned on
| | the part that Dirk says that "things may just work". I was reading
| about R not
| | been able to handle int64 and its implications for big matrices, and
| so I'm a
| | little concerned that if passing ARMA_64BIT_WORD works I may loose
| some
| | information on the way. For example, if I allow a users creating and
| | manipulating 1e6 x 1e6 sparse matrices, is there any chance for them
| to loose
| | information?
|
| It is entirely possibly that nobody stresses RcppArmadillo this way and
| that
| setting ARMA_64BIT_WORD gets unset. I only know that I cannot set it
| globally now as some packages (still) depend on ARMA_32BIT_WORD.
|
| So I encourage you to experiment and to try locally. Maybe something
| can
| (and even should) be improved.
|
| Dirk
|
|
| |
| | Thanks,
| |
| | George G. Vega Yon
| | +1 (626) 381 8171
| | http://www.its.caltech.edu/~gvegayon/
| | _______________________________________________
| | Rcpp-devel mailing list
| | Rcpp-devel at lists.r-forge.r-project.org
| | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/
| rcpp-devel
|
| --
| http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
|
|
|
|
--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
More information about the Rcpp-devel
mailing list