[Rcpp-devel] Sugar/inline and RTools

Romain Francois romain at r-enthusiasts.com
Mon Dec 13 11:59:44 CET 2010


Le 13/12/10 11:48, Wray, Christopher a écrit :
>> But now that I read your initial example more carefully, you could also do:
>>
>> fx<- cxxfunction( , '
>>     RNGScope scope ;
>>      NumericMatrix hrcc( 3, 10) ;
>>      for( int i=0; i<10; i++) hrcc(_,i) = runif(3) ;
>>      return hrcc ;
>> ', plugin = "Rcpp" )
>> fx()
>
> Yes, apologies my sugar example was a bit mis-typed and not very clear. thanks for your suggestions and *disambiguating the runif*.
> Your last one (the above) is how i'm doing it for now (but looping over hrcc(i,_)). Although in my real code the "10" is more like "10^8" and the "3" is between 10 and 200.
> so that hrcc is a matrix of (say) rows=10, cols=10^8.
> Hence trying to avoid using "for" altogether. I think looping over the smaller "row" dimension would probably be ok...I need to test this. But thanks.

it might be worth benchmarking. If I had to guess I'd say that the 
version using the small function "make_matrix" is the most efficient 
since it does less memory allocation.

Also, given how matrices are stored in memory, it is more efficient to 
work in terms of columns.

>> You don't necesarilly have to make a package for this. You can register
>> an inline plugin "manually".
>>
>> plug<- function(){
>>         settings<- getPlugin( "Rcpp" )
>>        settings$env[["PKG_CPPFLAGS"]]<- "whatever you need"
>>         settings
>> }
>> registerPlugin( "omp", plug )
>
> Nice. Thats exactly what I was looking for, ta!
>
> Re: RTools/windows. I think for the moment - I will be moving my "problem" over to linux...whereupon my problem disappears.
> Even if I could get around openmp support lacking in rtools, I guess there will always be the potential to come across
> other similar issues..given the "min" distribution.
>
> Thanks for all suggestions/comments. chris.

No trouble. Feel free to come again.

Romain

> ________________________________________
> From: rcpp-devel-bounces at lists.r-forge.r-project.org [rcpp-devel-bounces at lists.r-forge.r-project.org] on behalf of Romain Francois [romain at r-enthusiasts.com]
> Sent: 13 December 2010 10:07
> To: rcpp-devel at lists.r-forge.r-project.org
> Subject: Re: [Rcpp-devel] Sugar/inline and RTools
>
> Hello,
>
> Le 12/12/10 16:28, Dirk Eddelbuettel a écrit :
>>
>> Hi Chris,
>>
>> On 12 December 2010 at 15:03, Wray, Christopher wrote:
>> | Hi,
>> | Im using windows7, R2.12, Rtools2.12 and all the latest packages of inline,
>> | rcpp, etc. Over the last few weeks I have managed to get loads of the
>> | examples working, and all makes sense and is pretty clear. I had a couple of
>> | questions/comments:
>> |
>> | 1. RTools: On windows, I understand that Rcpp and R depend in a critical way upon RTools.
>>
>> Yes. While one conceptually could also use other builds of the gcc compiler,
>> in practice this can backfire when library interactions, underscore
>> conventions, ... go wrong.  For experts-only --- while "Depend" may be a
>> strong a term, for us as package authors it is simply an easy and reliable
>> way to follow R Core's lead and assume this toolchain to ensure identical
>> behaviour.  Now, this has one known cost which is ...
>>
>> | I have recently been rewriting some parallelized C++ code that uses omp.h
>> | and the RcppAramdillo plugin.
>>
>> ... that Rtools famously does not support OpenMP. I think this come up both
>> here and on r-devel in the last few weeks.
>>
>> Duncan M's response basically was that the support in Rtools is for what R /
>> R Core needs --- hence no Open MP.  I know of at least one package on CRAN
>> which disables Open MP on Windows (as CRAN doesn't have it) but provides Open
>> MP-enabled windows binaries at its site.
>>
>> | Im also a fan of running code snippits using inline, etc. RTools does not
>> | provide full suport for omp (no real surprise given the "minimum"GW
>> | distribution). So I had to add libpthreads, etc, to my RTools installation
>> | (libgomp is already included) and use "cppargs= -fopenmp" and "libargs =
>> | -lgomp -lpthread" when using "cfunction" from inline. All works fine
>> | though. Is it *only* the core R build process that determines what RTools
>>
>> That is my understanding.
>>
>> | toolchain contains? Could this change with more usage of Rcpp (on windows)?
>> | Duncan Murdoch who maintains RTools pointed me in the right direction, to
>> | find this solution.
>>
>> Yup. If you were to create, maintain, distribute, ... a new toolchain
>> including Open MP / libgomp for Windows it would certainly be appreciated.
>> CRAN may even end up using it.  Or maybe liaise with Duncan M / Brian R and
>> help them. Dunno.
>>
>> Romain and I are not particular heavy users of Windows so we are very
>> unlikely to be volunteering here. I plan to do some work with Open MP for
>> parallelised optimisation but I will surely do my work on Linux.  But if and
>> when I am done there I'd probably be asking the same questions. So let's keep
>> that question in mind for future discussions.
>
> I don't have much to add here. Duncan is definitely the one you should
> negociate with.
>
> I don't use windows, apart from testing that "Rcpp works" and I have not
> used OpenMP either.
>
>> | 2. As far as I can tell the "cxxfunction" does not allow the passing of
>> | *additional* PKG_CPPFLAGS and PKG_LIBS args explictly (these are set from
>> | "env<- settings$env" with "do.call(Sys.setenv, env)" if a plugin is
>> | used). Is there a cleaner way of passing compiler options/flags when using
>> | cxxfunction + plugin? I do appreciate that plugins circumvent the need for
>> | such compiler options, etc, and so I guess this is somewhat of a
>> | pathological example. Right now, I'm using my own hacked version of
>> | cxxfunction to pass the additional flags (in 1.) to get parallel code to
>> | compile (on windows).
>>
>> Maybe you can contribute a clean patch to inline (and its manual page etc)
>> that we can review?  Romain and I are ipso facto maintainers of inline now
>> that Oleg has no new needs whereas we do come across new use cases.
>>
>> Alternatively, you could possibly write yourself (and / or the world via
>> CRAN) a support package 'RcppOMP' whose sole / main purpose it is to provide
>> a plugin for inline.   The hooks are there...
>
> You don't necesarilly have to make a package for this. You can register
> an inline plugin "manually".
>
> plug<- function(){
>          settings<- getPlugin( "Rcpp" )
>          settings$env[["PKG_CPPFLAGS"]]<- "whatever you need"
>          settings
> }
> registerPlugin( "omp", plug )
>
> so that you can then do plugin = "omp"
>
>> | 3. Rcpp::sugar. I have been trying to replicate this R snippet using sugar, but have not had much luck:
>> | R>   sapply(rep(3,100), runif)
>> | This returns a column major matrix of size 3x10.
>> |
>> | I tried (column assigment):
>> |
>> | int N = as<int>(nn); //N=3
>> | NumericMatrix hrcc;
>> | for(int k=0; k<100; k++){
>> |  hrcc(_,k) = sapply(N, runif);}
>> |
>> | But the compiler complains about runif being overloaded and not resolved. I
>> | have checked for examples and docs and looked at the sugar unit tests
>> | also. Am I doing something stupid? Do I need to specify runif more
>> | carefully? Am I using or assiging NumericMatrix incorrectly?
>>
>> Not sure. I could try later. From the top of my head I'd try to split
>> operations first:  have sugar create a vector (all the r* vectors are
>> internally vectorised so runif(N) should give you a vector, no need for
>> sapply); test the vector and then assign the vectors to hrcc.  But you could
>> probably also the whole matrix at once.
>>
>> Dirk
>
> In R, I would do :
>
> matrix( runif( 300), nrow = 3 )
>
> You can sort of do the same in Rcpp :
>
> require(Rcpp)
> require(inline)
>
> fx<- cxxfunction( , '
>       RNGScope scope ;
>       NumericVector y =  runif( 30 ) ;
>       y.attr("dim") = Dimension(3,10) ;
>       return y ;
> ', plugin = "Rcpp" )
> fx()
>
> This is kind of messy because y will only have a vector interface, and
> not look like a matrix, you can perhaps cheat like this:
>
> fx<- cxxfunction( , '
>       RNGScope scope ;
>       NumericMatrix m = make_matrix(3, 10 ) ;
>      return m ;
> ', includes ='
> SEXP make_matrix( int nr, int nc){
>       NumericVector y =  runif( nr*nc) ;
>       y.attr("dim") = Dimension(nr,nc) ;
>       return y ;
> }
> ', plugin = "Rcpp" )
> fx()
>
>
>
> Otherwise, you have to help the compiler by disambiguating the runif
> overload manually:
>
> fx<- cxxfunction( , '
>       RNGScope scope ;
>       List res = sapply( rep(10,3), my_runif ) ;
>       return res ;
> ', includes='
>       NumericVector (*my_runif)(int) = runif ;
> ', plugin = "Rcpp" )
> fx()
>
> The line :
>
>    NumericVector (*my_runif)(int) = runif ;
>
> does the disambiguating.
>
> Note that sugar's sapply does not simplify its output to a matrix. This
> is somewhat too difficult to achieve.
>
>
> But now that I read your initial example more carefully, you could also do:
>
> fx<- cxxfunction( , '
>       RNGScope scope ;
>       NumericMatrix hrcc( 3, 10) ;
>       for( int i=0; i<10; i++) hrcc(_,i) = runif(3) ;
>       return hrcc ;
> ', plugin = "Rcpp" )
> fx()
>
> Romain
>
> --
> Romain Francois
> Professional R Enthusiast
> +33(0) 6 28 91 30 30
> http://romainfrancois.blog.free.fr
> |- http://bit.ly/fT2rZM : highlight 0.2-5
> |- http://bit.ly/gpCSpH : Evolution of Rcpp code size
> `- http://bit.ly/hovakS : RcppGSL initial release
>
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>


-- 
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release




More information about the Rcpp-devel mailing list