[Rcpp-devel] Inline and Parallel computing

[Marie Auger-Methe]

On 19/04/2012 4:35 PM, Dirk Eddelbuettel wrote:
> On 19 April 2012 at 16:11, Marie Auger-Methe wrote:
> | Hi list,
> |
> | sorry for bombarding the list with questions.
> | I'm trying to use Rcpp via the inline package and to use parallel computing at
> | the same time. But I get this error:
> |
> | first error: NULL value passed as symbol address
> | I've read a few posts (not related to parallel computing) discussing this error but I couldn't link it back to my specific problem.
> |
> | Here is an example:
> |
> | library(inline)
> | library(parallel)
> |
> | # A silly Rcpp function
> | sillyscr<- '
> |   int x = as<int>(i);
> |   double y = 5.6;
> |   NumericVector j(1);
> |   j[0] =  x + y;
> |
> |   return j;
> | '
> | silly<- cxxfunction(signature(i = "int"), body = sillyscr, plugin = "Rcpp")
> | silly(1) # Works!
> |
> | # Equivalent function in R to make sure the problem is not the parallel computing
> | silly_notcpp<- function(i){
> |   j<- i +5.6
> |   return(j)
> | }
> | silly_notcpp(1) # Works !
> |
> |
> | M<- detectCores()
> | cl<- makeCluster(M)
> | clusterExport(cl, varlist=list("silly","silly_notcpp"))
> | res1<- parSapply(cl, 1:10,silly_notcpp) # Works!
> | res1
> | res2<- parSapply(cl, 1:10,silly) # Does not work !
> | res2
> | stopCluster(cl)
> |
> | Many thanks!
>
> inline and cxxfunction create what looks to you like a normal function in the
> current session, here this function is called 'silly'.  It really does a
> .Call() of some object code compiled in the session temp directory.
>
> While you send 'silly' to the nodes, I suspect that what gets called does
> not. Recall that R really is single threaded, and that parallel (just like
> snow) creates new sessions for the nodes, and new session temp directories.
> Those have 'silly', but not the code it calls (eg the object code etc).
>
> For 'real applications', we generally suggest a package.
> Rcpp.package.skeleton() can take silly as an argument to aid you in building
> package.
>
> Else, you may have to have the cxxfunction() call, and hence the 'sillyscr'
> text variable, on each node.
>
> The r-sig-hpc lists discusses some of these issues (about how these simple
> parSapply calls can fail for lack of local resources) every now and then.
>
> Dirk
>
Thank you for the prompt reply. I was afraid the answer was to make a 
real package. No choice to learn it now.
I've tried your second option but it didn't work.

Many thanks,

Marie