[Rcpp-devel] Inline and Parallel computing

[Dirk Eddelbuettel]

(Resending, forgot to CC list on first reply, Sorry --Dirk)

Marie,

On 19 April 2012 at 16:11, Marie Auger-Methe wrote:
| Hi list,
| 
| sorry for bombarding the list with questions.
| I'm trying to use Rcpp via the inline package and to use parallel computing at
| the same time. But I get this error:
| 
| first error: NULL value passed as symbol address
| I've read a few posts (not related to parallel computing) discussing this error but I couldn't link it back to my specific problem.
| 
| Here is an example:
| 
| library(inline)
| library(parallel)
| 
| # A silly Rcpp function
| sillyscr <- '
|   int x = as<int>(i);
|   double y = 5.6;
|   NumericVector j(1);
|   j[0] =  x + y;
| 
|   return j;
| '
| silly <- cxxfunction(signature(i = "int"), body = sillyscr, plugin = "Rcpp")
| silly(1) # Works!
| 
| # Equivalent function in R to make sure the problem is not the parallel computing
| silly_notcpp <- function(i){
|   j <- i +5.6
|   return(j)
| }
| silly_notcpp(1) # Works !
| 
| 
| M <- detectCores()
| cl <- makeCluster(M)
| clusterExport(cl, varlist=list("silly","silly_notcpp"))
| res1 <- parSapply(cl, 1:10,silly_notcpp) # Works!
| res1
| res2 <- parSapply(cl, 1:10,silly) # Does not work !
| res2
| stopCluster(cl)
| 
| Many thanks!

inline and cxxfunction create what looks to you like a normal function in the
current session, here this function is called 'silly'.  It really does a
.Call() of some object code compiled in the session temp directory.

While you send 'silly' to the nodes, I suspect that what gets called does
not. Recall that R really is single threaded, and that parallel (just like
snow) creates new sessions for the nodes, and new session temp directories.
Those have 'silly', but not the code it calls (eg the object code etc).

For 'real applications', we generally suggest a package.
Rcpp.package.skeleton() can take silly as an argument to aid you in building
package.  

Else, you may have to have the cxxfunction() call, and hence the 'sillyscr'
text variable, on each node.

The r-sig-hpc lists discusses some of these issues (about how these simple
parSapply calls can fail for lack of local resources) every now and then.

Dirk

-- 
R/Finance 2012 Conference on May 11 and 12, 2012 at UIC in Chicago, IL
See agenda, registration details and more at http://www.RinFinance.com


-- 
R/Finance 2012 Conference on May 11 and 12, 2012 at UIC in Chicago, IL
See agenda, registration details and more at http://www.RinFinance.com