[Rcpp-devel] Stack imbalance warning when using Rcpp and OpenMP

[Dirk Eddelbuettel]

Michael,

On 26 July 2011 at 23:01, Michael Braun wrote:
| Dirk:
| 
| I am sorry I was not clearer about what I am trying to accomplish.  The issue is with operating on elements of an Rcpp::List object in parallel.

R is very famously single-threaded.  So when I experimented with OpenMP, I
tried a very basic and careful approach which, in pseudo code, does:

   function foo() {

      inits: get data from R, stick it into C++ data structures

      do work, possibly with OpenMP

      synchronise 

      pass data back into R data structure, return

   }

| Before posting to the list, I had looked at the example to which you refer.
| But I am missing where the example does anything with an Rcpp object in
| parallel at all. 

See above, maybe we shouldn't try to work in parallel with an R object. I'd
be happy to see an 'existence proof' example that does it.

| Nor do I see any instance of thread synchronization. 

Point taken. I played with some in my own experimentations trying to get
RcppDE working in parallel (and I still have unsolved issues there
w.r.t. RNGs under OpenMP).

| So I am not sure how the example helps.  

Well it does show how to use OpenMP and it uses a clever class and interrupt
handler to even allow you to interrupt out. That is neat.

| I have already tried the ordered
| directive (segfaults every time), and the critical directives around the
| call to func (no change).  I don't know what else I could do with OpenMP;
| this is such a simple example.  Perhaps you have something specific in
| mind.  Is there any way you (or someone else on the list) could give me
| another hint? 

Have you tried adding

  #pragma omp barrier

and the end of your OpenMP to force thread synchronisation?


Otherwise, try maybe developing the core of your parallel algo in a
standalone program, outside of R and Rcpp, to "make it work" and then worry
about connecting it to R via Rcpp.  One battle at a time...

Cheers, Dirk
 
| Thanks,
| 
| Michael
| 
| 
| 
| 
| On Jul 26, 2011, at 8:26 PM, Dirk Eddelbuettel wrote:
| 
| > 
| > Michael,
| > 
| > On 26 July 2011 at 17:08, Michael Braun wrote:
| > | I have an R list that I want to pass to C++ code through Rcpp, and then process each of the list elements with another function, in parallel through OpenMP.  Here is a simplified example in which I compute the square root of a number in each list element:
| > | 
| > | // file omp.cpp
| > | 
| > | #include <Rcpp.h>
| > | #include <omp.h>
| > | 
| > | double func(SEXP Y_) {
| > |   using namespace Rcpp;
| > |   double Y = as<double>(Y_);
| > |   double res = sqrt(Y);
| > |   return(res);
| > | }
| > | 
| > | RcppExport SEXP omp3 (SEXP R_) {
| > | 
| > |   BEGIN_RCPP
| > | 
| > |  using namespace Rcpp;
| > | 
| > |  List R = R_;
| > |  int n = R.size();
| > |  int i;
| > | 
| > |  NumericVector X(n);
| > |  
| > | #pragma omp parallel shared(R, n, X) private(i)
| > |  {
| > | #pragma omp for
| > |  for (i=0; i<n; i++) {
| > |    X(i) = func(R(i));
| > |  }
| > |  }
| > |  
| > |  return(wrap(X));
| > | 
| > |  END_RCPP
| > | 
| > | }
| > | 
| > | 
| > | The R code to call the omp3 C++ function is:
| > | 
| > | dyn.load("omp.so")
| > | 
| > | R <- vector("list",5)
| > | for(i in 1:5) R[[i]] = i  
| > | 
| > | res <- .Call("omp3",R)
| > | 
| > | dyn.unload("omp.so")
| > | 
| > | 
| > | When I run this program, I almost always get stack imbalance warnings like:
| > | 
| > | Warning: stack imbalance in '.Call', 33 then 32
| > | Warning: stack imbalance in '<-', 31 then 30
| > | Warning: stack imbalance in 'eval.with.vis', 27 then 26
| > | Warning: stack imbalance in '.Internal', 26 then 25
| > | Warning: stack imbalance in '<-', 20 then 19
| > | Warning: stack imbalance in '{', 18 then 17
| > | Warning: stack imbalance in 'if', 16 then 15
| > | Warning: stack imbalance in '{', 14 then 13
| > | Warning: stack imbalance in 'for', 8 then 7
| > | 
| > | I say almost because it does not happen all the time, but if I run the same script a few more times, I will invariably get a stack imbalance warning. (Also, the numbers are not always the same).
| > | 
| > | But there's more.  If I run the program a bunch of times, eventually I get:
| > | 
| > | Lost warning messages
| > | > Error in eval.with.vis(expr, envir, enclos) : 
| > |   REAL() can only be applied to a 'numeric', not a 'NULL'
| > | Error during wrapup: C stack usage is too close to the limit
| > | 
| > | 
| > | Once I get this error, I will no longer get stack imbalance warnings, but I cannot predict how many attempts it would take before the C stack usage error comes up.
| > | 
| > | And if I am really lucky, I'll get memory not mapped segfaults, but not every time.
| > | 
| > | I have run this on my Mac Pro with both g++ 4.6 and Intel C++ compiler 12.0.4.  I have also run it on a Red Hat Linux machine with g++ 4.4 (the segfaults happen much more frequently under Linux than under Mac OSX).  The stack imbalance warnings come up on both machines, with all compilers.
| > | 
| > | I also wonder if this is related to another problem with Rcpp and OpenMP that I posted back in late March.  In that case, the problem was using an old compiler; updating the compiler solved the compilation problem.   The difference between that case and this example is calling the second function with a SEXP as an argument. This example compiles, but does not run.
| > | 
| > | I know that Rcpp is rigorously tested, so the problem is probably something that I am doing wrong.  In particular, I wonder if each element of the list should be passed as a SEXP, or if there is a better way to do it.  In any case, I greatly appreciate your assistance.
| > 
| > I would have to agree with your conjecture here. You are simply not being
| > careful enough about OpenMP usage.  And R tells you that there are still
| > dynamically created objects floating around.
| > 
| > I would recommend stronger synchronisation before and particularly after your
| > OpenMP block.  For a working example of OpenMP with Rcpp, see the directory
| > examples/OpenMP/ which was added in the 0.9.5 release.
| > 
| > | (Also, I deeply apologize for not using the inline package with the example.  I could not figure out how to include the two different functions.)
| > 
| > See help(cxxfunction) and particularly the "includes=" argument. That said,
| > for OpenMP you need to pass -fopenmp and we currently do not have an option
| > for that in cxxfunction.
| > 
| > Hope this helps,  Dirk
| > 
| > 
| > | Thanks,
| > | 
| > | Michael
| > | 
| > | 
| > | 
| > | 
| > | 
| 
| 
| 
| 
| xapplication/pkcs7-signatu [Click mouse-2 to save to a file]

-- 
Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com