[Rcpp-devel] Rcpp & Multithreading

[Dirk Eddelbuettel]

Hi again,

On 30 April 2010 at 18:21, Jaiganesh Balasubramanian wrote:
|     Well, we already do that when RInside sets itself up:
| 
|        #ifdef CSTACK_DEFNS
|        R_CStackLimit = (uintptr_t)-1;      // Don't do any stack checking,
|     see R Exts, '8.1.5 Threading issues'
|        #endif
| 
| 
| Yes, i noticed this, and added the MACRO definition in my code as
| #define CSTACK_DEFNS 1
| 
| 
| What I would do now is to create a sample program with two threads, then
| 
|     initialize R in one and the query from the other. If that already tickles
|     the
|     issue then you are out of luck, and R has to stay in one thread.  You may
|     be
|     able to use inter-thread communication to 'request' data from the
|     R-containing thread by carefully brokering the requests.
| 
| 
| This is exactly what my program is doing right now, and it does not work.
| 
| If it works, it would be awesome, because, we could then perform multi-core
| programming with Rcpp/RInside in a seamless fashion with any native threading
| mechanism like Pthreads.

Yes, but we have right now is just wishing that by simply layering multi-
threading around the single-threaded R engine we would get multi-threaded
programming. 

So let's just ask: why should it work?  Do you have a good answer?

I don't.  So in the interim your best bet may the examples/mpi directory --
by wrapping mpi around the single-threaded apps we do get parallel
processing.  The open question is whether we can get something more
lightweight.  I don't have a lead on that. *Maybe* borrowing some ideas from
Simon's multicore package may help but I don't think that will be trivial.
You would have to lead this and try a few things.

No sorry, no entirely free lunch yet.

-- 
  Regards, Dirk