[Rcpp-devel] Segfault, is it because of iterators/pointers?

[Dirk Eddelbuettel]

On 12 February 2014 at 11:47, Alessandro Mammana wrote:
| Ok I was able to find the code causing the bug. So it looks like the

Thanks for the added detail.

| pointers you get from an Rcpp::Vector using .begin() become invalid
| after that the Rcpp::Vector goes out of scope (and this makes sense),
| what I do not understand is that this Rcpp::Vector was allocated in R
| and should still be "living" during the execution of the Rcpp call
| (that's why I wasn't expecting the pointer to be invalid).
| 
| This is the exact code (the one above is probably fine):
| @@@@@@@@@@@@@@ in CPP @@@@@@@@@@@@@@i
| 
| struct GapMat {
|     int* ptr;
|     int* colset;
|     int nrow;
|     int ncol;
| 
| 
|     inline int* colptr(int col){
|         return ptr + colset[col];
|     }
| 
|     GapMat(){}
| 
|     GapMat(int* _ptr, int* _colset, int _nrow, int _ncol):
|         ptr(_ptr), colset(_colset), nrow(_nrow), ncol(_ncol){}
| };
| 
| 
| GapMat getGapMat(Rcpp::List gapmat){
|     IntegerVector vec = gapmat["vec"];
|     IntegerVector pos = gapmat["colset"];
|     int nrow = gapmat["nrow"];
| 
|     return GapMat(vec.begin(), pos.begin(), nrow, pos.length());
| }
| 
| // [[Rcpp::export]]
| IntegerVector colSumsGapMat(Rcpp::List gapmat){
| 
|     GapMat mat = getGapMat(gapmat);
|     IntegerVector res(mat.ncol);
| 
|     for (int i = 0; i < mat.ncol; ++i){
|         for (int j = 0; j < mat.nrow; ++j){
|             res[i] += mat.colptr(i)[j];
|         }
|     }
| 
|     return res;
| }
| 
| @@@@@@@@@@@@@@ in R (with gdb debugger as suggested by Dirk) @@@@@@@@@@@@@@i
| library(Rcpp)
| sourceCpp("scratchpad.cpp")
| 
| vec <- rnbinom(3e7, mu=0.1, size=1); storage.mode(vec) <- "integer"
| nr <- 80
| 
| colset <- sample(3e7-nr, 1e7)
| foo <- vec[colset] #this is only to trigger some obscure garbage
| collection mechanisms...
| 
| for (i in 1:10){
|     colset <- sample(3e7-nr, 1e7)
|     gapmat <- list(vec=vec, nrow=nr, colset=colset-1)
|     cs <- colSumsGapMat(gapmat)
|     print(sum(cs))
| }
| 
| [1] 80000000
| [1] 80000000
| [1] 80016890
| [1] 80008144
| [1] 80016022
| [1] 80021609
| 
| Program received signal SIGSEGV, Segmentation fault.
| 0x00007ffff18a5455 in GapMat::colptr (this=0x7fffffffc120, col=0) at
| scratchpad.cpp:295
| 295            return ptr + colset[col];
| 
| @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
| 
| Why did it happen? What should I do to make sure that my pointers
| remain valid? My goal is to convert safely some vectors or matrices
| that "exist" in R to some pointers, how can I do that?

Not sure. It looks fine at first instance. But then it's early in the morning
and I had very little coffee yet... 

Maybe the fact that you tickle the gc() via vec[colset] has something to do
with it, maybe it has not.  Maybe I would try the decomposition of the List
object inside the colSumsGapMat() function to keep it simpler.  Or if you
_really_ want an external object to iterate over, memcpy it out.

With really large object, you may be stressing parts of the code that have
not been stressed the same way.  If it breaks, you do get to keep both pieces.

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com