[Rcpp-devel] Segfault, is it because of iterators/pointers?

[Alessandro Mammana]

Ok I was able to find the code causing the bug. So it looks like the
pointers you get from an Rcpp::Vector using .begin() become invalid
after that the Rcpp::Vector goes out of scope (and this makes sense),
what I do not understand is that this Rcpp::Vector was allocated in R
and should still be "living" during the execution of the Rcpp call
(that's why I wasn't expecting the pointer to be invalid).

This is the exact code (the one above is probably fine):
@@@@@@@@@@@@@@ in CPP @@@@@@@@@@@@@@i

struct GapMat {
    int* ptr;
    int* colset;
    int nrow;
    int ncol;


    inline int* colptr(int col){
        return ptr + colset[col];
    }

    GapMat(){}

    GapMat(int* _ptr, int* _colset, int _nrow, int _ncol):
        ptr(_ptr), colset(_colset), nrow(_nrow), ncol(_ncol){}
};


GapMat getGapMat(Rcpp::List gapmat){
    IntegerVector vec = gapmat["vec"];
    IntegerVector pos = gapmat["colset"];
    int nrow = gapmat["nrow"];

    return GapMat(vec.begin(), pos.begin(), nrow, pos.length());
}

// [[Rcpp::export]]
IntegerVector colSumsGapMat(Rcpp::List gapmat){

    GapMat mat = getGapMat(gapmat);
    IntegerVector res(mat.ncol);

    for (int i = 0; i < mat.ncol; ++i){
        for (int j = 0; j < mat.nrow; ++j){
            res[i] += mat.colptr(i)[j];
        }
    }

    return res;
}

@@@@@@@@@@@@@@ in R (with gdb debugger as suggested by Dirk) @@@@@@@@@@@@@@i
library(Rcpp)
sourceCpp("scratchpad.cpp")

vec <- rnbinom(3e7, mu=0.1, size=1); storage.mode(vec) <- "integer"
nr <- 80

colset <- sample(3e7-nr, 1e7)
foo <- vec[colset] #this is only to trigger some obscure garbage
collection mechanisms...

for (i in 1:10){
    colset <- sample(3e7-nr, 1e7)
    gapmat <- list(vec=vec, nrow=nr, colset=colset-1)
    cs <- colSumsGapMat(gapmat)
    print(sum(cs))
}

[1] 80000000
[1] 80000000
[1] 80016890
[1] 80008144
[1] 80016022
[1] 80021609

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff18a5455 in GapMat::colptr (this=0x7fffffffc120, col=0) at
scratchpad.cpp:295
295            return ptr + colset[col];

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Why did it happen? What should I do to make sure that my pointers
remain valid? My goal is to convert safely some vectors or matrices
that "exist" in R to some pointers, how can I do that?

Thanks a lot for your help

Ale

On Tue, Feb 11, 2014 at 3:44 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>
> In essence: "Yes"
>
> On 11 February 2014 at 15:18, Alessandro Mammana wrote:
> | Hi all,
> | I got another segfault using Rcpp. It is very difficult to understand
> | where it happens and to reduce it to a minimal example, so for now I
> | am not posting very precise code here, but I have a suspicion, maybe
> | you could help me saying if my suspect is right.
>
> Use a debugger:
>
>     R -d gdb
>
> and then proceed as normal.
>
> Make sure your compiler flags include -g as well.
>
> Dirk
>
>
> | I am doing something similar:
> |
> | in a .cpp file:
> | @@@@@@@@@@@@@@@@@@@
> | struct GapMat {
> |     int* ptr;
> |     int* colset;
> |     int nrow;
> |     int ncol;
> |
> |
> |     inline int* colptr(int col){
> |         return ptr + colset[col];
> |     }
> |
> |     GapMat(){}
> |
> |     GapMat(int* _ptr, int* _colset, int _nrow, int _ncol):
> |         ptr(_ptr), colset(_colset), nrow(_nrow), ncol(_ncol){}
> | };
> |
> |
> | // [[Rcpp::export]]
> | IntegerVector colSumsGapMat(Rcpp::IntegerVector vec,
> | Rcpp::IntegerVector pos, int nrow){
> |    GapMat mat(vec.begin(), pos.begin(), nrow, pos.length());
> |    IntegerVector res(pos.length());
> |
> |     for (int i = 0; i < pos.length(); ++i){
> |         for (int j = 0; j < nrow; ++j){
> |             res[i] += mat.colptr(i)[j];
> |         }
> |     }
> |
> |     return res;
> | }
> | @@@@@@@@@@@@@@@@@@@@@
> |
> | from R:
> |
> | vec <- a very big integer vector
> | nrow <- 80
> | pos <- a very big subset of positions, such that max(pos) + nrow < length(vec)
> | colsums <- colSumsGapMat(vec, pos, nrow)
> |
> |
> | from time to time I get a segfault.
> | Note: this is not exactly the code that produces the segfault (because
> | that one is very complicated), so it might be that this code is
> | totally fine.
> |
> | My suspicion:
> |
> | I am using the pointer "vec.begin()", but then I am allocating new
> | memory in the R area of memory with "IntegerVector res(pos.length())"
> | and R decides to move the original values of "vec" to some other
> | place, making the pointer invalid.
> |
> | Is that possible????
> |
> | Sorry for being very vague and thx in advance!!!
> | Ale
> |
> | --
> | Alessandro Mammana, PhD Student
> | Max Planck Institute for Molecular Genetics
> | Ihnestraße 63-73
> | D-14195 Berlin, Germany
> | _______________________________________________
> | Rcpp-devel mailing list
> | Rcpp-devel at lists.r-forge.r-project.org
> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
> --
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com



-- 
Alessandro Mammana, PhD Student
Max Planck Institute for Molecular Genetics
Ihnestraße 63-73
D-14195 Berlin, Germany