[Rcpp-devel] Memory error when using Dimension

[Dirk Eddelbuettel]

On 5 June 2012 at 13:19, Douglas Bates wrote:
| Some time ago I mentioned memory errors that seemed to be related the
| the Rcpp::Dimension class but I couldn't pin them down.  I have a more
| detailed, although not necessarily reproducible, example now.
| 
| In the merPredD::condVar method defined in the lme4/src/predModule.cpp
| source file there is the line
| 
| 	    Rcpp::NumericVector ansi(Rcpp::Dimension(ncti, ncti, nli));
| 
| When that method is eventually invoked from a version of lme4 compiled
| with -ltcmalloc for R-2.15.0 (Ubuntu Version: 2.15.0-1precise0)
| tcmalloc reports an attempt to free an invalid pointer.  The gdb
| traceback is
| 
| > print(dotplot(ranef(fm1, postVar = TRUE), strip = FALSE)[[1]])
| src/tcmalloc.cc:390] Attempt to free invalid pointer: 0x53d75d0
| 
| Program received signal SIGABRT, Aborted.
| 0x00007ffff726f445 in raise () from /lib/x86_64-linux-gnu/libc.so.6
| (gdb) where
| #0  0x00007ffff726f445 in raise () from /lib/x86_64-linux-gnu/libc.so.6
| #1  0x00007ffff7272bab in abort () from /lib/x86_64-linux-gnu/libc.so.6
| #2  0x00007ffff0e507fb in ?? () from /usr/lib/libtcmalloc.so.0
| #3  0x00007ffff0e50b04 in TCMalloc_CrashReporter::PrintfAndDie(char
| const*, ...) () from /usr/lib/libtcmalloc.so.0
| #4  0x00007ffff0e45840 in ?? () from /usr/lib/libtcmalloc.so.0
| #5  0x00007ffff0e64191 in tc_delete () from /usr/lib/libtcmalloc.so.0
| #6  0x00007ffff07fcde6 in deallocate (__p=<optimized out>,
| this=<optimized out>) at /usr/include/c++/4.6/ext/new_allocator.h:98
| #7  _M_deallocate (__p=<optimized out>, this=<optimized out>,
| __n=<optimized out>) at /usr/include/c++/4.6/bits/stl_vector.h:156
| #8  ~_Vector_base (this=<optimized out>, __in_chrg=<optimized out>) at
| /usr/include/c++/4.6/bits/stl_vector.h:142
| #9  ~vector (this=0x7fffffff9620, __in_chrg=<optimized out>) at
| /usr/include/c++/4.6/bits/stl_vector.h:351
| #10 ~Dimension (this=0x7fffffff9620, __in_chrg=<optimized out>) at
| /home/bates/R/x86_64-unknown-linux-gnu-library/2.15/Rcpp/include/Rcpp/Dimension.h:29
| #11 lme4::merPredD::condVar (this=0x5720000, rho=...) at predModule.cpp:105
| #12 0x00007ffff07dc608 in merPredDcondVar (ptr=0x474c8c0,
| rho=<optimized out>) at external.cpp:617
| ...
| (gdb) up 10
| #10 ~Dimension (this=0x7fffffff9620, __in_chrg=<optimized out>) at
| /home/bates/R/x86_64-unknown-linux-gnu-library/2.15/Rcpp/include/Rcpp/Dimension.h:29
| 29	    class Dimension {
| (gdb) p dims
| $1 = {<std::_Vector_base<int, std::allocator<int> >> = {_M_impl =
| {<std::allocator<int>> = {<__gnu_cxx::new_allocator<int>> = {<No data
| fields>}, <No data fields>}, _M_start = 0x53d75d0,
|       _M_finish = 0x53d75dc, _M_end_of_storage = 0x53d75dc}}, <No data fields>}
| 
| so the address in question is the _M_start of the dims
| std::vector<int> private member of the Rcpp::Dimension instance.  The
| length of 12 bytes (difference between _M_finish and _M_start) is what
| would be expected for a std::vector<int> of length 3.
| 
| Those memory locations have the values we would expect
| (gdb) up
| #11 lme4::merPredD::condVar (this=0x5720000, rho=...) at predModule.cpp:105
| 105		    Rcpp::NumericVector ansi(Rcpp::Dimension(ncti, ncti, nli));
| (gdb) p ncti
| $2 = 1
| (gdb) p nli
| $3 = 6
| (gdb) down
| #10 ~Dimension (this=0x7fffffff9620, __in_chrg=<optimized out>) at
| /home/bates/R/x86_64-unknown-linux-gnu-library/2.15/Rcpp/include/Rcpp/Dimension.h:29
| 29	    class Dimension {
| (gdb) p {int}0x53d75d0
| $4 = 1
| (gdb) p {int}0x53d75d4
| $5 = 1
| (gdb) p {int}0x53d75d8
| $6 = 6
| 
| To me it looks as if the pointer to the contents is being passed as
| the pointer to a malloc'd chunk, which has a header describing the
| size of the chunk.  This may be the common approach - I don't know
| enough about the innards of malloc and free to tell.
| 
| Anyway, I can't see anything suspicious in the code in
| Rcpp/src/Dimension.cpp.  For the time being I am going to fall back on
| the standard malloc/free
| 
| Well, actually I did another thing which is to replace the line 105 by
| 
| 	    Rcpp::NumericVector ansi(ncti * ncti * nli);
| 	    ansi.attr("dim") = Rcpp::IntegerVector::create(ncti, ncti, nli);
| 
| and now things are fine.  Still don't know what the problem was though.

I use the "let us decompose a more complex expression into its constituents"
approach when I get compiler issues.  It is a tad weirder to see this at the
run-time side.

But if you have something that works now ... 

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com