<div dir="ltr">Hi, William,<div><br></div><div>Can you give us a small piece of code to reproduce the error? That will ease our discussion.</div><div><br></div><div>Best,</div><div><br></div><div>KK</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Nov 6, 2018 at 10:46 PM William Nolan <<a href="mailto:will@landale.net">will@landale.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi all,<div><br></div><div>Longtime user and lurker here.</div><div><br></div><div>I got "index error" thrown by Rcpp when trying to subset a matrix with width * height == 644764 * 3776 greater than MAXINT:</div><div><br></div><div><div><font face="monospace, monospace">Allocating 647764 x 3776 matrix...</font></div><div><font face="monospace, monospace">Catchpoint 1 (exception thrown), 0x00007ffff4baa920 in __cxa_throw () from /usr/lib64/libstdc++.so.6</font></div></div><div><div><font face="monospace, monospace">#1 0x0000000000473a05 in Rcpp::stop<>(char const*) (fmt=0x7fffe7bc3545 "index error")</font></div><div><font face="monospace, monospace"> at /home/nolanw/R/x86_64-redhat-linux-gnu-library/3.5/Rcpp/include/Rcpp/exceptions/cpp11/exceptions.h:52</font></div><div><font face="monospace, monospace">52 throw Rcpp::exception( tfm::format(fmt, std::forward<Args>(args)... ).c_str() );</font></div><div><font face="monospace, monospace">(gdb) down</font></div><div><font face="monospace, monospace">#0 0x00007ffff4baa920 in __cxa_throw () from /usr/lib64/libstdc++.so.6</font></div><div><font face="monospace, monospace">(gdb) up</font></div><div><font face="monospace, monospace">#1 0x0000000000473a05 in Rcpp::stop<>(char const*) (fmt=0x7fffe7bc3545 "index error")</font></div><div><font face="monospace, monospace"> at /home/nolanw/R/x86_64-redhat-linux-gnu-library/3.5/Rcpp/include/Rcpp/exceptions/cpp11/exceptions.h:52</font></div><div><font face="monospace, monospace">52 throw Rcpp::exception( tfm::format(fmt, std::forward<Args>(args)... ).c_str() );</font></div><div><font face="monospace, monospace">(gdb) up</font></div><div><font face="monospace, monospace">#2 0x00007fffe7bb75a6 in Rcpp::SubsetProxy<14, Rcpp::PreserveStorage, 13, true, Rcpp::Vector<13, Rcpp::PreserveStorage> >::check_indices (this=0x7fffffffc450, x=0x13f9920,</font></div><div><font face="monospace, monospace"> n=3751, size=-1849010432) at /home/nolanw/R/x86_64-redhat-linux-gnu-library/3.5/Rcpp/include/Rcpp/vector/:138</font></div><div><font face="monospace, monospace">138 stop("index error");</font></div><div><font face="monospace, monospace">(gdb) up</font></div><div><font face="monospace, monospace">#3 0x00007fffe7bb6a3c in Rcpp::SubsetProxy<14, Rcpp::PreserveStorage, 13, true, Rcpp::Vector<13, Rcpp::PreserveStorage> >::get_indices (this=0x7fffffffc450, t=...)</font></div><div><font face="monospace, monospace"> at /home/nolanw/R/x86_64-redhat-linux-gnu-library/3.5/Rcpp/include/Rcpp/vector/Subsetter.h:149</font></div><div><font face="monospace, monospace">149 check_indices(ptr, rhs_n, lhs_n);</font></div><div><font face="monospace, monospace">(gdb) l</font></div><div><font face="monospace, monospace">144 #endif</font></div><div><font face="monospace, monospace">145</font></div><div><font face="monospace, monospace">146 void get_indices( traits::identity< traits::int2type<INTSXP> > t ) {</font></div><div><font face="monospace, monospace">147 indices.reserve(rhs_n);</font></div><div><font face="monospace, monospace">148 int* ptr = INTEGER(rhs);</font></div><div><font face="monospace, monospace">149 check_indices(ptr, rhs_n, lhs_n);</font></div><div><font face="monospace, monospace">150 for (int i=0; i < rhs_n; ++i) {</font></div><div><font face="monospace, monospace">151 indices.push_back( rhs[i] );</font></div><div><font face="monospace, monospace">152 }</font></div><div><font face="monospace, monospace">153 indices_n = rhs_n;</font></div></div><div><font face="monospace, monospace"><br></font></div><div><font face="arial, helvetica, sans-serif">As we can see from the stack trace and below, lhs.size() is negative when cast to int:</font></div><div><font face="monospace, monospace"><br></font></div><div><div style="font-family:monospace,monospace">(gdb) p (int)(lhs.size())</div><div style="font-family:monospace,monospace">$12 = -1849010432</div><div style="font-family:monospace,monospace"><br></div><div><font face="arial, helvetica, sans-serif">This is all coming from the assignment (via operator []) of the subsetting of one matrix to another matrix's subset:</font></div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">(static_cast<NumericVector&>(mat))[lhsI] = (static_cast<NumericVector&>(signals))[rhsI];<br></div><div style="font-family:monospace,monospace"><br></div><div><font face="arial, helvetica, sans-serif">(lhsI and rhsI are IntegerVector's)</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Now, setting aside whether I *should* be doing that -- what I *do* see in Subsetter.h (including what I understand to be the most recent version, 1.0.0 from github) is the use of int for indices all over the place in this file, including in the member variable:</font></div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">std::vector<int> indices;<br></div><div style="font-family:monospace,monospace"><br></div><div><font face="arial, helvetica, sans-serif">Is there any reason why the indices that Subsetter uses internally shouldn't be size_t or an equivalently capable type like R_xlen_t?</font></div><div><font face="arial, helvetica, sans-serif">For example, Subsetter's check_indices function takes int's as arguments, while Vector's size method returns R_xlen_t.</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">I'll change my code to manually copy elements via operator() using the row/column arguments for now. Seems like Subsetter is maybe not quite ready for prime time.</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div style="font-family:monospace,monospace"><br></div></div></div></div></div></div></div></div>
_______________________________________________<br>
Rcpp-devel mailing list<br>
<a href="mailto:Rcpp-devel@lists.r-forge.r-project.org" target="_blank">Rcpp-devel@lists.r-forge.r-project.org</a><br>
<a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel" rel="noreferrer" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel</a></blockquote></div>