[Rcpp-devel] Use size_t rather than int for vector sizes and indexing?

[chuck at chuckcode.com]

Thanks Jay for the lightning fast insightful response! Didn't realize that
this limitation was intrinsic to R itself. I'll have to go look at
bigmemory and some of the other projects.

Thanks,
-Chuck

On Wed, 13 Apr 2011 14:11:25 -0400, Jay Emerson <jayemerson at gmail.com>
wrote:
> Chuck,
> 
> Internally, R is using 4-byte integers for indexing, and the length of
> a vector is thus constrained to 2-billion-ish elements.  Two ways
> around this include packages ff and
> bigmemory, for example, or relying on database-like queries.  However,
> the resulting objects can't be used with standard R functions (with
> some special exceptions).
> 
> Jay
> 
> 
> On Wed, Apr 13, 2011 at 2:05 PM,  <chuck at chuckcode.com> wrote:
>>
>> Hi All,
>>
>> Thanks to Dirk Eddelbuettel and the other contributors for such a
>> wonderful package. Rcpp really transforms the way that I've been able
to
>> incorporate c++ code into R. Makes it possible to speed up the critical
>> computations while keeping all the great flexibility and features of R.
>>
>> I've run into a problem lately with particularly large vectors in Rcpp.
I
>> seem to be overflowing when my vectors get larger than 2^31 elements on
a
>> 64 bit system. It looks from the code of both the classic (included
>> below)
>> and the newer versions as though this is due to using ints rather than
>> something like size_t (http://en.wikipedia.org/wiki/Size_t) as the type
>> for
>> size and indexing into the vector. It looks like RcppResultsSet's add()
>> functions may also be using ints.
>>
>> Curious to know if there is a particular reason for using ints rather
>> than
>> something like size_t and if the project managers would be open to
>> changing
>> to accomodate larger vectors.
>>
>> Thanks,
>> -Chuck Sugnet
>>
>> template <typename T>
>> class RcppVector {
>> public:
>>        typedef T* iterator ;
>>        typedef const T* const_iterator ;
>>
>>    RcppVector(SEXP vec);
>>    RcppVector(int len);
>>    int size() const;
>>    T& operator()(int i) const;
>>    T *cVector() const;
>>    std::vector<T> stlVector() const;
>>
>>    inline const_iterator begin() const { return v ; }
>>    inline const_iterator end() const { return v + len ; }
>>
>>    inline iterator begin(){ return v ; }
>>    inline iterator end(){ return v + len ; }
>>
>> private:
>>    int len;
>>    T *v;
>> };
>> _______________________________________________
>> Rcpp-devel mailing list
>> Rcpp-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>>