[Rcpp-devel] Lots of lists

[Douglas Bates]

On Tue, Mar 13, 2012 at 1:57 PM, Romain Francois
<romain at r-enthusiasts.com> wrote:
> Le 13/03/12 19:18, Matthew Krachey a écrit :
>
>> Hi,
>>
>> I am working on a rather complicated MCMC simulation with a lot of return
>> objects, both vectors and matrices. We are on the verge of getting things to
>> go (to date, much faster than Numpy) but we are returning ~21 list objects
>> (again, both vectors and matrices) and we are getting error when the 21st
>> object is added. We would like all of this output to return to R for
>> post-processing. Is there a recommended way to deal with this type of
>> output?

Another approach is to use reference classes in R and link the
reference class object to an object of a C++ class.  Rcpp modules act
more-or-less like this.

I ended up writing a more specific interface based on external
pointers and checking for a null pointer, regenerating if necessary,
for the lme4Eigen package.  It takes advantage of the fact that you
can "map" R vectors or matrices to read-only Eigen objects of the
corresponding types.  The values in the R reference class object are
actually updated by the C++ code (which is legitimate for reference
class objects).

The reason for checking the external pointers is to be able to
serialize/unserialize (i.e. save/load) the R object.  Enough
information is retained in the R saved object to be able to regenerate
the C++ object after the object is loaded.

I realize this is a somewhat complicated approach but it has the
advantage of not requiring large objects to be copied to be able to
pass them back and forth between R and the compiled code.

I can give more detail if this sounds interesting.  The lme4Eigen
package is only on R-forge at present, under the lme4 project.

> I'm assuming you are using List::create to create your list with 20+
> elements. In that case, you hit the limit of our code bloat.
>
> In the absence of a proper way to deal with variable number of arguments (as
> offered in the C++11 standard with variadic templates), we had to
> semi-manually generated these.
> https://r-forge.r-project.org/scm/viewvc.php/pkg/Rcpp/inst/include/Rcpp/generated/Vector__create.h?view=markup&revision=3451&root=rcpp
>
> It would be possible to keep it up and pump it to say 30, 50, whatever, but
> you could also consider organising the information differently.
>
> What are these lists ? Do you need them to be next to each other or can you
> do it differently ?
>
> Romain
>
> --
> Romain Francois
> Professional R Enthusiast
> +33(0) 6 28 91 30 30
> R Graph Gallery: http://addictedtor.free.fr/graphiques
> blog:            http://romainfrancois.blog.free.fr
> |- http://bit.ly/xbKv0R : Crawling facebook with R
> |- http://bit.ly/v3WB8S : ... And now for solution 17, still using Rcpp
> `- http://bit.ly/uaQDGr : int64: 64 bit integer vectors for R
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel