[Rcpp-devel] Logic error causes R memory to be corrupt
anton.bossenbroek at me.com
Thu Oct 6 19:12:43 CEST 2016
Thanks for clarifying. I love the QuantLib package, I used while I was working in quant finance before moving to data science.
The idea behind my code is that I want to create a C++ backend to the GeneralTree package I recently committed on CRAN (https://cran.r-project.org/web/packages/GeneralTree/index.html <https://cran.r-project.org/web/packages/GeneralTree/index.html> ). I have two goals:
1. provide a fast and unified interface to xml, JSON, YAML parsers.
2. provide a easy way to handle different models based on the same data using decision rules in the tree
The reason I want to write the backend in C++ is because R6 objects are slow. I initially wanted to keep objects by reference to prevent data.frames or entire models to be copied, which could be more important in case 2 than 1. What is your thought on this?
I could rewrite all the shared_ptr to XPtr to make sure the entire memory is managed by R instead of a mixture model. But if you think copies are better I might choose that approach.
> On 6 Oct 2016, at 12:31, Dirk Eddelbuettel <edd at debian.org> wrote:
> Hi Anton,
> Thanks for posting here.
> On 6 October 2016 at 10:52, Anton Bossenbroek wrote:
> | I want to add a large number of objects in C++ that are managed by `shared_ptr` in a `vector`. However, when I push the limits of the amount that I want to allocate the data in R becomes inconsistent.
> A bit of background.
> Rcpp exists because of the excellent idea (by Dominick, around 2004 to 2005)
> that using C++ templates for SEXP++ would make interfacing QuantLib easier in
> the RQuantLib package I had started a few years earlier.
> QuantLib is a marvel. It started in the early 2000s and was the first project
> I saw which has bet "big" on Boost testing, and, very importantly,
> shared_ptr. QuantLib also makes extensive use of SWIG to govern interfaces
> to (a large number of) other languages. And here is the key: None of those
> can use shared_ptr objects. Neither does RQuantLib. Even though they are
> pervasive in QuantLib.
> Now, we are here to interface R. All that Rcpp does, in essence, is making
> it trivially easy to get objects in and out of R as proxy objects:
> lightweight, no copy. And the memory _is still managed by R_. So that you
> never need to use a SEXP directly in C++. That is a feature. [ Of course,
> sometimes you need the SEXP as a quasi-variant type, but that is not the
> common use. So let's just call it 'no SEXP in user code'. ]
> So I am not claiming that what you want to do here is impossible -- but I am
> not aware of another project having done this: hold R objects in a smart_ptr.
> Also, you haven't really explained what you want to do, what the life cyle of your
> objects is etc pp.
> At the simplest level, Rcpp use is synchronous: you call from R, pass some
> content, do some calculations, report something back -- and everything
> intermediary is destroyed. You can persist things, of course, when you
> manage the memory. But you need to be very sure the things are then away
> from R -- copies are probably best. XPtr objects are used in that context.
> Anyway, end of short speech. I would recommend you try to break you project
> into smaller, more fundamental tasks. Maybe we can help with those, one by one.
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rcpp-devel