[Rcpp-devel] Logic error causes R memory to be corrupt

[Anton Bossenbroek]

Hi Dirk,

Thanks for clarifying. I love the QuantLib package, I used while I was working in quant finance before moving to data science.

The idea behind my code is that I want to create a C++ backend to the GeneralTree package I recently committed on CRAN (https://cran.r-project.org/web/packages/GeneralTree/index.html <https://cran.r-project.org/web/packages/GeneralTree/index.html> ). I have two goals:
1. provide a fast and unified interface to xml, JSON, YAML parsers. 
2. provide a easy way to handle different models based on the same data using decision rules in the tree

The reason I want to write the backend in C++ is because R6 objects are slow. I initially wanted to keep objects by reference to prevent data.frames or entire models to be copied, which could be more important in case 2 than 1. What is your thought on this?

I could rewrite all the shared_ptr to XPtr to make sure the entire memory is managed by R instead of a mixture model. But if you think copies are better I might choose that approach.

Best,

Anton

> On 6 Oct 2016, at 12:31, Dirk Eddelbuettel <edd at debian.org> wrote:
> 
> 
> Hi Anton,
> 
> Thanks for posting here.
> 
> On 6 October 2016 at 10:52, Anton Bossenbroek wrote:
> | I want to add a large number of objects in C++ that are managed by `shared_ptr` in a `vector`. However, when I push the limits of the amount that I want to allocate the data in R becomes inconsistent.
> 
> A bit of background.
> 
> Rcpp exists because of the excellent idea (by Dominick, around 2004 to 2005)
> that using C++ templates for SEXP++ would make interfacing QuantLib easier in
> the RQuantLib package I had started a few years earlier.
> 
> QuantLib is a marvel. It started in the early 2000s and was the first project
> I saw which has bet "big" on Boost testing, and, very importantly,
> shared_ptr.  QuantLib also makes extensive use of SWIG to govern interfaces
> to (a large number of) other languages.  And here is the key: None of those
> can use shared_ptr objects.  Neither does RQuantLib. Even though they are
> pervasive in QuantLib.
> 
> Now, we are here to interface R.  All that Rcpp does, in essence, is making
> it trivially easy to get objects in and out of R as proxy objects:
> lightweight, no copy.  And the memory _is still managed by R_. So that you
> never need to use a SEXP directly in C++.  That is a feature.  [ Of course,
> sometimes you need the SEXP as a quasi-variant type, but that is not the
> common use. So let's just call it 'no SEXP in user code'. ]
> 
> So I am not claiming that what you want to do here is impossible -- but I am
> not aware of another project having done this: hold R objects in a smart_ptr.
> 
> Also, you haven't really explained what you want to do, what the life cyle of your
> objects is etc pp.
> 
> At the simplest level, Rcpp use is synchronous:  you call from R, pass some
> content, do some calculations, report something back -- and everything
> intermediary is destroyed.  You can persist things, of course, when you
> manage the memory.  But you need to be very sure the things are then away
> from R -- copies are probably best. XPtr objects are used in that context.
> 
> Anyway, end of short speech.  I would recommend you try to break you project
> into smaller, more fundamental tasks.  Maybe we can help with those, one by one.
> 
> Dirk
> 
> -- 
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20161006/85537891/attachment-0001.html>