[Rcpp-devel] Garbage collection PreserveObject.

[Romain Francois]

On 01/24/2010 06:13 PM, Guillaume Yziquel wrote:
> Hi again.
>
> Romain Francois a écrit :
>> On 01/24/2010 01:06 AM, Dirk Eddelbuettel wrote:
>>>
>>> Houston, we have lift-off. We have come full circle. Our use of
>>> preserve /
>>> release followed Simon's answers to your questions on r-devel.
>>>
>>> On 24 January 2010 at 00:44, Guillaume Yziquel wrote:
>>> | I recently noticed that the garbage-collection interfacing toping has
>>> | been quite active at the beginning of the month on this list, so I'm
>>> | asking here. My question concerns essentially my OCaml binding, but
>>> I'll
>>> | try to make it as language agnostic as possible to somehow stay within
>>> | the scope of this list.
>>>
>>> You are a nautical mile and a half off-topic for 'rcpp-devel', but we
>>> don't
>>> mind. We only charge after the third question. Just kidding.
>
> At the first question, or just after the third question? That could make
> a difference...
>
>> Yes. In this list you only have access to a very limited subset of
>> people following R-devel, so I'd still advise you to use R-devel ...
>
> But at least, you, Dirk and Whit all have an interest in garbage
> collection. So it's a more restricted audience, but not necessarily the
> worst. But I'll also ask a few questions of R-devel at .
>
>> unless maybe you want to rebase your binding on Rcpp ;-) but from I
>> read on R-devel you seem to be willing to go deeper in the api than we
>> are (we don't expose internal aspects of SEXPREC, etc ...)
>
> For the API of my binding, see the following URL (only available when my
> laptop is up):
>
> http://yziquel.homelinux.org/topos/api/ocaml-r/
>
> I only expose the internal structure of SEXPs in the following module:
>
> http://yziquel.homelinux.org/topos/api/ocaml-r/R.Pretty.html
>
> and they are read-only. I never intended to give the user to power to
> fool around too much.

That's good news.

> Nevertheless, it is true that there are some issues as to allowing
> lightweight threading between R code and OCaml code. I would need some
> features to make callbacks from within the R interpreter code itself to
> implement lightweight threading.
>
> So it depends what you mean by "going deeper".

We only use SEXP. There is no place where we expose what a SEXPREC is, 
because we just don't know and don't need to. Anything picking into 
SEXPREC is "going deeper" in my book

>>> | Two questions:
>>> |
>>> | -1- From what I gathered, it is fine to issue two PreserveObject
>>> | statements on the same SEXP, and then two ReleaseObject afterwards. It
>>> | seemed to me that reference counting was not needed, from my
>>> | understanding of the past discussion. Am I wrong?
>>>
>>> I think you are right. We simply the object along, but AFAIK R just
>>> flips a
>>> bit:
>>
>> That's not true. this does not modify the object itself, but attaches
>> it to the precious list. The object then gains protection simply
>> because it is part of the precious list. more details on the direct
>> response to Guillaume, or the mentionned R-devel thread.
>
> OK. So it's the precious list that protects R values.

yep. but there is nothing special about it. Just any object will do. if 
a SEXP (let's call it the protegee) is contained by another SEXP (let's 
call it the mentor), then the mentor automatically protects the protegee.

>>> void preserve(){ if( m_sexp != R_NilValue ) R_PreserveObject(m_sexp) ; }
>>> void release() { if( m_sexp != R_NilValue ) R_ReleaseObject(m_sexp) ; }
>>>
>>> | -2- Once you have, in your C/C++ code, obtained a pointer to a SEXP
>>> from
>>> | R, do you need to PROTECT it, then do a PreserveObject, then UNPROTECT
>>> | it? Or is PreserveObject simply fine? (I guess so, but I'm damn
>>> unsure).
>>
>> It is plain simple. I think.
>>
>> You need to make sure that the object is not subject of garbage
>> collection. If you only use it locally (e.g. in a function) :
>> PROTECT/UNPROTECT is fine are will be more efficient (see Simon's
>> emails).
>
> Yes, but this is not my situtation.

So you can rule out PROTECT, this is not what you should use.

>> If your object needs to persist, then you just cannot use
>> PROTECT/UNPROTECT, you have to do something else. Preserve/Release is
>> just a shortcut for attaching the object to another object that you
>> know for sure it is protected (the precious list), so your object gets
>> protected as well.
>>
>> There are other ways, you can create your own pairlist, or your own
>> environment, and manage this manually.
>>
>> Simon said that the most efficient way would be to allocate a big
>> enough generic vector and manage it yourself. Is it worth the trouble
>> ? maybe not.
>
> Yes. This is probably the approach I'll be implementing in the long run.
> A big enough generic vector to hold all R values, which would act itself
> as the above mentionned precious list.
>
> But there are quite a number of issues about this.
>
> Suppose you run R code, which allocates a value. You have not allocated
> it. So, by default, it is not in the big vector. Or perhaps you meant a
> big VecSxp vector, i.e. an array of SEXPs?
>
> But this could be worth it. The funny thing is that is the approach
> taken for the minor heap (first generation) of the generational garbage
> collector in OCaml. Stop & Copy garbage collection. So I'll have to try
> to chain the OCaml minor heap with the proposed generic vector.
>
> If you want to go with the big generic VecSxp vector for C++, you might
> just as well reuse and adapt to R Hans & Boehm's garbage collector for
> Mark & Sweep. I'll perhaps go this way when I have time for that.

That's a low priority thing. Currently we are fine with the 
Preserve/Release.

>> Hope this helps. Romain
>
> Thanks a lot. This is really helpful. I'll stop polluting this list.

It is no so much about polluting, it just is that we are only clients of 
the R API, where as in R-devel you have access to people who developped 
it or have the right to change it. All you get here is only based on 
experience of using the api and sometimes guesses.

Plus we also are on R-devel mailing list, so there's a chance you get 
our answers there too.

>>> More the latter, but it is complicated. We had replaced all PROTECT /
>>> UNPROTECT calls with preserve / release but we needed to go back for
>>> some
>>> reason. Romain has the gory details.
>>>
>>> Dirk
>
> All the best,
>


-- 
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://tr.im/KfKn : Rcpp 0.7.2
|- http://tr.im/JOlc : External pointers with Rcpp
`- http://tr.im/JFqa : R Journal, Volume 1/2, December 2009