[Rcpp-devel] Loading a package using Rcpp Modules results in memory corruption

[Dominick Samperi]

On Sat, Jan 8, 2011 at 9:48 AM, Douglas Bates <bates at stat.wisc.edu> wrote:

> On Sat, Jan 8, 2011 at 7:49 AM, Douglas Bates <bates at stat.wisc.edu> wrote:
> > On Sat, Jan 8, 2011 at 12:45 AM, Dominick Samperi <djsamperi at gmail.com>
> wrote:
> >> I checked things under Linux and Windows (using GCC and VC++ DLL's) and
> the
> >> same problem occurs at the same place, which is a good sign when it
> comes to
> >> memory issues. Basically, the Rcpp::Reference(std::string) constructor
> that
> >> is
> >> part of S4_field, or S4_CppOverloadedMethods constructors fails,
> depending
> >> on
> >> which comes first (whether there are fields or not). This only happens
> when
> >> gctorture() is turned on, so R must be clobbering an unprotected SEXP
> >> somewhere...
> >
> > Thanks, Dominick. I too have been working on tracking this down and
> > got to that point.  If you follow a little further you will find that
> > it is the evaluation of the R function getCurrentErrorMessage in the
> > Rcpp::Evaluator::run method where things start to go bad, as far as I
> > can see.  I hope to be able to isolate this today as I need a working
> > version of lme4a by tomorrow.
>
> My current theory is that Rcpp_cache is being blown away by the
> garbage collector.  Because we want this to be persistent I think the
> simplest thing to do is to assign it to a name like .Rcpp_cache in the
> namespace during the .onLoad function.  I'll try that.
>

I tested this theory by cutting out the Evaluator code and it appears that
the cache is not harmed by gctorture(). I suspect that the problem may
be earlier in the chain of construction, either a SEXP that was not
preserved,
or an aliasing problem. It appears that init_Rcpp_cache() is called twice
at start-up, which does not seem right, but suppressing the second attempt
does not fix the problem.


>
> >> On Fri, Jan 7, 2011 at 9:47 AM, Romain Francois <
> romain at r-enthusiasts.com>
> >> wrote:
> >>>
> >>> Hmm. I commited 2845 and 2846 today.
> >>>
> >>> Anyway, if you see it also with 0.9.0 this means more detective work.
> >>>
> >>> Le 07/01/11 15:05, Douglas Bates a écrit :
> >>>>
> >>>> On Fri, Jan 7, 2011 at 5:34 AM, Romain Francois
> >>>> <romain at r-enthusiasts.com>  wrote:
> >>>>>
> >>>>> Le 05/01/11 18:52, Douglas Bates a écrit :
> >>>>>>
> >>>>>> I don't know whether this is through error on my part or because of
> an
> >>>>>> "infelicity" in the Rcpp module code but the lme4a package, which
> now
> >>>>>> uses Rcpp modules extensively, ends up with some difficult-to-trace
> >>>>>> memory corruption issues.  Yesterday i finally bit the bullet and
> ran
> >>>>>> a test with gctorture(TRUE) and valgrind enabled.  It takes a very
> >>>>>> long time and results in a segfault when trying to load the package.
> >>>>>> I enclose the transcript.  I should say that this is using
> Rcpp_0.9.0
> >>>>>> from CRAN, not the SVN version of Rcpp.
> >>>>>>
> >>>>>> I just got these results this morning (it was running overnight) and
> >>>>>> haven't looked at the code in Module.cpp and cache.cpp yet.  If it
> >>>>>> seems likely that the code is beyond me I can try to work out a
> >>>>>> simpler example that triggers the problem.
> >>>>>
> >>>>> Hi Doug,
> >>>>>
> >>>>> Sorry for the delay, I'm not fully operational yet.
> >>>>>
> >>>>> All this might be related to some code I put in during holidays and
> did
> >>>>> not
> >>>>> have a chance to fully test.
> >>>>>
> >>>>> Can you try with rev 2845 and let me know if you still see the
> problem.
> >>>>>
> >>>>> Romain
> >>>>
> >>>> Regrettably the problem persists with rev 2845 (which was from
> >>>> 2011-01-04, is that the one you meant?) but it is also present when
> >>>> using Rcpp_0.9.0
> >>>
> >>>
> >>> --
> >>> Romain Francois
> >>> Professional R Enthusiast
> >>> +33(0) 6 28 91 30 30
> >>> http://romainfrancois.blog.free.fr
> >>> |- http://bit.ly/fT2rZM : highlight 0.2-5
> >>> |- http://bit.ly/gpCSpH : Evolution of Rcpp code size
> >>> `- http://bit.ly/hovakS : RcppGSL initial release
> >>>
> >>>
> >>> _______________________________________________
> >>> Rcpp-devel mailing list
> >>> Rcpp-devel at lists.r-forge.r-project.org
> >>>
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20110108/f9ecb5f7/attachment-0001.htm>