[Rcpp-devel] "Error during wrapup"

John Mous john.mous0 at gmail.com
Thu May 29 21:46:40 CEST 2014


Hi everyone,

Thank for you sticking with the problem even though it is so hard to
recreate. I believe I have solved it, but the solution is based on a
workaround, and I believe there is still a problem in Rcpp. First, the
problem is not limited to wrapping map objects. I changed the code to
return a NumericVector and had the same issue, although perhaps it's
funneled through the same wrap code down the line -- again I'm not really
familiar with Rcpp internals. Second, I noticed the problem occurred only
when I used R's random number generator and RNGScope; code using C++'s RNG
worked fine. Third, with that information in hand, I ran across this
thread:
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2013-May/005838.html,
which suggests that the order in which the destructor of the RNGScope
object and the returned object fire can muck things up. Based on this idea,
I changed things so the returned NumericVector is created before the
RNGScope object, and the already existing object is populated before being
returned. This appears to have fixed it.

This solution is good enough for my immediate problem. Unfortunately for
helping you fix this in Rcpp, all of this was done in the original code. I
still can't create a small reproducible example. The problem is really hard
to recreate -- the code in question is used for numerical simulation and
the function runs hundreds of thousands of times before failing. But
perhaps this new information can help inspire someone. I will be away for a
few weeks, but am happy to test patches on the original code (if someone is
inspired) if it's helpful once I return.




On Thu, May 29, 2014 at 1:10 AM, Kevin Ushey <kevinushey at gmail.com> wrote:

> What OS are you running on (Mac, Linux, Windows)? I really can't tease
> this out (with Mac + clang) thus far.
>
> On Wed, May 28, 2014 at 9:56 PM, Romain Francois
> <romain at r-enthusiasts.com> wrote:
> >
> > Le 29 mai 2014 à 00:35, John Mous <john.mous0 at gmail.com> a écrit :
> >
> > No luck with gctorture on yet, but I haven't run it for very long. It
> slowed
> > things down to a complete crawl, but maybe in the long run it can
> recreate
> > the problem faster than running the original code (about 12 hours, the
> > previous estimate of a day was too high), so this may still be
> worthwhile.
> >
> > On a whim I did try changing the two instances of Shield to Armor this
> > morning (I was able to locate the code in wrap.h by digging around
> before I
> > saw your e-mail) and ran the original full code without gctorture on, and
> > the problem still persists.
> >
> >
> > Would not be enough, you'd also have to do:
> >
> >> x = ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
> >
> >
> > so that the result of setAttrib gets protected even if it is a new SEXP,
> > which AFAIU can happen with setAttrib.
> >
> > But I'm not sure this is the problem.
> >
> > Romain
> >
> >
> > On Wed, May 28, 2014 at 3:44 PM, Romain Francois <
> romain at r-enthusiasts.com>
> > wrote:
> >>
> >> (now with some links):
> >>
> >> Le 28 mai 2014 à 16:31, John Mous <john.mous0 at gmail.com> a écrit :
> >>
> >> The object really is just built as part of the return statement. i.e.
> the
> >> lines from my prior e-mail exist as-is in the full code,
> >>
> >>
> >> Sure. What happens is that Rcpp::export generates something that calls
> >> wrap( std::map<std::string,int> ).
> >>
> >> there's just more that actually builds the variables X1-X4 beforehand.
> >>
> >>
> >> It should not be relevant.
> >>
> >> So I'm not sure where to debug from the client side. I'm a C/C++
> >> developer, but have no experience with Rcpp internals or the general
> >> interface between R and C. I can insert some debug statements on the
> Rcpp
> >> side if you can guide me to where wrap is defined.
> >>
> >>
> >> well wrap is defined here:
> >>
> >>
> https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap_end.h#L28
> >>
> >> It then performs a series of dispatch, that eventually lead to something
> >> that handles std::map<std::string,int>:
> >>
> >>
> https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap.h#L255
> >>
> >>
> >> template <typename InputIterator, typename T>
> >> inline SEXP range_wrap_dispatch___impl__cast( InputIterator first,
> >> InputIterator last, ::Rcpp::traits::false_type ){
> >>      size_t size = std::distance( first, last ) ;
> >>      const int RTYPE = ::Rcpp::traits::r_sexptype_traits<typename
> >> T::second_type>::rtype ;
> >>      Shield<SEXP> x( Rf_allocVector( RTYPE, size ) );
> >>      Shield<SEXP> names( Rf_allocVector( STRSXP, size ) ) ;
> >>      typedef typename ::Rcpp::traits::storage_type<RTYPE>::type CTYPE ;
> >>      CTYPE* start = r_vector_start<RTYPE>(x) ;
> >>      size_t i =0;
> >>      std::string buf ;
> >>      for( ; i<size; i++, ++first){
> >>              start[i] = (*first).second ;
> >>              buf = (*first).first ;
> >>              SET_STRING_ELT( names, i, Rf_mkChar(buf.c_str()) ) ;
> >>      }
> >>      ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
> >>      return wrap_extra_steps<T>( x ) ;
> >> }
> >>
> >>
> >> what I suspect to be the problem is this line:
> >>
> >>  ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
> >>
> >>
> >> What Romain says makes sense to me, despite my lack of expertise in this
> >> area.. the really intermittent nature of the problem and the fact that I
> >> can't recreate it in a small / fast running example suggests that
> perhaps
> >> this manifests when R happens to garbage collect at an unfortunate time
> (if
> >> I understood correctly). Thanks again.
> >>
> >>
> >> Something like that. as hadley hinted, please try this under gc torture,
> >> or preferably through a debugger.
> >>
> >> On Wed, May 28, 2014 at 10:12 AM, Dirk Eddelbuettel <edd at debian.org>
> >> wrote:
> >>>
> >>>
> >>> On 28 May 2014 at 10:02, John Mous wrote:
> >>> | Hmm, unfortunately the GitHub version failed also.
> >>>
> >>> Darn.
> >>>
> >>> | The attributes on the failed
> >>> | object are a little different though, here's what they look like:
> >>> |
> >>> | Browse[1]> str(results)
> >>> |  atomic [1:4] 1 1 2270 0
> >>> |  - attr(*, "")= symbol sim
> >>> |  - attr(*, "value")= promise to  NULL
> >>>
> >>> I am not sure what we can do without a reproducible example. :-/
> >>> The code just got a review / refreshment over the last few months.
> >>>
> >>> You best bet may the slow and tedious insertion of debug statements to
> >>> see
> >>> when / if the object changes.
> >>>
> >>> Dirk
> >>>
> >>> --
> >>> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> >>
> >>
> >> _______________________________________________
> >> Rcpp-devel mailing list
> >> Rcpp-devel at lists.r-forge.r-project.org
> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
> >>
> >>
> >
> >
> >
> > _______________________________________________
> > Rcpp-devel mailing list
> > Rcpp-devel at lists.r-forge.r-project.org
> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20140529/401d8815/attachment.html>


More information about the Rcpp-devel mailing list