[Rcpp-devel] "Error during wrapup"
William Dunlap
wdunlap at tibco.com
Thu May 29 22:53:48 CEST 2014
You can get more information about intermittent problems like this by
running R under valgrind and using gctorture(TRUE). It will run very
slowly, but valgrind will flag the first time you use memory that is
not currently allocated by new or malloc. Without valgrind you only
see a problem if the memory misuse is so flagrant that you access
memory outside the process's memory. Without gctorture(TRUE) memory
you shouldn't be using may or or may not be freed when you step out of
bounds.
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Thu, May 29, 2014 at 12:46 PM, John Mous <john.mous0 at gmail.com> wrote:
> Hi everyone,
>
> Thank for you sticking with the problem even though it is so hard to
> recreate. I believe I have solved it, but the solution is based on a
> workaround, and I believe there is still a problem in Rcpp. First, the
> problem is not limited to wrapping map objects. I changed the code to return
> a NumericVector and had the same issue, although perhaps it's funneled
> through the same wrap code down the line -- again I'm not really familiar
> with Rcpp internals. Second, I noticed the problem occurred only when I used
> R's random number generator and RNGScope; code using C++'s RNG worked fine.
> Third, with that information in hand, I ran across this thread:
> http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2013-May/005838.html,
> which suggests that the order in which the destructor of the RNGScope object
> and the returned object fire can muck things up. Based on this idea, I
> changed things so the returned NumericVector is created before the RNGScope
> object, and the already existing object is populated before being returned.
> This appears to have fixed it.
>
> This solution is good enough for my immediate problem. Unfortunately for
> helping you fix this in Rcpp, all of this was done in the original code. I
> still can't create a small reproducible example. The problem is really hard
> to recreate -- the code in question is used for numerical simulation and the
> function runs hundreds of thousands of times before failing. But perhaps
> this new information can help inspire someone. I will be away for a few
> weeks, but am happy to test patches on the original code (if someone is
> inspired) if it's helpful once I return.
>
>
>
>
> On Thu, May 29, 2014 at 1:10 AM, Kevin Ushey <kevinushey at gmail.com> wrote:
>>
>> What OS are you running on (Mac, Linux, Windows)? I really can't tease
>> this out (with Mac + clang) thus far.
>>
>> On Wed, May 28, 2014 at 9:56 PM, Romain Francois
>> <romain at r-enthusiasts.com> wrote:
>> >
>> > Le 29 mai 2014 à 00:35, John Mous <john.mous0 at gmail.com> a écrit :
>> >
>> > No luck with gctorture on yet, but I haven't run it for very long. It
>> > slowed
>> > things down to a complete crawl, but maybe in the long run it can
>> > recreate
>> > the problem faster than running the original code (about 12 hours, the
>> > previous estimate of a day was too high), so this may still be
>> > worthwhile.
>> >
>> > On a whim I did try changing the two instances of Shield to Armor this
>> > morning (I was able to locate the code in wrap.h by digging around
>> > before I
>> > saw your e-mail) and ran the original full code without gctorture on,
>> > and
>> > the problem still persists.
>> >
>> >
>> > Would not be enough, you'd also have to do:
>> >
>> >> x = ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
>> >
>> >
>> > so that the result of setAttrib gets protected even if it is a new SEXP,
>> > which AFAIU can happen with setAttrib.
>> >
>> > But I'm not sure this is the problem.
>> >
>> > Romain
>> >
>> >
>> > On Wed, May 28, 2014 at 3:44 PM, Romain Francois
>> > <romain at r-enthusiasts.com>
>> > wrote:
>> >>
>> >> (now with some links):
>> >>
>> >> Le 28 mai 2014 à 16:31, John Mous <john.mous0 at gmail.com> a écrit :
>> >>
>> >> The object really is just built as part of the return statement. i.e.
>> >> the
>> >> lines from my prior e-mail exist as-is in the full code,
>> >>
>> >>
>> >> Sure. What happens is that Rcpp::export generates something that calls
>> >> wrap( std::map<std::string,int> ).
>> >>
>> >> there's just more that actually builds the variables X1-X4 beforehand.
>> >>
>> >>
>> >> It should not be relevant.
>> >>
>> >> So I'm not sure where to debug from the client side. I'm a C/C++
>> >> developer, but have no experience with Rcpp internals or the general
>> >> interface between R and C. I can insert some debug statements on the
>> >> Rcpp
>> >> side if you can guide me to where wrap is defined.
>> >>
>> >>
>> >> well wrap is defined here:
>> >>
>> >>
>> >> https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap_end.h#L28
>> >>
>> >> It then performs a series of dispatch, that eventually lead to
>> >> something
>> >> that handles std::map<std::string,int>:
>> >>
>> >>
>> >> https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap.h#L255
>> >>
>> >>
>> >> template <typename InputIterator, typename T>
>> >> inline SEXP range_wrap_dispatch___impl__cast( InputIterator first,
>> >> InputIterator last, ::Rcpp::traits::false_type ){
>> >> size_t size = std::distance( first, last ) ;
>> >> const int RTYPE = ::Rcpp::traits::r_sexptype_traits<typename
>> >> T::second_type>::rtype ;
>> >> Shield<SEXP> x( Rf_allocVector( RTYPE, size ) );
>> >> Shield<SEXP> names( Rf_allocVector( STRSXP, size ) ) ;
>> >> typedef typename ::Rcpp::traits::storage_type<RTYPE>::type CTYPE ;
>> >> CTYPE* start = r_vector_start<RTYPE>(x) ;
>> >> size_t i =0;
>> >> std::string buf ;
>> >> for( ; i<size; i++, ++first){
>> >> start[i] = (*first).second ;
>> >> buf = (*first).first ;
>> >> SET_STRING_ELT( names, i, Rf_mkChar(buf.c_str()) ) ;
>> >> }
>> >> ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
>> >> return wrap_extra_steps<T>( x ) ;
>> >> }
>> >>
>> >>
>> >> what I suspect to be the problem is this line:
>> >>
>> >> ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
>> >>
>> >>
>> >> What Romain says makes sense to me, despite my lack of expertise in
>> >> this
>> >> area.. the really intermittent nature of the problem and the fact that
>> >> I
>> >> can't recreate it in a small / fast running example suggests that
>> >> perhaps
>> >> this manifests when R happens to garbage collect at an unfortunate time
>> >> (if
>> >> I understood correctly). Thanks again.
>> >>
>> >>
>> >> Something like that. as hadley hinted, please try this under gc
>> >> torture,
>> >> or preferably through a debugger.
>> >>
>> >> On Wed, May 28, 2014 at 10:12 AM, Dirk Eddelbuettel <edd at debian.org>
>> >> wrote:
>> >>>
>> >>>
>> >>> On 28 May 2014 at 10:02, John Mous wrote:
>> >>> | Hmm, unfortunately the GitHub version failed also.
>> >>>
>> >>> Darn.
>> >>>
>> >>> | The attributes on the failed
>> >>> | object are a little different though, here's what they look like:
>> >>> |
>> >>> | Browse[1]> str(results)
>> >>> | atomic [1:4] 1 1 2270 0
>> >>> | - attr(*, "")= symbol sim
>> >>> | - attr(*, "value")= promise to NULL
>> >>>
>> >>> I am not sure what we can do without a reproducible example. :-/
>> >>> The code just got a review / refreshment over the last few months.
>> >>>
>> >>> You best bet may the slow and tedious insertion of debug statements to
>> >>> see
>> >>> when / if the object changes.
>> >>>
>> >>> Dirk
>> >>>
>> >>> --
>> >>> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
>> >>
>> >>
>> >> _______________________________________________
>> >> Rcpp-devel mailing list
>> >> Rcpp-devel at lists.r-forge.r-project.org
>> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>> >>
>> >>
>> >
>> >
>> >
>> > _______________________________________________
>> > Rcpp-devel mailing list
>> > Rcpp-devel at lists.r-forge.r-project.org
>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
>
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
More information about the Rcpp-devel
mailing list