[Rcpp-devel] DataFrame::create() in endless loop
Douglas Bates
bates at stat.wisc.edu
Tue Feb 28 17:44:19 CET 2012
On Tue, Feb 28, 2012 at 10:16 AM, Steffen Neumann <sneumann at ipb-halle.de> wrote:
> Hi,
>
> I am currently hunting a problem where a call to
> Rcpp::DataFrame::create() goes into a seemingly
> endless loop. seqNum,acquisitionNum and msLevel
> are all std::vector<int> of length N (~3.9 million),
> and if I change the code to only include seqNum,acquisitionNum
> this runs fine. msLevel contains values \in {0,1}.
>
> I am currently on R-devel (2012-01-16 r58124) x86_64-unknown-linux-gnu
> using Rcpp_0.9.7. I've added a gdb stack trace at the end,
> although most useful information seems to have been optimized away :-(
>
> Any ideas what to check next ? Try a new Rcpp ?
>
> Yours,
> Steffen
>
> std::vector<int> seqNum(N);
> std::vector<int> acquisitionNum(N);
> std::vector<int> msLevel(N);
>
> // works:
> allScanHeaderInfo = Rcpp::DataFrame::create( Rcpp::_["seqNum"] = seqNum,
> Rcpp::_["acquisitionNum"] = acquisitionNum)
> // endless (?) loop
> allScanHeaderInfo = Rcpp::DataFrame::create( Rcpp::_["seqNum"] = seqNum,
> Rcpp::_["acquisitionNum"] = acquisitionNum,
> Rcpp::_["msLevel"] = msLevel)
>
>
> -------------------------------------------
>
> Program received signal SIGINT, Interrupt.
> 0x00007ffff79564d4 in RunGenCollect (size_needed=<value optimized out>) at memory.c:1515
> 1515 PROCESS_NODES();
> (gdb) where
> #0 0x00007ffff79564d4 in RunGenCollect (size_needed=<value optimized out>) at memory.c:1515
> #1 R_gc_internal (size_needed=<value optimized out>) at memory.c:2590
> #2 0x00007ffff7958e2a in Rf_cons (car=0x2db18b80, cdr=0x609a58) at memory.c:2085
> #3 0x00007ffff790adfb in Rf_promiseArgs (el=0x26244b8, rho=0x63c570) at eval.c:1965
> #4 0x00007ffff791961b in Rf_eval (e=0x26244f0, rho=<value optimized out>) at eval.c:514
> #5 0x00007ffff791a695 in do_eval (call=0x19bb860, op=<value optimized out>, args=0x2623810, rho=0x26240b0) at eval.c:2086
> #6 0x00007ffff790ed28 in bcEval (body=<value optimized out>, rho=0x26240b0, useCache=<value optimized out>) at eval.c:4445
> #7 0x00007ffff7919372 in Rf_eval (e=0x19bc720, rho=<value optimized out>) at eval.c:401
> #8 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x2623f60, rho=<value optimized out>,
> suppliedenv=0x2624078) at eval.c:840
> #9 0x00007ffff7919648 in Rf_eval (e=0x2624598, rho=<value optimized out>) at eval.c:515
> #10 0x00007ffff7919ac7 in forcePromise (e=0x26246b0) at eval.c:320
> #11 0x00007ffff7919a00 in Rf_eval (e=0x66aff8, rho=<value optimized out>) at eval.c:425
> #12 0x00007ffff791bae4 in do_return (call=0x19be910, op=<value optimized out>, args=0xf1, rho=0x2624790) at eval.c:1439
> #13 0x00007ffff791976b in Rf_eval (e=0x19be910, rho=<value optimized out>) at eval.c:471
> #14 0x00007ffff7919ac7 in forcePromise (e=0x26249f8) at eval.c:320
> #15 0x00007ffff7919a00 in Rf_eval (e=0x66aff8, rho=<value optimized out>) at eval.c:425
> #16 0x00007ffff791bb8e in do_begin (call=<value optimized out>, op=<value optimized out>, args=0x19bf568, rho=0x2624b48) at eval.c:1422
> #17 0x00007ffff791976b in Rf_eval (e=0x19bf220, rho=<value optimized out>) at eval.c:471
> #18 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x2624a30, rho=<value optimized out>,
> suppliedenv=0x2624b10) at eval.c:840
> #19 0x00007ffff7919648 in Rf_eval (e=0x19be8a0, rho=<value optimized out>) at eval.c:515
> #20 0x00007ffff7920410 in do_set (call=0x19be7c0, op=0x615648, args=0x19be7f8, rho=0x2624790) at eval.c:1726
> #21 0x00007ffff791976b in Rf_eval (e=0x19be7c0, rho=<value optimized out>) at eval.c:471
> #22 0x00007ffff791bb8e in do_begin (call=<value optimized out>, op=<value optimized out>, args=0x19be788, rho=0x2624790) at eval.c:1422
> #23 0x00007ffff791976b in Rf_eval (e=0x19bfd98, rho=<value optimized out>) at eval.c:471
> #24 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x26246e8, rho=<value optimized out>,
> suppliedenv=0x2624758) at eval.c:840
> #25 0x00007ffff7919648 in Rf_eval (e=0x2624608, rho=<value optimized out>) at eval.c:515
> #26 0x00007ffff409317b in Rcpp::Evaluator::run (expr=<value optimized out>, env=<value optimized out>) at Evaluator.cpp:36
> #27 0x00007ffff43b897e in create<Rcpp::traits::named_object<std::vector<int, std::allocator<int> > >, Rcpp::traits::named_object<std::vector<int, std::allocator<int> > >, Rcpp::traits::named_object<std::vector<int, std::allocator<int> > > > (this=<value optimized out>)
> at /vol/R/R-devel/lib64/R/library/Rcpp/include/Rcpp/DataFrame_generated.h:50
> #28 RcppRamp::getAllScanHeaderInfo (this=<value optimized out>) at RcppRamp.cpp:256
> #29 0x00007ffff43bf9ef in Rcpp::CppMethod0<RcppRamp, Rcpp::DataFrame>::operator() (this=<value optimized out>, object=<value optimized out>)
> at /vol/R/R-devel/lib64/R/library/Rcpp/include/Rcpp/module/Module_generated_CppMethod.h:31
> #30 0x00007ffff43c231d in Rcpp::class_<RcppRamp>::invoke_notvoid (this=<value optimized out>, method_xp=<value optimized out>, object=0x2514140,
> args=0x7fffffffa2b0, nargs=<value optimized out>) at /vol/R/R-devel/lib64/R/library/Rcpp/include/Rcpp/Module.h:493
> #31 0x00007ffff4d5cd47 in CppMethod__invoke_notvoid (args=<value optimized out>) at Module.cpp:250
> #32 0x00007ffff78e640e in do_External (call=0x19c8c20, op=<value optimized out>, args=0x2383e78, env=<value optimized out>) at dotcode.c:794
> #33 0x00007ffff791994b in Rf_eval (e=<value optimized out>, rho=<value optimized out>) at eval.c:497
> #34 0x00007ffff791bb8e in do_begin (call=<value optimized out>, op=<value optimized out>, args=0x19c8ef8, rho=0x2383e40) at eval.c:1422
> #35 0x00007ffff791976b in Rf_eval (e=0x19c8b40, rho=<value optimized out>) at eval.c:471
> #36 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x609a58, rho=<value optimized out>,
> suppliedenv=0x2383e08) at eval.c:840
> #37 0x00007ffff7919648 in Rf_eval (e=0x2657eb0, rho=<value optimized out>) at eval.c:515
> #38 0x00007ffff791bae4 in do_return (call=0x2657f20, op=<value optimized out>, args=0xf1, rho=0x264e548) at eval.c:1439
> #39 0x00007ffff791976b in Rf_eval (e=0x2657f20, rho=<value optimized out>) at eval.c:471
> #40 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x264e4d8, rho=<value optimized out>,
> ---Type <return> to continue, or q <return> to quit---
> suppliedenv=0x264e510) at eval.c:840
> #41 0x00007ffff7919648 in Rf_eval (e=0x2657cf0, rho=<value optimized out>) at eval.c:515
> #42 0x00007ffff791bb8e in do_begin (call=<value optimized out>, op=<value optimized out>, args=0x2657d28, rho=0x264e238) at eval.c:1422
> #43 0x00007ffff791976b in Rf_eval (e=0x26572c8, rho=<value optimized out>) at eval.c:471
> #44 0x00007ffff791c3af in R_execClosure (call=<value optimized out>, op=<value optimized out>, arglist=<value optimized out>, rho=<value optimized out>,
> newrho=<value optimized out>) at eval.c:939
> #45 0x00007ffff791c65d in R_execMethod (op=0x26576b8, rho=0x2624468) at eval.c:1028
> #46 0x00007ffff5b20dcc in R_dispatchGeneric (fname=<value optimized out>, ev=0x2624468, fdef=<value optimized out>) at methods_list_dispatch.c:1029
> #47 0x00007ffff7964849 in do_standardGeneric (call=0x0, op=<value optimized out>, args=<value optimized out>, env=0x2624468) at objects.c:1130
> #48 0x00007ffff791989e in Rf_eval (e=0x263e2e8, rho=<value optimized out>) at eval.c:500
> #49 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x2624388, rho=<value optimized out>,
> suppliedenv=0x2624430) at eval.c:840
> #50 0x00007ffff7919648 in Rf_eval (e=0x20ca688, rho=<value optimized out>) at eval.c:515
> #51 0x00007ffff7920410 in do_set (call=0x20ca730, op=0x615648, args=0x20ca6f8, rho=0x63c570) at eval.c:1726
> #52 0x00007ffff791976b in Rf_eval (e=0x20ca730, rho=<value optimized out>) at eval.c:471
> #53 0x00007ffff7919ac7 in forcePromise (e=0x20ca998) at eval.c:320
> #54 0x00007ffff7919c70 in FORCE_PROMISE (value=0x1dbcc070, symbol=<value optimized out>, rho=0xf1, keepmiss=498909408) at eval.c:3360
> #55 0x00007ffff791545e in bcEval (body=0x7ffff7ddc628, rho=0x20caa78, useCache=<value optimized out>) at eval.c:4207
> #56 0x00007ffff7919372 in Rf_eval (e=0x2517470, rho=<value optimized out>) at eval.c:401
> #57 0x00007ffff791ca1b in Rf_applyClosure (call=<value optimized out>, op=<value optimized out>, arglist=0x20ca9d0, rho=<value optimized out>,
> suppliedenv=0x20caa40) at eval.c:840
> #58 0x00007ffff7919648 in Rf_eval (e=0x20ca848, rho=<value optimized out>) at eval.c:515
> #59 0x00007ffff7950d38 in Rf_ReplIteration (rho=0x63c570, savestack=0, browselevel=<value optimized out>, state=0x7fffffffd330) at main.c:256
> #60 0x00007ffff7950fc9 in R_ReplConsole (rho=0x63c570, savestack=0, browselevel=0) at main.c:305
> #61 0x00007ffff7951500 in run_Rmainloop () at main.c:986
> #62 0x000000000040092b in main (ac=<value optimized out>, av=<value optimized out>) at Rmain.c:32
> #63 0x00007ffff6e81c4d in __libc_start_main (main=<value optimized out>, argc=<value optimized out>, ubp_av=<value optimized out>,
> init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fffffffe468) at libc-start.c:226
> #64 0x0000000000400819 in _start ()
It's a memory problem from R. It is trying to do garbage collection
(RunGenCollector) and getting hung up, probably because of exhausting
one of the stacks in R. Your std::vector<int> objects need to be
copied to R objects. Is it possible to define them as IntegerVector's
instead, in which case the Create method will (I think) just move some
pointers around.
More information about the Rcpp-devel
mailing list