[Rcpp-devel] Caching temporary results in long chains.

[Whit Armstrong]

unless you are preallocating space to store your answer, then every
reassignment could potentially cause the whole vector to be
reallocated.

if you know in advance how many runs you will do, and the thinning,
then preallocate the vector and assign into it.

Alternatively, implement your mcmc model in this:
https://github.com/armstrtw/CppBugs

and hopefully your run time will improve by 100x.

-Whit


On Thu, Dec 9, 2010 at 2:56 PM, Andrew Redd <amredd at gmail.com> wrote:
> I'm having a problem with running long MCMC chains.  When I run the chain
> for 1000 runs it takes about  30 seconds
> ---
>> n<-1e3
>> time1<- system.time({
> + run1 <- d3$run(n, 1,1, 1,1, 50)
> + })
>> time1
>    user  system elapsed
>  30.120   0.950  31.105
> ---
> When I run for 100 000 runs I should expect  something in the neighborhood
> of an hour.  What I'm getting is about 8 hours.  And I have confirmed that
>  the longer  the run the longer each iteration takes.   My best guess is
> that this has something to do with efficient memory management.  Monitoring
> my system I know that things are staying in physical memory and not going to
> swap.  It might be that it is able to stay cached on chip, rather than
> memory.  Does anyone have good intuition for this?
> Either way it looks like the way to proceed is to implement a caching
> mechanism where I can compute about 1000 iterations then store the results
> and keep going.  The problem is I have no intuition of how to go about this
> in Rcpp or anything else.
> I R I would just perform something like
>  results[1:1000 +(i-1)*1000,] <- results_temp
> I don't know how to do that in Rcpp or if it would be more efficient.  Any
> direction and discussion would be helpful.  Sorry for being vague.
> Thanks,
> Andrew
>
>
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
>