[Rcpp-devel] Caching temporary results in long chains.

[Andrew Redd]

I'm having a problem with running long MCMC chains.  When I run the chain
for 1000 runs it takes about  30 seconds
---
> n<-1e3
> time1<- system.time({
+ run1 <- d3$run(n, 1,1, 1,1, 50)
+ })
> time1
   user  system elapsed
 30.120   0.950  31.105
---

When I run for 100 000 runs I should expect  something in the neighborhood
of an hour.  What I'm getting is about 8 hours.  And I have confirmed that
 the longer  the run the longer each iteration takes.   My best guess is
that this has something to do with efficient memory management.  Monitoring
my system I know that things are staying in physical memory and not going to
swap.  It might be that it is able to stay cached on chip, rather than
memory.  Does anyone have good intuition for this?

Either way it looks like the way to proceed is to implement a caching
mechanism where I can compute about 1000 iterations then store the results
and keep going.  The problem is I have no intuition of how to go about this
in Rcpp or anything else.

I R I would just perform something like
 results[1:1000 +(i-1)*1000,] <- results_temp

I don't know how to do that in Rcpp or if it would be more efficient.  Any
direction and discussion would be helpful.  Sorry for being vague.

Thanks,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20101209/3de6094f/attachment.htm>