[Rsiena-help] New Memory Issue

[Ruth M. Ripley]

Dear Nate,

I have just done a quick comparison on a simple fit in 32 bit windows, and 
my version 141 fit is smaller than 129, which actually stored the data. So 
it is nothing obvious... Things have been added to the model, but nothing 
large that I can think of, and we no longer store the data. But it could 
be something specific to the options you have used: eg if you use maximum 
likelihood fitting... If you would like to send me direct one of your 
sienaFit's I will try to work out what is going on. If you also send me a 
data object, effect object and model, I can run some comparisons with 
each version.

Regards,

Ruth
--
Ruth M. Ripley,                         Email:ruth at stats.ox.ac.uk
Dept. of Statistics,                    http://www.stats.ox.ac.uk/~ruth/
University of Oxford,                   Tel:   01865 282851
1 South Parks Road, Oxford OX1 3TG, UK  Fax:   01865 272595


On Tue, 8 Mar 2011, Nate Doogan wrote:

> Hi group.
>
> I have a very large time series. I am analyzing it by dividing it into small
> sections. I estimate parameters for many small sections of time and combine
> them using siena08(). I have done this using 54 individual estimations in
> the past with little trouble on a machine with 4 gigabytes of RAM.
>
> Recently, I have tried this again with no success. R spits out a memory
> error, even if I use a machine with 8 gigabytes of memory and the 64bit
> version of R and the OS. The problem is that all of the sienaFit objects
> need to be loaded and there has to be enough memory left for siena08() to do
> its job. I have tried breaking the estimations down into theoretically
> interesting subsets, but that still results in a set of sienaFit objects of
> size 40--still too big. Also, if I monitor the computer's RAM usage, it
> clearly sky rockets when I load() the saved sienaFit objects. In fact, the
> sienaFit objects appear to consume more memory than they have in the past.
>
> I am currently using RSiena version 141. First, I want to confirm that
> RSiena is actually storing more information in the sienaFit objects in
> recent updates of the software. Second, if this is true, and short of
> finding a machine with more RAM, does anyone have a suggestion for working
> around the resulting problem? Thank you.
>
> Nathan Doogan
> Doctoral Student
> The Ohio State University
>