[Robast-commits] r569 - branches/robast-0.9/pkg/RobExtremes/inst/AddMaterial/interpolation

[noreply at r-forge.r-project.org]

Author: ruckdeschel
Date: 2013-01-27 03:47:07 +0100 (Sun, 27 Jan 2013)
New Revision: 569

Added:
   branches/robast-0.9/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
Log:
RobExtremes: committed a write up to interpolation strategies

Added: branches/robast-0.9/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
===================================================================
--- branches/robast-0.9/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt	                        (rev 0)
+++ branches/robast-0.9/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt	2013-01-27 02:47:07 UTC (rev 569)
@@ -0,0 +1,153 @@
+WRITE-UP:
+Interpolation Grids in R-Packages
+Peter Ruckdeschel, Jan 27 2013
+
+1. Starting Point:
+
+Computation of optimally robust ICs in our R pkgs works well, but may be slow,
+in particular in case of lack of invariance---like in scale-shape models, when
+we cannot move the IC from one parameter value theta to the next by invariance.
+Then we have to recompute the IC for each theta anew.
+As the opt-rob ICs are given through Lagrange multipliers LMs) which are continuous
+in theta as shown by Matthias Kohl, we may compute opt-rob ICs for a grid of
+theta values offline and then, for a new theta value use interpolation.
+The same strategy applies to speed up evaluation of scale functional Sn (Croux)
+for Shape-Scale-models.
+
+2. Problem when passing from R-2.15 to R-3.0 
+
+From R-2.15 to R-3.0 Brian Ripley (R-Core) has changed some non-exported 
+interfaces to C-code. This affects approxfun and splinefun which we
+are using for interpolation because saving the results of approxfun and 
+splinefun comes up much faster (experiments by Matthias) than using approx 
+resp. spline. Now approxfun, splinefun from R-3.0 on return functions
+whose body contains R code which is not yet interpretable with R-2.15
+and the code from R-2.15 no longer runs from R-3.0 on.
+We solved this (after long discussions with R-core ...) 
+saving two interpolating functions, one with suffix ".O" for < R-2.16, 
+one with suffix ".N" for >2.16, and at run time determine the current
+R version and take the suitable one.
+
+3. Size of packages
+
+With CRAN getting larger (>4000 pkgs) people from CRAN have set up some
+limitations on size of pkgs. Taking up our interpolation idea naively,
+however, quite easily ends up with huge pkgs. So care has to be taken.
+
+4. Implementation 
+
+R allows to provide pre-computed R objects in a pkg in a particular
+file sysdata.rda (to be generated by save()) to be put into the R 
+folder of the pkg. Works easily for simple objects like numerics, 
+characters; gets more complicated with functions. In addition to
+the code (in the body of the function) and the argument list,
+this also comprises an environment, which is used to bind objects
+which are not generated in the body and are not passed through
+the argument list (lexical scoping). This environment easily gets
+large, in particular if whole S4 class hierarchies have to be
+provided ... As a consequence, when loading such a "small" sysdata.rda
+file R may have to load several packages under the hood to reconstruct
+the environment. More to this issue later. 
+At any rate, we will store our interpolation grids and functions in
+such a sysdata.rda file.
+
+5. Datastructure
+
+As we deal with several robust optimality criteria (MSE, RMX, MBR) 
+and several models (GPD, GEVD, Gamma, Weibull), our grids are stored
+in nested lists. Debatable, but done so for the moment.
+Let's use the following notation for describing the list structure:
+Each layer in the hierarchie gives one ">" and an item is inserted
+below the item next left to it with number of ">" by 1 smaller than
+its own. I-fct denotes the interpolating function to the grid left
+to it. Then our structure goes as follows:
+[OptCrit1], >[model1], >>[grid], >>[I-fct], >[model2], >>[grid], >>[I-fct], 
+..., [OptCrit2], >[model1], >>[grid], >>[I-fct], ...
+On the top layer the elements come in pairs in sysdata.rda, e.g. 
+.OMSE.O and OMSE.N, which up to entries [I-fct] are identical. 
+We did so because this would ease deletion of the .O items after a
+certain time.
+
+Still, it would have been cheaper as to disk space to use the structure
+[OptCrit], >[model1], >>[grid], >>[I-fct.O], >>[I-Fct.N],
+>[model2], >>[grid], >>[I-fct.O], >>[I-Fct.N], ...
+Maybe we will do so...
+
+6. Namespace issue
+
+It is absolutely necessary that functions I-fct (or I-fct.O, I-fct.N)
+be generated _in_ the namespace of the pkg; otherwise conflicts arise,
+as namespaces have to be loaded twice (and hence pkg installation already
+fails)
+Finding this out took me quite some time!
+
+My initial idea w.r.t to point 3. was to save the grids to some rda file
+and then, to get rid of all RobASt S4 infrastructure, to load them in 
+a virgin R session, maybe separately for <2.16 and >2.16, to call splinefun 
+therein and then to save this as a (small) sysdata.rda file.
+
+THIS DOES NOT WORK.
+
+Hence, now there is infrastructure / functions in RobExtremes, i.e. functions 
+getShapeGrid, getSnGrid, .getLMGrid, .svInt, .saveInterpGrid, .MakeGridList, 
+.recomputeInterpolators, .renameGridName, .copyGrid, and .mergeF (see ?.mergeF), 
+which allow manipulation of the grids _within_ the package namespace.
+
+In particular, to generate I-fct, .MakeGridList calls R base function
+splinefun() _in_ the namespace of RobExtremes hence this namespace figures
+as one of the parent environments of the environment of I-fct.
+
+7. Size of sysdata.rda 
+
+To reduce the size of sysdate.rda, it was a very important trick to explicitely 
+assign a very small environment generated by new.env() to the results of 
+splinefun in .MakeGridList [which still has the namespace of RobExtremes in 
+its parents].
+
+8. Namespace-conformal Inspection / Manipulation 
+
+As seen, to manipulate an existing sysdata.rda files, it is a bad idea to
+load them into a new R-session and then use the save()-d result as new 
+sysdata.rda in the pkg. Still, for checking, inspection is possible 
+with load(). To avoid getting confused with objects in the workspace
+generated elsewhere, a good idea is to load sysdata.rda with 
+load(file, env) into an environment env generated by new.env() particularly
+for this purpose. Access is provided by get(<symbol>, env) and modification
+by assign(<symbol>, value, env).
+
+To do manipulations within the namespace of RobExtremes, we provide helper
+functions .renameGridName, .copyGrid, and .mergeF; the latter is similar to 
+load() but when in both the file and the environment there is a list with
+the same name, the one from the environment does not get overwritten by
+the one from the file, but rather their items get merged.
+
+9. Exports
+
+As this is functionality which should not bother the standard user of
+Robextremes, most of the infrastructure in 8. is not exported. OTOH, 
+these functions could be of interest to the user wanting to generate
+new interpolators for new scale shape families or, say simply for the
+95% VaR, each function remains in the R folder of the pkgs and is
+documented in Rd.
+As a consequence, to use these functions one has to access them by
+RobExtremes:::fct (or, as in interpolationscripts.R, define shortcuts like 
+fct <- RobExtremes:::fct
+
+10. Reproducibility of grid generation
+
+To be able to reproduce all operations which I used to generate the
+contents of sysdata.rda, I provide scripts interpolationscripts.R and
+interpolationsmanipulations.R (not quite executable scripts, rather
+to be used line-wise for copy&paste) im pkg subfolder 
+\inst\AddMaterial\interpolation ; hence they are available after
+pkg installation in the library as Robextremes\AddMaterial\interpolation
+ 
+11. Final Place for Utilities
+
+It might be a good idea to move parts of the interpolation code from 
+RobExtremes to ROptEst, to make it available to others (in particular
+to Matthias with RobLox) without having to import RobExtremes.
+This concerns functions .saveInterpGrid, .MakeGridList, 
+.recomputeInterpolators, .renameGridName, .copyGrid, and .mergeF 
+
+Comments & Suggestions are welcome.
\ No newline at end of file