[Robast-commits] r966 - in branches/robast-1.1/pkg: RobAStRDA/inst/AddMaterial/interpolation RobExtremes/inst/AddMaterial/interpolation
noreply at r-forge.r-project.org
noreply at r-forge.r-project.org
Wed Jul 18 11:03:10 CEST 2018
Author: ruckdeschel
Date: 2018-07-18 11:03:09 +0200 (Wed, 18 Jul 2018)
New Revision: 966
Modified:
branches/robast-1.1/pkg/RobAStRDA/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
branches/robast-1.1/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
Log:
[RobAStRDA&RobExtremes] in branch 1.1: updated WriteUp-Interpolators.txt
Modified: branches/robast-1.1/pkg/RobAStRDA/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
===================================================================
--- branches/robast-1.1/pkg/RobAStRDA/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt 2018-07-18 08:32:29 UTC (rev 965)
+++ branches/robast-1.1/pkg/RobAStRDA/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt 2018-07-18 09:03:09 UTC (rev 966)
@@ -1,24 +1,26 @@
WRITE-UP:
Interpolation Grids in R-Packages
-Peter Ruckdeschel, Jan 27 2013 revised Mar 12 2013
+Peter Ruckdeschel, Jan 27 2013 revised Mar 12 2013 revised Jul 18 2018
1. Starting Point:
-Computation of optimally robust ICs in our R pkgs works well, but may be slow,
-in particular in case of lack of invariance---like in scale-shape models, when
-we cannot move the IC from one parameter value theta to the next by invariance.
+Computation of optimally robust influence curves [=opt-rob ICs] in our R pkgs
+works well, but may be slow, in particular in case of lack of invariance---like
+in scale-shape models, when we cannot move the IC from one parameter value theta
+to the next by invariance.
Then we have to recompute the IC for each theta anew.
-As the opt-rob ICs are given through Lagrange multipliers LMs) which are continuous
-in theta as shown by Matthias Kohl, we may compute opt-rob ICs for a grid of
-theta values offline and then, for a new theta value use interpolation.
-The same strategy applies to speed up evaluation of scale functional Sn (Croux)
-for Shape-Scale-models.
+As the opt-rob ICs are given through Lagrange multipliers [=LMs]) which are
+continuous in theta as shown by Matthias Kohl (see his PhD thesis), we may
+compute opt-rob ICs for a grid of theta values offline and then, for a new
+theta value use interpolation.
+The same strategy applies to speed up evaluation of scale functional Sn
+(Croux, Rousseeuw) for Shape-Scale-models.
2. Problem when passing from R-2.15 to R-3.0
From R-2.15 to R-3.0 Brian Ripley (R-Core) has changed some non-exported
-interfaces to C-code. This affects approxfun and splinefun which we
-are using for interpolation because saving the results of approxfun and
+interfaces to C-code. This affects approxfun and splinefun which we are
+using for interpolation because saving the results of approxfun and
splinefun comes up much faster (experiments by Matthias) than using approx
resp. spline. Now approxfun, splinefun from R-3.0 on return functions
whose body contains R code which is not yet interpretable with R-2.15
@@ -28,6 +30,13 @@
one with suffix ".N" for >2.16, and at run time determine the current
R version and take the suitable one.
+[added Jul 2018]: In principle, this could be solved now, requiring
+R >= 3.3 (which is no longer "too new"), but for compatibility, we
+leave it as is (the respective sysdata.rda file containing functions
+for both R >= 3.0 and R <= 2.15 only grows by less than 50 KB) and
+indicated how to use this for R <= 2.15 in a file HowTo in the
+package main folder.
+
3. Size of packages
With CRAN getting larger (>4000 pkgs) people from CRAN have set up some
@@ -54,36 +63,41 @@
5. Datastructure
As we deal with several robust optimality criteria (MSE, RMX, MBR)
-and several models (GPD, GEVD, Gamma, Weibull), our grids are stored
-in nested lists. Debatable, but done so for the moment.
+and several models---currently: GPD, GEVD (with and without knowing
+mu), Gamma, Weibull---, our grids are stored in nested lists.
+Debatable, but done so for the moment. Similarly, we store---for
+the same models---fast interpolators for the Sn estimator for scale.
-After a discussion with Gerald Kroisandt, we now have a more sparse
+After a discussion with Gerald Kroisandt, we now have a sparser
data structure.
Let's use the following notation for describing the list structure:
-Each layer in the hierarchie gives one ">" and an item is inserted
+Each layer in the hierarchy gives one ">" and an item is inserted
below the item next left to it with number of ">" by 1 smaller than
its own. I-fct denotes the interpolating function to the grid left
-to it (named "fun"). {} denote optional entries and capture that one may want
-to smooth out the original interpolation grids in entries 'grid',
-giving smoothed grids written into entries 'gridS'.
+to it (named "fun"). {} denote optional entries and capture that one
+may want to smooth out the original interpolation grids in
+entries 'grid', giving smoothed grids written into entries 'gridS'.
OptCrit for the time being is either in ".OMSE", ".MBRE", ".RMXE" or ".Sn".
Models for the time being are GPD, GEVD, Gamma, Weibull.
Then our structure goes as follows:
-[OptCrit], >[model1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
->[model2], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N], ...
+[model1], >[OptCrit1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+[model1], >[OptCrit2], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+...
+[model1], >Sn, >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+[model2], >[OptCrit1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N], ...
-For instance, to get the clipping height "b" in OMSE for "GEV" at
-theta = (xi=0.3) for >R-2.16, we may write
- .OMSE[["GEVFamily"]][["fun.N"]][[1]](0.3)
+For instance, to get the clipping height "b" in OMSE for "GEV" with
+known parameter mu at theta = (xi=0.3) for >R-2.16, we may write
+ .GEV[["OMSE"]][["fun.N"]][[1]](0.3)
-
6. Namespace issue
It is absolutely necessary that functions I-fct (or I-fct.O, I-fct.N)
be generated _in_ the namespace of the pkg; otherwise conflicts arise,
as namespaces have to be loaded twice (and hence pkg installation already
fails)
+
Finding this out took me quite some time!
My initial idea w.r.t to point 3. was to save the grids to some rda file
@@ -102,13 +116,16 @@
non-exported functions .RMXE.th, .MBRE.th, .OMSE.th and .getLMGrid to compute
Lagrange multipliers for "OMSE"-, "MBR"-, "RMX"-ICs, .generateInterpGrid to
produce the grid and .saveGridToCSV, and .readGridFromCSV to read grids from
-files and write grids to files.
+files and write grids to files (all in file interpolLM.R).
-Infrastructure particular for scale-shape models is maintained in pkg 'RobExtremes',
-i.e. non-exported functions .RMXE.xi, .OMSE.xi, .MBEE.xi, .modify.xi.PFam.call,
+Infrastructure particular for scale-shape models is maintained in pkg 'RobExtremes'.
+I.e. non-exported functions .RMXE.xi, .OMSE.xi, .MBEE.xi, .modify.xi.PFam.call,
.getLMGrid to compute respective Lagrante multipliers, and .svInt and
-.generateInterpGridSn to generate the grids for LM's and Sn. Finally, .getPsi
-to given set of interpolators generates an optimally robust IC.
+.generateInterpGridSn to generate the grids for LM's and Sn (all in file interpolLM.R).
+The respective infrastructure for the grids for Sn is provided through functions
+getShapeGrid, getSnGrid, .generateInterpGridSn (all in file interpolSn.R)
+Finally, .getPsi (in file internal-getpsi.R) to given set of interpolators generates
+an optimally robust IC, and .Sn.intp accesses the interpolator for Sn (in file SnQn.R)
Actual code to produce the interpolators and to manipulate the grids
(including smoothing grids out) is maintained in pkg 'RobAStRDA' which
@@ -147,7 +164,8 @@
9. Exports
As this is functionality which should not bother the standard user of
-RobExtremes, basicall all infrastructure in 6. and 8. is not exported.
+'RobExtremes', basically all infrastructure mentioned in sections 6
+and 8 of this write-up is not exported.
OTOH, these functions could be of interest to the user wanting to generate
new interpolators for new scale shape families or, say simply for the
95% VaR, each function remains in the R folder of the pkgs and is
@@ -167,4 +185,4 @@
hence they are available after pkg installation in the library as
RobAStRDA\AddMaterial\interpolation resp. Robextremes\AddMaterial\interpolation
-Comments & Suggestions are welcome.
\ No newline at end of file
+Comments & Suggestions are welcome.
Modified: branches/robast-1.1/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt
===================================================================
--- branches/robast-1.1/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt 2018-07-18 08:32:29 UTC (rev 965)
+++ branches/robast-1.1/pkg/RobExtremes/inst/AddMaterial/interpolation/WriteUp-Interpolators.txt 2018-07-18 09:03:09 UTC (rev 966)
@@ -1,24 +1,26 @@
WRITE-UP:
Interpolation Grids in R-Packages
-Peter Ruckdeschel, Jan 27 2013 revised Mar 12 2013
+Peter Ruckdeschel, Jan 27 2013 revised Mar 12 2013 revised Jul 18 2018
1. Starting Point:
-Computation of optimally robust ICs in our R pkgs works well, but may be slow,
-in particular in case of lack of invariance---like in scale-shape models, when
-we cannot move the IC from one parameter value theta to the next by invariance.
+Computation of optimally robust influence curves [=opt-rob ICs] in our R pkgs
+works well, but may be slow, in particular in case of lack of invariance---like
+in scale-shape models, when we cannot move the IC from one parameter value theta
+to the next by invariance.
Then we have to recompute the IC for each theta anew.
-As the opt-rob ICs are given through Lagrange multipliers LMs) which are continuous
-in theta as shown by Matthias Kohl, we may compute opt-rob ICs for a grid of
-theta values offline and then, for a new theta value use interpolation.
-The same strategy applies to speed up evaluation of scale functional Sn (Croux)
-for Shape-Scale-models.
+As the opt-rob ICs are given through Lagrange multipliers [=LMs]) which are
+continuous in theta as shown by Matthias Kohl (see his PhD thesis), we may
+compute opt-rob ICs for a grid of theta values offline and then, for a new
+theta value use interpolation.
+The same strategy applies to speed up evaluation of scale functional Sn
+(Croux, Rousseeuw) for Shape-Scale-models.
2. Problem when passing from R-2.15 to R-3.0
From R-2.15 to R-3.0 Brian Ripley (R-Core) has changed some non-exported
-interfaces to C-code. This affects approxfun and splinefun which we
-are using for interpolation because saving the results of approxfun and
+interfaces to C-code. This affects approxfun and splinefun which we are
+using for interpolation because saving the results of approxfun and
splinefun comes up much faster (experiments by Matthias) than using approx
resp. spline. Now approxfun, splinefun from R-3.0 on return functions
whose body contains R code which is not yet interpretable with R-2.15
@@ -28,6 +30,13 @@
one with suffix ".N" for >2.16, and at run time determine the current
R version and take the suitable one.
+[added Jul 2018]: In principle, this could be solved now, requiring
+R >= 3.3 (which is no longer "too new"), but for compatibility, we
+leave it as is (the respective sysdata.rda file containing functions
+for both R >= 3.0 and R <= 2.15 only grows by less than 50 KB) and
+indicated how to use this for R <= 2.15 in a file HowTo in the
+package main folder.
+
3. Size of packages
With CRAN getting larger (>4000 pkgs) people from CRAN have set up some
@@ -54,28 +63,33 @@
5. Datastructure
As we deal with several robust optimality criteria (MSE, RMX, MBR)
-and several models (GPD, GEVD, Gamma, Weibull), our grids are stored
-in nested lists. Debatable, but done so for the moment.
+and several models---currently: GPD, GEVD (with and without knowing
+mu), Gamma, Weibull---, our grids are stored in nested lists.
+Debatable, but done so for the moment. Similarly, we store---for
+the same models---fast interpolators for the Sn estimator for scale.
-After a discussion with Gerald Kroisandt, we now have a more sparse
+After a discussion with Gerald Kroisandt, we now have a sparser
data structure.
Let's use the following notation for describing the list structure:
-Each layer in the hierarchie gives one ">" and an item is inserted
+Each layer in the hierarchy gives one ">" and an item is inserted
below the item next left to it with number of ">" by 1 smaller than
its own. I-fct denotes the interpolating function to the grid left
-to it (named "fun"). {} denote optional entries and capture that one may want
-to smooth out the original interpolation grids in entries 'grid',
-giving smoothed grids written into entries 'gridS'.
+to it (named "fun"). {} denote optional entries and capture that one
+may want to smooth out the original interpolation grids in
+entries 'grid', giving smoothed grids written into entries 'gridS'.
OptCrit for the time being is either in ".OMSE", ".MBRE", ".RMXE" or ".Sn".
Models for the time being are GPD, GEVD, Gamma, Weibull.
Then our structure goes as follows:
-[OptCrit], >[model1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
->[model2], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N], ...
+[model1], >[OptCrit1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+[model1], >[OptCrit2], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+...
+[model1], >Sn, >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N],
+[model2], >[OptCrit1], >>[grid], {>>[gridS],} >>[I-fct.O], >>[I-Fct.N], ...
-For instance, to get the clipping height "b" in OMSE for "GEV" at
-theta = (xi=0.3) for >R-2.16, we may write
- .OMSE[["GEVFamily"]][["fun.N"]][[1]](0.3)
+For instance, to get the clipping height "b" in OMSE for "GEV" with
+known parameter mu at theta = (xi=0.3) for >R-2.16, we may write
+ .GEV[["OMSE"]][["fun.N"]][[1]](0.3)
6. Namespace issue
@@ -83,6 +97,7 @@
be generated _in_ the namespace of the pkg; otherwise conflicts arise,
as namespaces have to be loaded twice (and hence pkg installation already
fails)
+
Finding this out took me quite some time!
My initial idea w.r.t to point 3. was to save the grids to some rda file
@@ -101,13 +116,16 @@
non-exported functions .RMXE.th, .MBRE.th, .OMSE.th and .getLMGrid to compute
Lagrange multipliers for "OMSE"-, "MBR"-, "RMX"-ICs, .generateInterpGrid to
produce the grid and .saveGridToCSV, and .readGridFromCSV to read grids from
-files and write grids to files.
+files and write grids to files (all in file interpolLM.R).
-Infrastructure particular for scale-shape models is maintained in pkg 'RobExtremes',
-i.e. non-exported functions .RMXE.xi, .OMSE.xi, .MBEE.xi, .modify.xi.PFam.call,
+Infrastructure particular for scale-shape models is maintained in pkg 'RobExtremes'.
+I.e. non-exported functions .RMXE.xi, .OMSE.xi, .MBEE.xi, .modify.xi.PFam.call,
.getLMGrid to compute respective Lagrante multipliers, and .svInt and
-.generateInterpGridSn to generate the grids for LM's and Sn. Finally, .getPsi
-to given set of interpolators generates an optimally robust IC.
+.generateInterpGridSn to generate the grids for LM's and Sn (all in file interpolLM.R).
+The respective infrastructure for the grids for Sn is provided through functions
+getShapeGrid, getSnGrid, .generateInterpGridSn (all in file interpolSn.R)
+Finally, .getPsi (in file internal-getpsi.R) to given set of interpolators generates
+an optimally robust IC, and .Sn.intp accesses the interpolator for Sn (in file SnQn.R)
Actual code to produce the interpolators and to manipulate the grids
(including smoothing grids out) is maintained in pkg 'RobAStRDA' which
@@ -146,7 +164,8 @@
9. Exports
As this is functionality which should not bother the standard user of
-RobExtremes, basicall all infrastructure in 6. and 8. is not exported.
+'RobExtremes', basically all infrastructure mentioned in sections 6
+and 8 of this write-up is not exported.
OTOH, these functions could be of interest to the user wanting to generate
new interpolators for new scale shape families or, say simply for the
95% VaR, each function remains in the R folder of the pkgs and is
@@ -166,4 +185,4 @@
hence they are available after pkg installation in the library as
RobAStRDA\AddMaterial\interpolation resp. Robextremes\AddMaterial\interpolation
-Comments & Suggestions are welcome.
\ No newline at end of file
+Comments & Suggestions are welcome.
More information about the Robast-commits
mailing list