[Reddyproc-users] Constraining Gap Filling to Certain "Epochs"

[Thomas Wutzler]

Dear Kevin and REddyProc users,

Currently this epoch-processing is not available with REddyProc.

Our reason was, that standard REddyProc processing should check for
sufficiently enough data so that applied standard methods are robust
(See e.g. Ladislav's reply on estimating temperature sensitivity E0).
Hence, there is this three month-data limit.

However, there is a way to include a user-defined variable to identify
similar periods for the Lookup-Table-based gap-Filling with the
lower-level functions. I recommend that you define a dummy variable with
an distinct integer value that identifies your epochs or seasons as I
called it. This can already be applied for the uStar-Threshold
estimation (see help on argument seasonFactor.v in usEstUstarThreshold).
In the following I assume that you defined integer column 'seasonFactor'
in your dataset and loaded it to the reference class object.

The specialized gapfilling Method sFillLUT allows to specify five
variables and tolerances. Usually its called from sMDSGapFill with the
three variables:

    ,V1.s='Rg'            ##<< Condition variable 1 (default: Global
radiation 'Rg' in  W m-2)
    ,T1.n=50              ##<< Tolerance interval 1 (default: 50 W m-2)
    ,V2.s='VPD'           ##<< Condition variable 2 (default: Vapour
pressure deficit 'VPD' in hPa)
    ,T2.n=5               ##<< Tolerance interval 2 (default: 5 hPa)
    ,V3.s='Tair'          ##<< Condition variable 3 (default: Air
temperature 'Tair' in degC)
    ,T3.n=2.5             ##<< Tolerance interval 3 (default: 2.5 degC)

You can manually call the Lookup-Table-based gapFilling with additional
variable like
REddyProcObject$sFillLUT(7, V1.s, T1.n, V2.s, T2.n, V3.s, T3.n,
V4.s='seasonFactor', T4.n=0.1 )
This invokes the gapfilling looking at similar peridos within +-7days,
but not regarding periods where variable seasonFactor has a different
value (assuming its identifier values are more than 0.1 apart).
Note, that the naming of the outputs is different from sMDSGapFill, and
you have to do the work of method sMDSGapFill yourself (implement the
sequence successively weaker conditions for filling).

We could extend standard sMDSGapFill to support an additional
user-defined variable. However, there are complicating special cases
that are hard to document consistently (e.g. how to handle the
Mean-Diurnal cycle-based gapfilling for longer gaps that does not
support the seasons)
Maybe, its better to implement a modified more flexible version of
sMDSGapFill with a different name, and let the original sMDSGapFill call
the more flexible version.
If you come up with a good solution, we are happy if you show and
discuss it on r-forge or github.

Best regards
Thomas

On 04.02.2017 18:06, Kevin Wolz wrote:
> Hello REddyProc Users,
> I have an interesting, non-standard requirement for gap filling that I
> am trying to determine how to implement. Our eddy covariance tower is
> placed in a grass field that is cut for hay approximately once per
> month. These cutting events create dramatic, instantaneous shifts in NEE
> that last for several weeks as the grass recovers. You can see our NEE
> data for 2016 is the attached file, including the raw data supplied to
> REddyProcWeb and the gap-filled data after each confidence level of gap
> filling. Overall, the gap filling works quite well. However, it is clear
> that the dramatic shifts in the state of the ecosystem that take place
> during grass cutting do create issues with the algorithm because the
> algorithm is "blind" to these events. You can see the result of this in
> the second attachment, which is zoomed in and identifies erroneous gaps
> that have been filled with data from the wrong "epoch" of pre- versus
> "post" grass cutting.
> 
> While I understand that this type of point shift in NEE is not very
> common in natural ecosystems, there are now more and more eddy
> covariance towers in managed ecosystems that do experience
> management-induced point shifts in NEE. I am wondering if it is somehow
> possible to deal with this situation using the REddyProcWeb tool. My
> first thought was to only supply the tool with data from one "epoch" at
> a time, but the minimum amount of data to pass to the tool is 3 months,
> whereas some of these "epochs" only last a few weeks. Consequently, I
> wonder if it might be necessary to consider another possible column
> input to the tool, similar to the u* season column, that is used to
> restrict the search windows of the gap filling algorithm to within a
> certain "epoch".
> 
> I would very much appreciate any advice on how to make this possible!
> 
> Thank you!
> Kevin
> --
> Kevin Wolz
> PhD Candidate, DeLucia Lab
> University of Illinois at Urbana-Champaign
> Phone: (708) 476-9929

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4990 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.r-forge.r-project.org/pipermail/reddyproc-users/attachments/20170208/2bfc96d4/attachment.bin>