[Biomod-commits] RasterStack
Wilfried Thuiller
wilfried.thuiller at ujf-grenoble.fr
Mon Jul 5 18:39:44 CEST 2010
Hi again,
RasterStack is like a list of raster objects. A raster object is an object from package "raster" that can be loaded from grid ascii files, or ESRI grids, or IMG files.
For each environmental variable, there is one raster object. Thus to make a projection, you need to to group them together (as you can do when you have multiple vectors for the environmental layers and the group them in a dataframe) in order to make tell BIOMOD or whatever model where are the data.
I gather everyone knows the Worldclim data (http://www.worldclim.org/). They can be downloaded easily from the web in different format. Let's say we have them in ESRI grids.
library(raster)
bio1 = raster("/Data/Wordclim/bio1")
bio2 = raster("/Data/Wordclim/bio2")
....
bio19 = raster("/Data/Worldclim/bio19")
Then you can create a stack from these 19 variables
worldclim=stack(bio_1, bio_2, bio_3, bio_4, bio_5, bio_6, bio_7, bio_8, bio_9, bio_10, bio_11, bio_12, bio_13,
bio_14, bio_15, bio_16, bio_17, bio_18, bio_19)
Now you have a stack a raster that you can use to project the models (if calibrated on the same data of varuables!!)
A more sexy way of doing that is:
bio <- paste(rep('bio', times=19), seq(1:19), sep="_")
library(raster)
for(i in bio)
eval(parse(text=paste(i, "<- raster('/Data/Wordclim/", i,"')", sep="")))
worldclim=stack(bio_1, bio_2, bio_3, bio_4, bio_5, bio_6, bio_7, bio_8, bio_9, bio_10, bio_11, bio_12, bio_13,
bio_14, bio_15, bio_16, bio_17, bio_18, bio_19)
********** Important********
1] The rasters must have the same extent (Xmin, Ymin, Xmas, Ymax) to be stacked.
Use the function "crop", "mask' or "resample" from package raster to make sure.
2] You can plot the rasters or stack very easily
> plot(worldclim)
and then add your points if you have the coordinates (make sure the projections is the same. Worldclim is in WGS84)
> points(X,Y)
Le 5 juil. 2010 à 18:05, César Capinha a écrit :
>
>
> Hi all again,
>
> Can someone provide a deeper insight on how to define and create the class 'RasterStack' needed to make the raster projections?
>
> Best of wishes,
>
> César
>
>
>
> --- On Mon, 5/7/10, biomod-commits-request at lists.r-forge.r-project.org <biomod-commits-request at lists.r-forge.r-project.org> wrote:
>
> From: biomod-commits-request at lists.r-forge.r-project.org <biomod-commits-request at lists.r-forge.r-project.org>
> Subject: Biomod-commits Digest, Vol 15, Issue 1
> To: biomod-commits at lists.r-forge.r-project.org
> Date: Monday, 5 July, 2010, 16:33
>
> Send Biomod-commits mailing list submissions to
> biomod-commits at lists.r-forge.r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> or, via email, send a message with subject or body 'help' to
> biomod-commits-request at lists.r-forge.r-project.org
>
> You can reach the person managing the list at
> biomod-commits-owner at lists.r-forge.r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Biomod-commits digest..."
>
>
> Today's Topics:
>
> 1. new raster functions (C?sar Capinha)
> 2. Re: new raster functions (Wilfried Thuiller)
> 3. memory error- again... (PEP Serra Diaz)
> 4. Re: memory error- again... (Wilfried Thuiller)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 5 Jul 2010 12:05:33 +0000 (GMT)
> From: C?sar Capinha <nrevistada at yahoo.co.uk>
> Subject: [Biomod-commits] new raster functions
> To: biomod-commits at lists.r-forge.r-project.org
> Message-ID: <394679.98688.qm at web27405.mail.ukl.yahoo.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
>
> I'm trying to make some projections of predictions and I would very much benefit from using the new BIOMOD capabilities for projecting directly to raster files. Still, I've downloaded the latest version (windows) and it seems to come without the CHM helpfile. I'm not about the syntax of these new functions.
>
> Can someone provide insights on them?
>
> Best Wishes,
> C?sar Capinha
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100705/7a728d20/attachment-0001.htm>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 5 Jul 2010 14:16:10 +0200
> From: Wilfried Thuiller <wilfried.thuiller at ujf-grenoble.fr>
> Subject: Re: [Biomod-commits] new raster functions
> To: C?sar Capinha <nrevistada at yahoo.co.uk>
> Cc: biomod-commits at r-forge.wu-wien.ac.at
> Message-ID: <36D7AE5F-2430-4A14-96AE-E81DCBE11A6E at ujf-grenoble.fr>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Dear C?sar,
> If you type this, does it work?
>
> ?Projection.raster
>
> Wilfried
>
>
> Le 5 juil. 2010 ? 14:05, C?sar Capinha a ?crit :
>
> > Hi all,
> >
> > I'm trying to make some projections of predictions and I would very much benefit from using the new BIOMOD capabilities for projecting directly to raster files. Still, I've downloaded the latest version (windows) and it seems to come without the CHM helpfile. I'm not about the syntax of these new functions.
> >
> > Can someone provide insights on them?
> >
> > Best Wishes,
> > C?sar Capinha
> >
> > _______________________________________________
> > Biomod-commits mailing list
> > Biomod-commits at lists.r-forge.r-project.org
> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> --------------------------
> Dr. Wilfried Thuiller
> Laboratoire d'Ecologie Alpine, UMR CNRS 5553
> Universit? Joseph Fourier
> BP53, 38041 Grenoble cedex 9, France
> tel: +33 (0)4 76 51 44 97
> fax: +33 (0)4 76 51 42 79
>
> Email: wilfried.thuiller at ujf-grenoble.fr
> Home page: http://www.will.chez-alice.fr
> Website: http://www-leca.ujf-grenoble.fr/equipes/tde.htm
>
> FP6 European MACIS project: http://www.macis-project.net
> FP6 European EcoChange project: http://www.ecochange-project.eu
>
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100705/55d7a47f/attachment-0001.htm>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 05 Jul 2010 15:47:22 +0100
> From: PEP Serra Diaz <Josep.Serra at uab.cat>
> Subject: [Biomod-commits] memory error- again...
> To: biomod-commits at r-forge.wu-wien.ac.at
> Message-ID: <9e9e8098525c.4c31fe8a at uab.es>
> Content-Type: text/plain; charset=us-ascii
>
>
> Dear BIOMODers,
>
> I have a hughe dataset (more or less 70000 rows) and the error is, of course: cannot allocate vector size of 459.1 MB. (I already increased memory.limt () )
>
> In that huge DF many species are recorded, so I though creating a litle loop so I could go species by species and therefore would not load a huge dataset (Question 1- Is that making any sense?). In each species model, presences and absences are equaled, so we keep prevalence constant, and the strategy of PseudoA selection is SRE. (see script below)
>
> For the first species (only 4500 presences) R retrieves this memory error when computing RF, which I cannot understand for a DF of 9000 rows (P+A), and I even the number of repetitions is set to 1.
>
> Secondly, I checked that BIOMOD creates an object: Biomod.PA.data with 40000 records aprox. I assume that these are the presences and the ALL absences selected outside the SurfaceRange (strategy selected), and later on BIOMOD will select the number of absences indicated (=Number of Presences in this case). Question 2- Is this correct or is it loading ALL absences outside the range? (maybe that's why it retrieves an error.)
>
> Question 3- Any suggestion?
>
> # i corresponds to the column where the species Presence or Absence is located
>
> for (i in "spColumID") {
>
> Initial.State (Response=Sp.Env[,i], Explanatory=Sp.Env[,c(250,248,251,199,164,181)],IndependentResponse=NULL, IndependentExplanatory=NULL,sp.name=names[i])
>
> Models(GLM = T, TypeGLM = "quad", Test = "AIC", GBM = T, No.trees = 3000, GAM = T, Spline=2, CTA = T, CV.tree = 50, ANN = T, CV.ann = 2, SRE = F, FDA = T, MARS = T, RF = T, NbRunEval = 1, DataSplit = 70, Yweights=NULL, Roc=T, Optimized.Threshold.Roc=T, Kappa=T, TSS=T, KeepPredIndependent = F, VarImport=5, NbRepPA=1, strategy="sre", coor=Coor, distance=2, nb.absences=sum(Sp.Env[,i]))
>
> }
>
>
> Thank you very much in advance for reading such a long post with 3 Questions!
>
> Pep
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 5 Jul 2010 17:33:32 +0200
> From: Wilfried Thuiller <wilfried.thuiller at ujf-grenoble.fr>
> Subject: Re: [Biomod-commits] memory error- again...
> To: PEP Serra Diaz <Josep.Serra at uab.cat>
> Cc: biomod-commits at r-forge.wu-wien.ac.at
> Message-ID: <096790AC-CF60-4B41-872F-ED5A12F4BF80 at ujf-grenoble.fr>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Dear Pep,
>
> > In that huge DF many species are recorded, so I though creating a litle loop so I could go species by species and therefore would not load a huge dataset (Question 1- Is that making any sense?). In each species model, presences and absences are equaled, so we keep prevalence constant, and the strategy of PseudoA selection is SRE. (see script below)
>
> It does make sense to run BIOMOD species by species. This what I am usually doing when using grid systems.
>
> > For the first species (only 4500 presences) R retrieves this memory error when computing RF, which I cannot understand for a DF of 9000 rows (P+A), and I even the number of repetitions is set to 1.
> >
> > Secondly, I checked that BIOMOD creates an object: Biomod.PA.data with 40000 records aprox. I assume that these are the presences and the ALL absences selected outside the SurfaceRange (strategy selected), and later on BIOMOD will select the number of absences indicated (=Number of Presences in this case). Question 2- Is this correct or is it loading ALL absences outside the range? (maybe that's why it retrieves an error.)
>
> However, it will much more powerful to also create a workspace and data folder to store the results for each species.
> From what I understand from your code, everything will be re-written at each iteration.
>
> > Question 3- Any suggestion?
>
> Yes indeed.
> First of all, I would rather use random pseudo-absence instead of SRE and select a large number of pseudo-absence. BIOMOD will automatically make sure that the weighted sum of 1 equals the weighted sum of 0 (weighted prevalence = 0.5). We are preparing a MS with virtual data to show that this is a much better solution than using SRE or anything else.
>
> In the case where most of your species have less than 5000 occurrences, selected 20000 absences and it should work rather well.
>
> >
> > # i corresponds to the column where the species Presence or Absence is located
> >
> > for (i in "spColumID") {
> >
> > Initial.State (Response=Sp.Env[,i], Explanatory=Sp.Env[,c(250,248,251,199,164,181)],IndependentResponse=NULL, IndependentExplanatory=NULL,sp.name=names[i])
> >
> > Models(GLM = T, TypeGLM = "quad", Test = "AIC", GBM = T, No.trees = 3000, GAM = T, Spline=2, CTA = T, CV.tree = 50, ANN = T, CV.ann = 2, SRE = F, FDA = T, MARS = T, RF = T, NbRunEval = 1, DataSplit = 70, Yweights=NULL, Roc=T, Optimized.Threshold.Roc=T, Kappa=T, TSS=T, KeepPredIndependent = F, VarImport=5, NbRepPA=1, strategy="sre", coor=Coor, distance=2, nb.absences=sum(Sp.Env[,i]))
> >
> > }
>
> Depending on which OS you are on, there are different solutions. With MacOS or Unic, you could use R in batch mode.
> On Windows, what you wrote is fine excepted that you re-write everything on the same .RData which is not what you want to do (it explains the confusion with the number of records).
>
> First of all, you should create a folder for each species in the loop, then run BIOMOD in this folder, and then once Models is finished, removed everything from the workspce to make sure you are using some data from the previous run.
>
> It could be simply something along these lines, probably to adapt a bit with for your data and OS.
>
>
>
> setwd("YOUR FOLDER WHERE YOU WANT TO STORE THE RESULS")
>
> path = getwd()
>
> for (i in "spColumID") {
>
> #Create a folder you store the results for each species separately.
> dir.create(paste(path, "/", names[i], sep=""))
>
> #Set the workspace in this folder
> setwd(paste(path, "/", names[i], sep=""))
>
> Initial.State (Response=Sp.Env[,i], Explanatory=Sp.Env[,c(250,248,251,199,164,181)],IndependentResponse=NULL, IndependentExplanatory=NULL,sp.name=names[i])
>
> Models(GLM = T, TypeGLM = "quad", Test = "AIC", GBM = T, No.trees = 3000, GAM = T, Spline=2, CTA = T, CV.tree = 50, ANN = T, CV.ann = 2, SRE = F, FDA = T, MARS = T, RF = T, NbRunEval = 1, DataSplit = 70, Yweights=NULL, Roc=T, Optimized.Threshold.Roc=T, Kappa=T, TSS=T, KeepPredIndependent = F, VarImport=5, NbRepPA=1, strategy="sre", coor=Coor, distance=2, nb.absences=sum(Sp.Env[,i]))
>
> #keep the files you want to keep in a vector before deleting everything
> t= c("path", "Sp.Env", "names")
> Rem = ls()
>
> #Delete everything but the object in t.
> rm(list=(Rem[is.na(match(Rem, t))]))
>
> }
>
>
>
> Hope it helps,
> Best
> Wilfried
>
>
>
>
>
>
> >
> >
> > Thank you very much in advance for reading such a long post with 3 Questions!
> >
> > Pep
> >
> >
> > _______________________________________________
> > Biomod-commits mailing list
> > Biomod-commits at lists.r-forge.r-project.org
> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> --------------------------
> Dr. Wilfried Thuiller
> Laboratoire d'Ecologie Alpine, UMR CNRS 5553
> Universit? Joseph Fourier
> BP53, 38041 Grenoble cedex 9, France
> tel: +33 (0)4 76 51 44 97
> fax: +33 (0)4 76 51 42 79
>
> Email: wilfried.thuiller at ujf-grenoble.fr
> Home page: http://www.will.chez-alice.fr
> Website: http://www-leca.ujf-grenoble.fr/equipes/tde.htm
>
> FP6 European MACIS project: http://www.macis-project.net
> FP6 European EcoChange project: http://www.ecochange-project.eu
>
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100705/cec1a13b/attachment.htm>
>
> ------------------------------
>
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
>
> End of Biomod-commits Digest, Vol 15, Issue 1
> *********************************************
>
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
--------------------------
Dr. Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR CNRS 5553
Université Joseph Fourier
BP53, 38041 Grenoble cedex 9, France
tel: +33 (0)4 76 51 44 97
fax: +33 (0)4 76 51 42 79
Email: wilfried.thuiller at ujf-grenoble.fr
Home page: http://www.will.chez-alice.fr
Website: http://www-leca.ujf-grenoble.fr/equipes/tde.htm
FP6 European MACIS project: http://www.macis-project.net
FP6 European EcoChange project: http://www.ecochange-project.eu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100705/f5a68407/attachment-0001.htm>
More information about the Biomod-commits
mailing list