From andrea.bertolini80 at gmail.com Sat Dec 22 15:19:12 2018 From: andrea.bertolini80 at gmail.com (Andrea Bertolini) Date: Sat, 22 Dec 2018 15:19:12 +0100 Subject: [Rnomads-user] rNOMADS no longer downloads data Message-ID: Good morning, from today some important rNOMADS commands no longer work: > urls.out <- CrawlModels (abbrev = "gfs_1p00", depth = 2) *Error in basename (grib.filter): a character vector argument expected* Is there a way to overcome the problem? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.galda at gmail.com Sat Dec 22 17:53:03 2018 From: alex.galda at gmail.com (Alexey Galda) Date: Sat, 22 Dec 2018 17:53:03 +0100 Subject: [Rnomads-user] rNOMADS no longer downloads data In-Reply-To: References: Message-ID: I'm getting the same error with GetDODSDates() function. The entire website is down because of these errors now. Would be good to know if there is an easy fix for this. Thanks, Alexey On Sat, 22 Dec 2018 at 15:19, Andrea Bertolini wrote: > Good morning, > > from today some important rNOMADS commands no longer work: > > > urls.out <- CrawlModels (abbrev = "gfs_1p00", depth = 2) > *Error in basename (grib.filter): a character vector argument expected* > > Is there a way to overcome the problem? > > Thanks in advance > _______________________________________________ > Rnomads-user mailing list > Rnomads-user at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danny.c.bowman at gmail.com Sat Dec 22 18:52:30 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Sat, 22 Dec 2018 10:52:30 -0700 Subject: [Rnomads-user] rNOMADS no longer downloads data In-Reply-To: References: Message-ID: I will look into it. I know NOMADS was switching over to https but I thought that was happening in January. It could be their website is down, or it could be they changed how it?s configured. Give me about 48 hours. On Sat, Dec 22, 2018 at 09:53 Alexey Galda wrote: > I'm getting the same error with GetDODSDates() function. > The entire website is down because of these errors now. > > Would be good to know if there is an easy fix for this. > > Thanks, > Alexey > > On Sat, 22 Dec 2018 at 15:19, Andrea Bertolini < > andrea.bertolini80 at gmail.com> wrote: > >> Good morning, >> >> from today some important rNOMADS commands no longer work: >> >> > urls.out <- CrawlModels (abbrev = "gfs_1p00", depth = 2) >> *Error in basename (grib.filter): a character vector argument expected* >> >> Is there a way to overcome the problem? >> >> Thanks in advance >> _______________________________________________ >> Rnomads-user mailing list >> Rnomads-user at lists.r-forge.r-project.org >> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user >> > _______________________________________________ > Rnomads-user mailing list > Rnomads-user at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danny.c.bowman at gmail.com Sat Dec 22 19:11:40 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Sat, 22 Dec 2018 11:11:40 -0700 Subject: [Rnomads-user] rNOMADS no longer downloads data In-Reply-To: References: Message-ID: UPDATE: seems like https migration has occurred after all. I?ll release a fix as soon as I can. On Sat, Dec 22, 2018 at 10:52 Daniel Bowman wrote: > I will look into it. I know NOMADS was switching over to https but I > thought that was happening in January. It could be their website is down, > or it could be they changed how it?s configured. Give me about 48 hours. > > On Sat, Dec 22, 2018 at 09:53 Alexey Galda wrote: > >> I'm getting the same error with GetDODSDates() function. >> The entire website is down because of these errors now. >> >> Would be good to know if there is an easy fix for this. >> >> Thanks, >> Alexey >> >> On Sat, 22 Dec 2018 at 15:19, Andrea Bertolini < >> andrea.bertolini80 at gmail.com> wrote: >> >>> Good morning, >>> >>> from today some important rNOMADS commands no longer work: >>> >>> > urls.out <- CrawlModels (abbrev = "gfs_1p00", depth = 2) >>> *Error in basename (grib.filter): a character vector argument expected* >>> >>> Is there a way to overcome the problem? >>> >>> Thanks in advance >>> _______________________________________________ >>> Rnomads-user mailing list >>> Rnomads-user at lists.r-forge.r-project.org >>> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user >>> >> _______________________________________________ >> Rnomads-user mailing list >> Rnomads-user at lists.r-forge.r-project.org >> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danny.c.bowman at gmail.com Sun Dec 23 07:05:09 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Sat, 22 Dec 2018 23:05:09 -0700 Subject: [Rnomads-user] rNOMADS progress Message-ID: I had hoped a simple string replace http with https would fix it. No such luck, it seems like NCEP changed their website structure too. I will continue to work on the problem. Fortunately it is Christmas break and I have the bandwidth to actually learn how to parse html tables in R (instead of using a generic stackoverflow solution). I will send out a message when I have everything up and running again. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.galda at gmail.com Sun Dec 23 14:56:41 2018 From: alex.galda at gmail.com (Alexey Galda) Date: Sun, 23 Dec 2018 14:56:41 +0100 Subject: [Rnomads-user] rNOMADS progress In-Reply-To: References: Message-ID: Danny, On behalf of all people who downloaded your package 26158 times since 2013, I would like to thank you for this awesome package and the time you spent creating and supporting it since. Happy holidays everyone! Cheers, Alexey On Sun, 23 Dec 2018 at 07:05, Daniel Bowman wrote: > I had hoped a simple string replace http with https would fix it. No such > luck, it seems like NCEP changed their website structure too. I will > continue to work on the problem. Fortunately it is Christmas break and I > have the bandwidth to actually learn how to parse html tables in R (instead > of using a generic stackoverflow solution). > > I will send out a message when I have everything up and running again. > _______________________________________________ > Rnomads-user mailing list > Rnomads-user at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: rNOMADS downloads.png Type: image/png Size: 5443 bytes Desc: not available URL: From danny.c.bowman at gmail.com Mon Dec 24 04:57:39 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Sun, 23 Dec 2018 20:57:39 -0700 Subject: [Rnomads-user] reached out to stackoverflow for NCEP parsing issue Message-ID: Looks like the NCEP website was changed, and my parser that reads the real time model list is now failing. I have reached out to the stackoverflow community for help: https://stackoverflow.com/questions/53909028/html-nodes-no-longer-extracts-table-rows-after-website-update This issue is beyond my expertise...if you know any xpath/css parser wizards, please point them in my direction. Thanks! Danny From danny.c.bowman at gmail.com Tue Dec 25 21:39:59 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Tue, 25 Dec 2018 13:39:59 -0700 Subject: [Rnomads-user] Progress Message-ID: rNOMADS users: I thought I?d give you a nice Christmas present of a new version but it seems like I have to rewrite all the html parsing code...so still working on it. The good news is that at least I know what I?m doing now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danny.c.bowman at gmail.com Fri Dec 28 06:51:04 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Thu, 27 Dec 2018 22:51:04 -0700 Subject: [Rnomads-user] rNOMADS with grib fix is ready for download Message-ID: Hi all, Many thanks for your patience while I worked through the issues wrought by NCEP's transition to https. Originally I thought this would be a simple fix as it was for the archived models via NCDC...but no such luck. I've had to rewrite the html parsers all by hand. CRAN is down until January 2nd but I have a new release ready for download: https://r-forge.r-project.org/scm/viewvc.php/trunk/rNOMADS/rNOMADS_2.4.0.tar.gz?root=rnomads&view=log Real time grib downloading (via GribGrab) now works; DODS (openDAP-alt) still does not. I think enough users rely on Grib that it was worth releasing anyway. Best, Danny From danny.c.bowman at gmail.com Fri Dec 28 06:53:14 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Thu, 27 Dec 2018 22:53:14 -0700 Subject: [Rnomads-user] stay in touch with rNOMADS via twitter Message-ID: Also, anyone who uses Twitter please keep tabs on the new @rNOMADS_r account - I'll be posting rNOMADS news to that as well, and eventually daily rNOMADS/NCEP/NCDC status updates. From BillM at acespower.com Fri Dec 28 13:57:20 2018 From: BillM at acespower.com (Bill Morris) Date: Fri, 28 Dec 2018 12:57:20 +0000 Subject: [Rnomads-user] stay in touch with rNOMADS via twitter In-Reply-To: References: Message-ID: Thank you, Danny! Bill Morris | Senior Meteorologist ACES | 4140 West 99th Street | Carmel, IN 46032 o: 317.344.7042 | m: 317.696.9045 | billm at acespower.com Think before you print NOTICE: This email message and any attachments are for the sole and confidential use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and delete the original message from your computer system and destroy any copies of the message as well as any attachments and notify me immediately at (317) 344-7000. -----Original Message----- From: Rnomads-user On Behalf Of Daniel Bowman Sent: Friday, December 28, 2018 12:53 AM To: Announcements, feature requests, and help using rNOMADS Subject: [Rnomads-user] stay in touch with rNOMADS via twitter *-*EXTERNAL*-* Think before clicking links or attachments. Also, anyone who uses Twitter please keep tabs on the new @rNOMADS_r account - I'll be posting rNOMADS news to that as well, and eventually daily rNOMADS/NCEP/NCDC status updates. _______________________________________________ Rnomads-user mailing list Rnomads-user at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user From danny.c.bowman at gmail.com Fri Dec 28 21:51:45 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Fri, 28 Dec 2018 13:51:45 -0700 Subject: [Rnomads-user] issues with r-forge version of rNOMADS Message-ID: All, There's an issue with the tar archive on r-forge, so I'm attempting to attach a working version to this message. Failing that, please just go ahead and download the .R files here: https://r-forge.r-project.org/scm/viewvc.php/trunk/rNOMADS/rNOMADS/R/?root=rnomads and source them. I am sorry for the inconvenience! CRAN is back online on January 2nd, so unless you need these fixes urgently I'd just wait until I announce that the new version is online before installing. Barring any issues that should be on the first week of January. Sorry for the inconvenience! Danny -------------- next part -------------- A non-text attachment was scrubbed... Name: rNOMADS_2.4.0.tar.gz Type: application/gzip Size: 40528 bytes Desc: not available URL: From danny.c.bowman at gmail.com Sat Dec 29 06:45:16 2018 From: danny.c.bowman at gmail.com (Daniel Bowman) Date: Fri, 28 Dec 2018 22:45:16 -0700 Subject: [Rnomads-user] DODS working (for now) Message-ID: All, Attached is an rNOMADS source that should keep DODS working for now. I expect things to fail again when NCEP switches to https (DODS is still http for some reason). I will be fixing the html parsers to hopefully avoid this from happening. Until then, please use this release for your grib and DODS needs. At present both my grib and dods vignettes (downloading GFS data and doing some basic manipulation) are running fine with this version. Let me know if anything is still not working for you. Best, Danny -------------- next part -------------- A non-text attachment was scrubbed... Name: rNOMADS_2.4.0.tar.gz Type: application/gzip Size: 40583 bytes Desc: not available URL: From guinasso at yahoo.com Sat Dec 29 07:50:58 2018 From: guinasso at yahoo.com (Norman Guinasso) Date: Sat, 29 Dec 2018 06:50:58 +0000 (UTC) Subject: [Rnomads-user] DODS working (for now) In-Reply-To: References: Message-ID: <1912235700.3780285.1546066258410@mail.yahoo.com> Thank you for doing this over your Christmas holiday. I was trying to get it to work and noticed that DODS did not like the https.?Norman Sent from Yahoo Mail for iPad On Friday, December 28, 2018, 11:45 PM, Daniel Bowman wrote: All, Attached is an rNOMADS source that should keep DODS working for now. I expect things to fail again when NCEP switches to https (DODS is still http for some reason).? I will be fixing the html parsers to hopefully avoid this from happening.? Until then, please use this release for your grib and DODS needs. At present both my grib and dods vignettes (downloading GFS data and doing some basic manipulation) are running fine with this version. Let me know if anything is still not working for you. Best, Danny _______________________________________________ Rnomads-user mailing list Rnomads-user at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From guinasso at yahoo.com Sat Dec 29 16:39:43 2018 From: guinasso at yahoo.com (Norman Guinasso) Date: Sat, 29 Dec 2018 15:39:43 +0000 (UTC) Subject: [Rnomads-user] DODS working (for now) In-Reply-To: References: Message-ID: <464791232.3867443.1546097983243@mail.yahoo.com> > library (rNOMADS) Loading required package: rvest Loading required package: xml2 **** Welcome to rNOMADS! Questions? Follow @rNOMADS_r on Twitter or send a message to rnomads-user at lists.r-forge.r-project.org Using rNOMADS as a data source for a publication?? Please cite it! I'm an early career researcher and every citation matters. **** > sessionInfo() R version 3.5.2 (2018-12-20) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.1 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 locale: ?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C?????????????? LC_TIME=en_US.UTF-8??????? LC_COLLATE=en_US.UTF-8?? ? ?[5] LC_MONETARY=en_US.UTF-8??? LC_MESSAGES=en_US.UTF-8??? LC_PAPER=en_US.UTF-8?????? LC_NAME=C??????????????? ? ?[9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C???????????? LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C????? ? attached base packages: [1] stats???? graphics? grDevices utils???? datasets? methods?? base??? ? other attached packages: [1] rNOMADS_2.4.0 rvest_0.3.2?? xml2_1.2.0? ? loaded via a namespace (and not attached): ?[1] httr_1.3.1??????? compiler_3.5.2??? R6_2.2.2????????? magrittr_1.5????? tools_3.5.2?????? bizdays_1.0.6???? yaml_2.2.0????? ? ?[8] Rcpp_0.12.18????? lubridate_1.7.4?? stringi_1.2.4???? timeDate_3043.102 stringr_1.3.1???? jsonlite_1.5??? ? > GetDODSDates("nam") Error in GetDODSDates("nam") : ? The specified URL does not exist!? Make sure your model information is correct.? It is also possible the NOMADS server is down. Details:? Attempted to access https://nomads.ncep.noaa.gov:9090/dods/nam/ but did not succeed... > traceback() 2: stop(paste0("The specified URL does not exist!? Make sure your model information is correct.? It is also possible the NOMADS server is down.\n", ?????? "Details:? Attempted to access ", top.url, " but did not succeed...")) 1: GetDODSDates("nam") I note that if I paste the url into the browser with http and the :9090 removed it works. Norman Guinasso On Friday, December 28, 2018, 11:51:21 PM CST, Daniel Bowman wrote: All, Attached is an rNOMADS source that should keep DODS working for now. I expect things to fail again when NCEP switches to https (DODS is still http for some reason).? I will be fixing the html parsers to hopefully avoid this from happening.? Until then, please use this release for your grib and DODS needs. At present both my grib and dods vignettes (downloading GFS data and doing some basic manipulation) are running fine with this version. Let me know if anything is still not working for you. Best, Danny _______________________________________________ Rnomads-user mailing list Rnomads-user at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rnomads-user -------------- next part -------------- An HTML attachment was scrubbed... URL: