[datatable-help] Reading character strings into fread

[Hideyoshi Maeda]

I am having a bit of trouble reading character string tables into fread.

The character string was generated as part of the output from an API GET request from dropbox. (the API link to dropbox was done via the httr package)

Basically I would like freed to do something similar to read.csv…but it seems to not work….please see the below example, i which i download a csv file, with the content of response as a character string….

I would rather not use read.csv to write a csv file first then and get fread to read it in, as it seems like it is a bit of a round about way of doing it, rather than reading in the table directly from the character string.

any help would be much appreciated.

thanks

p.s. @MD…i thought i responded to your previous email…but clearly I didn't….so i thought i might as well email the list as well..

> db.app <- oauth_app("db",key=getOption("DropboxKey"), secret=getOption("DropboxSecret"))
> db.sig <- sign_oauth1.0(db.app, token=getOption("DropboxOAuthKey"), token_secret=getOption("DropboxOAuthSecret"))
> 
> response <- GET(url=paste0("https://api-content.dropbox.com/1/files/dropbox/",gsub("%2F","/",curlEscape("!! test folder/new file.csv"))),config=c(db.sig,add_headers(Accept="x-dropbox-metadata")))
> response
Response [https://api-content.dropbox.com/1/files/dropbox/%21%21%20test%20folder/new%20file.csv]
  Status: 200
  Content-type: text/csv; charset=ascii
"Date.and.Time","Open","High","Low","Close","Volume"
"2007/01/02 01:46:00",20083,20088,20071,20075,212
"2007/01/02 01:47:00",20075,20120,20075,20106,328
"2007/01/02 01:48:00",20105,20110,20094,20096,256
"2007/01/02 01:49:00",20096,20106,20085,20099,177
"2007/01/02 01:50:00",20098,20100,20081,20092,184
"2007/01/02 01:51:00",20091,20094,20087,20093,48
"2007/01/02 01:52:00",20093,20095,20085,20088,147
"2007/01/02 01:53:00",20088,20090,20086,20089,26
"2007/01/02 01:54:00",20089,20100,20089,20091,116 ...
> require(data.table)
Loading required package: data.table
data.table 1.8.7  For help type: help("data.table")
> x <- fread(content(response),sep=",",verbose=TRUE)
Input contains a \n (or is ""), taking this to be text input (not a filename)
Detected eol as \n only (no \r afterwards), the UNIX and Mac standard.
Looking for supplied sep ',' on line 30 (the last non blank line in the first 30) ... found
Found 6 columns
First row with 6 fields occurs on line 1 (either column names or first row of data)
All the fields on line 1 are character fields. Treating as the column names.
Count of eol after first data row: 82
Subtracted 0 for last eol and any trailing empty lines, leaving 82 data rows
Type codes: 300000 (first 5 rows)
Type codes: 300000 (+middle 5 rows)
Error in fread(content(response), sep = ",", verbose = TRUE) : 
  Expected sep (',') but '
> x <- read.csv(text=content(response),header=TRUE,stringsAsFactors=FALSE)
> head(x)
        Date.and.Time  Open  High   Low Close Volume
1 2007/01/02 01:46:00 20083 20088 20071 20075    212
2 2007/01/02 01:47:00 20075 20120 20075 20106    328
3 2007/01/02 01:48:00 20105 20110 20094 20096    256
4 2007/01/02 01:49:00 20096 20106 20085 20099    177
5 2007/01/02 01:50:00 20098 20100 20081 20092    184
6 2007/01/02 01:51:00 20091 20094 20087 20093     48
> str(content(response))
 chr "\"Date.and.Time\",\"Open\",\"High\",\"Low\",\"Close\",\"Volume\"\n\"2007/01/02 01:46:00\",20083,20088,20071,20075,212\n\"2007/0"| __truncated__
> str(x)
'data.frame':	100 obs. of  6 variables:
 $ Date.and.Time: chr  "2007/01/02 01:46:00" "2007/01/02 01:47:00" "2007/01/02 01:48:00" "2007/01/02 01:49:00" ...
 $ Open         : int  20083 20075 20105 20096 20098 20091 20093 20088 20089 20090 ...
 $ High         : int  20088 20120 20110 20106 20100 20094 20095 20090 20100 20093 ...
 $ Low          : int  20071 20075 20094 20085 20081 20087 20085 20086 20089 20083 ...
 $ Close        : int  20075 20106 20096 20099 20092 20093 20088 20089 20091 20093 ...
 $ Volume       : int  212 328 256 177 184 48 147 26 116 47 ...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20130201/a6d58924/attachment.html>