[datatable-help] fread from RAM DIsk
Raj Dasgupta
xbsd at yahoo.com
Sat Jul 13 02:04:44 CEST 2013
Hi all,
I have been looking at lowering the time spent in I/O while using fread on a csv file. Following a suggestion on the mailing list, I attempted to use fread on a csv file stored on a ramdisk. It took 5 times longer to read from the Ram Disk than it did to read from the SSD.
I presume this has to do with the fact that the data is being copied from RAM to RAM and requires more effort rather than from Disk to RAM.
Any suggestions on if there are alternative methods to read files in faster using a RAM Disk would be very helpful.
Benchmarks on time taken for reading from SSD vs Ramdisk on the same 416MB file is given below
####
> timer = proc.time(); z <- fread("testInRAM.csv"); proc.time() - timer
user system elapsed
25.067 0.433 25.485 ##### Read from RAMDisk
> setwd("/Users/xbsd/")
> timer = proc.time(); z <- fread("test.csv"); proc.time() - timer
user system elapsed
5.507 0.177 5.680 ###### Read from SSD
> system("ls -alh test.csv")
-rw-r--r-- 1 xbsd staff 416M Jul 12 19:30 test.csv
> system("ls -alh /Volumes/ramdisk/testInRAM.csv")
-rw-r--r-- 1 xbsd staff 416M Jul 12 19:32 /Volumes/ramdisk/testInRAM.csv
Thanks in advance,
- Raj.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20130712/b8abc561/attachment.html>
More information about the datatable-help
mailing list