[Rcpp-devel] Read csv and export object in R

[ogami musashi]

Hello all,

Recently i've been forced to output results from a Wavelet transform to 
files because storing of objects in R took too much memory.

The exported objects are lighter than the objects produced by the 
Wavelet package in R so i can re-import them in R.

Problem is..i have 400 object of 16,5 Mb each. and it take about 6 hours 
to reimport in R! I use the readr package as this is the fastest base 
function in R.

I thus explore the benefits of using directly C++ with Rcpp to see if it 
can lead me to significant gains.

The idea is to read the file and store it into an Rcpp matrix being 
returned in R.

I adapted a C++ code to use Rcpp, it compiles but when using it it 
crashes R:

#include <Rcpp.h>
#include <fstream>
#include <sstream>
#include <string>
using namespace Rcpp;



// [[Rcpp::export]]
NumericMatrix readfilecpp()
{

NumericMatrix output;
std::ifstream myfile("/home/ogami/outBvTem_17dwt_1.csv");

for (int row=0; row<15; ++row)
{
     std::string line;
     std::getline(myfile,line);
     //if(!myfile.good())
         //break;

     std::stringstream iss(line);

for (int col=0; col<4600; ++col )
{
     std::string val;
     std::getline(iss,val,',');
     //if (!iss.good())
         //break;

     std::stringstream convertor(val);
     convertor >> output(row,col);
}
}
return(output);
}

any idea?

Thank you!