[Rcpp-devel] String encoding (UTF-8 conversion)

[Jeroen Ooms]

I'm interfacing a c++ library which assumes strings are UTF-8. However
strings from R can have various encodings. It's not clear to me how I
need to account for that in Rcpp. For example:

// [[Rcpp::export]]
std::string echo(std::string src){
  return src;
}

This program does not work on windows for non-ascii strings:

> test = "東京"
> echo(test)
[1] "æ ±äº¬

In C programs I always use translateCharUTF8 on all input to make sure
it is UTF8 before I start working with it:

  translateCharUTF8(STRING_ELT(x, i));

Similarly on the output, I explicitly set the encoding to let R know
it this is UTF8:

  SET_STRING_ELT(out, 0, mkCharCE(olds, CE_UTF8));

This ensures that code works across platforms and locales. How do we
go about this in Rcpp?