From shabbychef at gmail.com Mon Mar 2 23:04:46 2015 From: shabbychef at gmail.com (steven pav) Date: Mon, 2 Mar 2015 14:04:46 -0800 Subject: [Rcpp-devel] Is it possible to call an R internal C function from Rcpp? Message-ID: hi all; I want to write the density function for the doubly non-central F distribution using Rcpp. The algorithm (from Paolella's Intermediate Probability, listing 10.8) computes the weighted sum of the PDF function of the singly non-central F distribution. This is available in R via df; the internal df function is apparently function (x, df1, df2, ncp, log = FALSE) { if (missing(ncp)) .External(C_df, x, df1, df2, log) else .External(C_dnf, x, df1, df2, ncp, log) } Is it possible to call this C_dnf from Rcpp? One thought I had was to use Rinside, but I want to avoid that overhead if possible. cheers, -- --sep -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Mon Mar 2 23:17:45 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Mon, 2 Mar 2015 16:17:45 -0600 Subject: [Rcpp-devel] Is it possible to call an R internal C function from Rcpp? In-Reply-To: References: Message-ID: <21748.57737.778567.537776@max.nulle.part> On 2 March 2015 at 14:04, steven pav wrote: | I want to write the density function for the doubly non-central F distribution | using Rcpp. The algorithm (from Paolella's Intermediate Probability, listing | 10.8) computes the weighted sum of the PDF function of the singly non-central F | distribution. This is available in R via df; the internal df function is | apparently? | | function (x, df1, df2, ncp, log = FALSE)? | { | ? ? if (missing(ncp))? | ? ? ? ? .External(C_df, x, df1, df2, log) | ? ? else .External(C_dnf, x, df1, df2, ncp, log) | } | | Is it possible to call this C_dnf from Rcpp? One thought I had was to use | Rinside, but I want to avoid that overhead if possible. "Quite possibly" in this particular case as we have both the sugar functions for df and dnf, and the single-value wrapper of the Rmath functions. For sugar, definitions are a tad, ahem, "cryptic" for use of macros. From git/rcpp/inst/include/Rcpp/stats/f.h: RCPP_DPQ_2(f,::Rf_df,::Rf_pf,::Rf_qf) git/rcpp/inst/include/Rcpp/stats/nf.h: RCPP_DPQ_3(nf,::Rf_dnf,::Rf_pnf,::Rf_qnf) where the key are '2' and '3' for the number of arguments, as above. These become sugar functions Rcpp::df(), Rcpp::qf(), Rcpp::pf() and Rcpp::dnf(), Rcpp::qnf(), Rcpp::pnf() And from git/rcpp/inst/include/Rcpp/Rmath.h /* F Distibution */ inline double df(double x, double df1, double df2, int lg) { return ::Rf_df(x, df1, df2, lg); } /* Non-central F Distribution */ inline double dnf(double x, double df1, double df2, double ncp, int lg) { return ::Rf_dnf(x, df1, df2, ncp, lg); } Give that a whirl, and benchmark carefully against known values. Once you have, please do send me a short writeup for the Rcpp Gallery now that we have mathjax (yay!!) and graphs (yay!!) thanks to Jonathan. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From jtipton25 at gmail.com Fri Mar 6 23:03:36 2015 From: jtipton25 at gmail.com (John Tipton) Date: Fri, 6 Mar 2015 15:03:36 -0700 Subject: [Rcpp-devel] Linking to c++ files in a package without Rcpp overhead for faster computation Message-ID: Hey Rcpp developers, I am brand spanking new to the Rcpp community and I want to introduce myself with what is likely a very simple question that I haven't found an answer for yet. My question is on package development and linking c++ files from one package directly to another package without having to create a CCallable function . As a relatively experienced R programmer but novice c++ programmer, I am still learning as I go so I created a minimal example below that I believe better illustrates my question. I am building a package with RStudio and I want to make available to another package the .h header files - (more explicitly the c++ subroutines in those headers) without the loss of computation speed shown below. My question can best be summarized with two parts 1) Why when I directly source testPack.h is the function call so much slower than when I define the function in the same .cpp file (i.e. why is fun1.cpp so much slower (~40%) than fun2.cpp in test.R) 2) How do I build a library in an automated fashion (maybe there is a CPPFLAGS variable?) so I can use the functionality of something like #include ?path/to/file/hello_world2.h? in fun2.cpp without using an absolute path? I am building a large MCMC sampler so every bit of marginal speed gain is important. In other words, how can I replicate the behavior of fun2.cpp in a function that sources my library testPack I have the following files (outside my package) that provide testing. Thanks for any help. test.R fun1.cpp <- includes header, runs slow fun2.cpp <- includes ?hello_world2.h? header, runs faster I built a package skeleton in RStudio using Rcpp. Within the src directory I have the files hello_world2.h cpp_hello_world.cpp The files are as follows ## ## ## ???? begin test.R script ???? library(rbenchmark) n <- 10000 Rcpp::sourceCpp('~/test/fun1.cpp?) Rcpp::sourceCpp('~/test/fun2.cpp?) benchmark(fun1(n), fun2(n), replications = 500) ???? end test.R script ???? ## ## ## ???? begin fun1.cpp ???? #include #include // [[Rcpp::depends(testPack)]] // [[Rcpp::depends(RcppArmadillo)]] //using namespace Rcpp; //[[Rcpp::export]] void fun1(const int& n) { for(int i = 0; i < n; i++){ testPack::rcpp_hello_world(); } } ???? end fun1.cpp ???? ## ## ## ???? begin fun2.cpp ???? #include #include // [[Rcpp::depends(RcppArmadillo)]] //[[Rcpp::export]] void fun2(const int& n) { for(int i = 0; i < n; i++){ rcpp_hello_world2(); } } ???? end fun2.cpp ???? ## ## ## ???? begin hello_world2.h ???? Rcpp::List rcpp_hello_world2() { Rcpp::CharacterVector x = Rcpp::CharacterVector::create("foo", "bar"); Rcpp::NumericVector y = Rcpp::NumericVector::create(0.0, 1.0); Rcpp::List z = Rcpp::List::create(x, y); return z ; } ???? end hello_world2.h ???? ## ## ## ???? begin rcpp_hello_world.cpp ???? #include // [[Rcpp::depends(RcppArmadillo)]] // [[Rcpp::interfaces(cpp)]] using namespace Rcpp; //[[Rcpp::export]] List rcpp_hello_world() { CharacterVector x = CharacterVector::create( "foo", "bar" ) ; NumericVector y = NumericVector::create( 0.0, 1.0 ) ; List z = List::create( x, y ) ; return z ; } ???? end rcpp_hello_world.cpp ???? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jj.allaire at gmail.com Fri Mar 6 23:09:44 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Fri, 6 Mar 2015 17:09:44 -0500 Subject: [Rcpp-devel] Linking to c++ files in a package without Rcpp overhead for faster computation In-Reply-To: References: Message-ID: The overhead comes from the fact that the parameters are converted to/from SEXP and pushed through a C interface. There's also overhead that's associated with error checking, etc. This is a fixed overhead so as more work gets done inside your C++ function the % overhead will approach zero. In terms of re-using code across packages, everything in your inst/include directory is automatically added to the include search path of any package that has a LinkingTo relationship to yours (and to any sourceCpp module that does an Rcpp::depends on it). No absolute paths are therefore necessary (R resolves the path to your package at build time). If you want to re-use Rcpp code across packages the most straightforward way to do it is inside a header file (this has no linking or associated C interface overhead, and has no versioning problems). You can just define your function with the 'inline' keyword and it can be re-used in an arbitrary number of other C++ files with no linker conflicts. J.J. On Fri, Mar 6, 2015 at 5:03 PM, John Tipton wrote: > Hey Rcpp developers, > > I am brand spanking new to the Rcpp community and I want to introduce myself > with what is likely a very simple question that I haven't found an answer > for yet. My question is on package development and linking c++ files from > one package directly to another package without having to create a CCallable > function . As a relatively experienced R programmer but novice c++ > programmer, I am still learning as I go so I created a minimal example below > that I believe better illustrates my question. > > I am building a package with RStudio and I want to make available to another > package the .h header files - (more explicitly the c++ subroutines in those > headers) without the loss of computation speed shown below. > > My question can best be summarized with two parts > > 1) Why when I directly source testPack.h is the function call so much slower > than when I define the function in the same .cpp file (i.e. why is fun1.cpp > so much slower (~40%) than fun2.cpp in test.R) > > 2) How do I build a library in an automated fashion (maybe there is a > CPPFLAGS variable?) so I can use the functionality of something like > #include ?path/to/file/hello_world2.h? in fun2.cpp without using an absolute > path? I am building a large MCMC sampler so every bit of marginal speed gain > is important. In other words, how can I replicate the behavior of fun2.cpp > in a function that sources my library testPack > > > I have the following files (outside my package) that provide testing. Thanks > for any help. > > test.R > fun1.cpp <- includes header, runs slow > fun2.cpp <- includes ?hello_world2.h? header, runs faster > > I built a package skeleton in RStudio using Rcpp. Within the src directory I > have the files > hello_world2.h > cpp_hello_world.cpp > > > The files are as follows > ## > ## > ## > ???? begin test.R script ???? > > library(rbenchmark) > n <- 10000 > Rcpp::sourceCpp('~/test/fun1.cpp?) > Rcpp::sourceCpp('~/test/fun2.cpp?) > > benchmark(fun1(n), fun2(n), replications = 500) > > ???? end test.R script ???? > ## > ## > ## > ???? begin fun1.cpp ???? > > #include > #include > // [[Rcpp::depends(testPack)]] > // [[Rcpp::depends(RcppArmadillo)]] > > //using namespace Rcpp; > > //[[Rcpp::export]] > void fun1(const int& n) { > for(int i = 0; i < n; i++){ > testPack::rcpp_hello_world(); > } > } > > ???? end fun1.cpp ???? > ## > ## > ## > ???? begin fun2.cpp ???? > > #include > #include > > // [[Rcpp::depends(RcppArmadillo)]] > > //[[Rcpp::export]] > void fun2(const int& n) { > for(int i = 0; i < n; i++){ > rcpp_hello_world2(); > } > } > > ???? end fun2.cpp ???? > ## > ## > ## > ???? begin hello_world2.h ???? > > Rcpp::List rcpp_hello_world2() { > > Rcpp::CharacterVector x = Rcpp::CharacterVector::create("foo", "bar"); > Rcpp::NumericVector y = Rcpp::NumericVector::create(0.0, 1.0); > Rcpp::List z = Rcpp::List::create(x, y); > > return z ; > } > > ???? end hello_world2.h ???? > ## > ## > ## > ???? begin rcpp_hello_world.cpp ???? > > #include > > // [[Rcpp::depends(RcppArmadillo)]] > // [[Rcpp::interfaces(cpp)]] > > using namespace Rcpp; > > //[[Rcpp::export]] > List rcpp_hello_world() { > > CharacterVector x = CharacterVector::create( "foo", "bar" ) ; > NumericVector y = NumericVector::create( 0.0, 1.0 ) ; > List z = List::create( x, y ) ; > > return z ; > } > > ???? end rcpp_hello_world.cpp ???? > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From edd at debian.org Fri Mar 6 23:30:49 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Fri, 6 Mar 2015 16:30:49 -0600 Subject: [Rcpp-devel] Linking to c++ files in a package without Rcpp overhead for faster computation In-Reply-To: References: Message-ID: <21754.10905.29416.243889@max.nulle.part> On 6 March 2015 at 17:09, JJ Allaire wrote: | The overhead comes from the fact that the parameters are converted | to/from SEXP and pushed through a C interface. There's also overhead | that's associated with error checking, etc. This is a fixed overhead | so as more work gets done inside your C++ function the % overhead will | approach zero. | | In terms of re-using code across packages, everything in your | inst/include directory is automatically added to the include search | path of any package that has a LinkingTo relationship to yours (and to | any sourceCpp module that does an Rcpp::depends on it). No absolute | paths are therefore necessary (R resolves the path to your package at | build time). | | If you want to re-use Rcpp code across packages the most | straightforward way to do it is inside a header file (this has no | linking or associated C interface overhead, and has no versioning | problems). You can just define your function with the 'inline' keyword | and it can be re-used in an arbitrary number of other C++ files with | no linker conflicts. "What JJ said." Coincidentally, the same questin came up at StackOverflow yesterday, with a wrong initial Subject: asking for header-onlu and only later revealing that full linking was required. In bits and pieces I answerd something very similar to what JJ says here: http://stackoverflow.com/questions/28883314/rcpp-include-header-from-package-into-cppfunction Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From qkou at umail.iu.edu Sat Mar 7 01:06:49 2015 From: qkou at umail.iu.edu (Qiang Kou) Date: Fri, 6 Mar 2015 19:06:49 -0500 Subject: [Rcpp-devel] Linking to c++ files in a package without Rcpp overhead for faster computation In-Reply-To: References: Message-ID: Hi, John, Making your code header-only is your best choice and it will make everything easier. If header-only is impossible, this can happen, there is one trick you may want to try. As I remember the early version of Rcpp, maybe before 0.9, it builds a libRcpp.so when installing Rcpp. You can also do this by adding several lines in Makevars [1] and write helper functions to help system find the shared library you want to link to [2]. But I don't think this is good practice for Rcpp developing. Best wishes, KK [1] https://github.com/thirdwing/RcppMLPACK/blob/master/src/Makevars#L44-L61 [2] https://github.com/thirdwing/RcppMLPACK/blob/master/R/flags.R On Fri, Mar 6, 2015 at 5:03 PM, John Tipton wrote: > Hey Rcpp developers, > > I am brand spanking new to the Rcpp community and I want to introduce > myself with what is likely a very simple question that I haven't found an > answer for yet. My question is on package development and linking c++ files > from one package directly to another package without having to create a > CCallable function . As a relatively experienced R programmer but novice > c++ programmer, I am still learning as I go so I created a minimal example > below that I believe better illustrates my question. > > I am building a package with RStudio and I want to make available to > another package the .h header files - (more explicitly the c++ subroutines > in those headers) without the loss of computation speed shown below. > > My question can best be summarized with two parts > > 1) Why when I directly source testPack.h is the function call so much > slower than when I define the function in the same .cpp file (i.e. why is > fun1.cpp so much slower (~40%) than fun2.cpp in test.R) > > 2) How do I build a library in an automated fashion (maybe there is a > CPPFLAGS variable?) so I can use the functionality of something like > #include ?path/to/file/hello_world2.h? in fun2.cpp without using an > absolute path? I am building a large MCMC sampler so every bit of marginal > speed gain is important. In other words, how can I replicate the behavior > of fun2.cpp in a function that sources my library testPack > > > I have the following files (outside my package) that provide testing. > Thanks for any help. > > test.R > fun1.cpp <- includes header, runs slow > fun2.cpp <- includes ?hello_world2.h? header, runs faster > > I built a package skeleton in RStudio using Rcpp. Within the src directory > I have the files > hello_world2.h > cpp_hello_world.cpp > > > The files are as follows > ## > ## > ## > ???? begin test.R script ???? > > library(rbenchmark) > n <- 10000 > Rcpp::sourceCpp('~/test/fun1.cpp?) > Rcpp::sourceCpp('~/test/fun2.cpp?) > > benchmark(fun1(n), fun2(n), replications = 500) > > ???? end test.R script ???? > ## > ## > ## > ???? begin fun1.cpp ???? > > #include > #include > // [[Rcpp::depends(testPack)]] > // [[Rcpp::depends(RcppArmadillo)]] > > //using namespace Rcpp; > > //[[Rcpp::export]] > void fun1(const int& n) { > for(int i = 0; i < n; i++){ > testPack::rcpp_hello_world(); > } > } > > ???? end fun1.cpp ???? > ## > ## > ## > ???? begin fun2.cpp ???? > > #include > #include > > // [[Rcpp::depends(RcppArmadillo)]] > > //[[Rcpp::export]] > void fun2(const int& n) { > for(int i = 0; i < n; i++){ > rcpp_hello_world2(); > } > } > > ???? end fun2.cpp ???? > ## > ## > ## > ???? begin hello_world2.h ???? > > Rcpp::List rcpp_hello_world2() { > > Rcpp::CharacterVector x = Rcpp::CharacterVector::create("foo", "bar"); > Rcpp::NumericVector y = Rcpp::NumericVector::create(0.0, 1.0); > Rcpp::List z = Rcpp::List::create(x, y); > > return z ; > } > > ???? end hello_world2.h ???? > ## > ## > ## > ???? begin rcpp_hello_world.cpp ???? > > #include > > // [[Rcpp::depends(RcppArmadillo)]] > // [[Rcpp::interfaces(cpp)]] > > using namespace Rcpp; > > //[[Rcpp::export]] > List rcpp_hello_world() { > > CharacterVector x = CharacterVector::create( "foo", "bar" ) ; > NumericVector y = NumericVector::create( 0.0, 1.0 ) ; > List z = List::create( x, y ) ; > > return z ; > } > > ???? end rcpp_hello_world.cpp ???? > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > -- Qiang Kou qkou at umail.iu.edu School of Informatics and Computing, Indiana University -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack at jackwasey.com Tue Mar 10 02:40:16 2015 From: jack at jackwasey.com (Jack Wasey) Date: Mon, 09 Mar 2015 21:40:16 -0400 Subject: [Rcpp-devel] symbols "major" and "minor" under C++11 Message-ID: <54FE4B80.3030907@jackwasey.com> Dear All, I followed your interesting and detailed discussion about the problem in Rcpp 0.11.3 which resulted in compilation failure with "major", "minor" or "makedev" as function arguments due to leakage from system headers on Linux. https://github.com/RcppCore/Rcpp/issues/227 I am posting follow-up here because I'm not sure if this is a valid bug, but I am seeing the same problem when compiling with clang 3.7 and gcc 4.9.1 on 64 bit ubuntu 14.10 using C++11. Simply writing the function which demonstrated #227 isn't enough: cppFunction("int foo(int major) { return(2*major); }") does compile with the addition of -std=c++11 However, if I create an Rcpp.package.skeleton with attributes and the example code, and simply add function argument "CharacterVector major" to the hello world function: #include using namespace Rcpp; // [[Rcpp::export]] List rcpp_hello_world(CharacterVector major) { CharacterVector x = CharacterVector::create( "foo", "bar" ) ; NumericVector y = NumericVector::create( 0.0, 1.0 ) ; List z = List::create( x, y ) ; return z ; } then the RcppExports.cpp doesn't compile. Here is the clang compiler error: clang++ -I/usr/share/R/include -DNDEBUG -I"/usr/local/lib/R/site-library/Rcpp/include" -std=c++11 -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o RcppExports.cpp:15:44: error: use of undeclared identifier 'major' __result = Rcpp::wrap(rcpp_hello_world(major)); Here is the code from RcppExports.cpp: #include using namespace Rcpp; // rcpp_hello_world List rcpp_hello_world(CharacterVector major); RcppExport SEXP majmin11_rcpp_hello_world(SEXP majorSEXP) { BEGIN_RCPP Rcpp::RObject __result; Rcpp::RNGScope __rngScope; Rcpp::traits::input_parameter< CharacterVector >::type major(majorSEXP); __result = Rcpp::wrap(rcpp_hello_world(major)); return __result; END_RCPP } As it seems to arise from RcppExports, I hope you agree that using Rcpp.package.skeleton and making a small change constitutes an appropriate minimal example, since I couldn't reproduce with one single snippet. I may well be doing something wrong, but it does seem again like the system symbols leaking through, since renaming from "major" to "mjr" fixes the problem (as does removing C++11 compiler flag). I did look at the merged solution to #227, but honestly it's beyond me to attempt to dig any deeper. I realize this is quite a specific and exasperating problem. If you think it's a bug, I could comment on #227 or open a new issue. Thanks, Jack From edd at debian.org Tue Mar 10 02:52:33 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Mon, 9 Mar 2015 20:52:33 -0500 Subject: [Rcpp-devel] symbols "major" and "minor" under C++11 In-Reply-To: <54FE4B80.3030907@jackwasey.com> References: <54FE4B80.3030907@jackwasey.com> Message-ID: <21758.20065.963344.515925@max.nulle.part> Jack, Which Rcpp release did you use? We just released 0.11.5. Please try that if you haven't yet. This is "still not reproducible" as I would have to execute a number of steps you have not spilled out in an easy-to-follow and unamibguous way. (Your description was close though.) Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From edd at debian.org Tue Mar 10 02:57:02 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Mon, 9 Mar 2015 20:57:02 -0500 Subject: [Rcpp-devel] [ANN] Rcpp 0.11.5 Message-ID: <21758.20334.947364.270449@max.nulle.part> A new release Rcpp 0.11.5 arrived on CRAN on Friday -- my bad for forgetting to announce it then. A short blog post about it is at http://dirk.eddelbuettel.com/blog/ http://dirk.eddelbuettel.com/blog/2015/03/06#rcpp_0.11.5 and included as text (without links) below. This release represents contributions from a number of people--but JJ in particular is responsible for most of the changes. As always, thanks to everybody for testing, suggestions and input. On behalf of Rcpp Core, Dirk Rcpp 0.11.5 The new release 0.11.5 of Rcpp just reached the CRAN network for GNU R, and a Debian package has also been be uploaded. Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 345 packages on CRAN depend on Rcpp for making analyses go faster and further; BioConductor adds another 41 packages, and casual searches on GitHub suggests dozens mores. This release continues the 0.11.* release cycle, adding another large number of small bug fixes, polishes and enhancements. Since the previous release in January, we incorporated a number of pull requests and changes from several contributors. This time, JJ deserves a special mention as he is responsible for a metric ton of the changes listed below, making Rcpp Attributes even more awesome. As always, you can follow the development via the GitHub repo and particularly the Issue tickets and Pull Requests. And any discussions, questions, ... regarding Rcpp are always welcome at the rcpp-devel mailing list. See below for a detailed list of changes extracted from the NEWS file. Changes in Rcpp version 0.11.5 (2015-03-04) * Changes in Rcpp API: * An error handler for tinyformat was defined to prevent the assert() macro from spilling. * The Rcpp::warning function was added as a wrapper for Rf_warning. * The XPtr class was extended with new checked_get and release functions as well as improved behavior (throw an exception rather than crash) when a NULL external pointer is dereferenced. * R code is evaluated within an R_toplevelExec block to prevent user interrupts from bypassing C++ destructors on the stack. * The Rcpp::Environment constructor can now use a supplied parent environment. * The Rcpp::Function constructor can now use a supplied environment or namespace. * The attributes_hidden macro from R is used to shield internal functions; the R_ext/Visibility.h header is now included as well. * A Rcpp::print function was added as a wrapper around Rf_PrintValue. * Changes in Rcpp Attributes: * The pkg_types.h file is now included in RcppExports.cpp if it is present in either the inst/include or src. * sourceCpp was modified to allow includes of local files (e.g. #include "foo.hpp"). Implementation files (.cc; .cpp) corresponding to local includes are also automatically built if they exist. * The generated attributes code was simplified with respect to RNGScope and now uses RObject and its destructor rather than SEXP protect/unprotect. * Support addition of the rng parameter in Rcpp::export to suppress the otherwise automatic inclusion of RNGScope in generated code. * Attributes code was made more robust and can e.g. no longer recurse. * Version 3.2 of the Rtools is now correctly detected as well. * Allow 'R' to come immediately after '***' for defining embedded R code chunks in sourceCpp. * The attributes vignette has been updated with documentation on new features added over the past several releases. * Changes in Rcpp tests: * On Travis CI, all build dependencies are installed as binary .deb packages resulting in faster tests. Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings. /code/rcpp | permanent link -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From jack at jackwasey.com Tue Mar 10 03:02:58 2015 From: jack at jackwasey.com (Jack Wasey) Date: Mon, 09 Mar 2015 22:02:58 -0400 Subject: [Rcpp-devel] symbols "major" and "minor" under C++11 In-Reply-To: <21758.20065.963344.515925@max.nulle.part> References: <54FE4B80.3030907@jackwasey.com> <21758.20065.963344.515925@max.nulle.part> Message-ID: <54FE50D2.8030303@jackwasey.com> Dirk, I'm on Rcpp 0.11.5 Here are the steps: 1. in R: ``` library(Rcpp) Rcpp.package.skeleton(attributes = TRUE, example_code = TRUE, name="majortest") ``` 2. create majortest/src/Makevars containing: ``` PKG_CXXFLAGS = $(CXX1XSTD) ``` 3. Edit rcpp_hello_world.cpp replacing: ``` List rcpp_hello_world() { ``` with ``` List rcpp_hello_world(CharacterVector major) { ``` 4. both `devtools::load_all()` or `R CMD INSTALL majortest` then produce the previously described compilation errors, with my compiler set to gcc or clang. Hope that helps. Jack On 03/09/2015 09:52 PM, Dirk Eddelbuettel wrote: > > Jack, > > Which Rcpp release did you use? We just released 0.11.5. Please try that if > you haven't yet. > > This is "still not reproducible" as I would have to execute a number of steps > you have not spilled out in an easy-to-follow and unamibguous way. (Your > description was close though.) > > Dirk > From edd at debian.org Tue Mar 10 03:16:35 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Mon, 9 Mar 2015 21:16:35 -0500 Subject: [Rcpp-devel] symbols "major" and "minor" under C++11 In-Reply-To: <54FE50D2.8030303@jackwasey.com> References: <54FE4B80.3030907@jackwasey.com> <21758.20065.963344.515925@max.nulle.part> <54FE50D2.8030303@jackwasey.com> Message-ID: <21758.21507.554936.233712@max.nulle.part> Jack, On 9 March 2015 at 22:02, Jack Wasey wrote: | I'm on Rcpp 0.11.5 | | Here are the steps: | | 1. in R: | ``` | library(Rcpp) | Rcpp.package.skeleton(attributes = TRUE, example_code = TRUE, | name="majortest") | ``` | | 2. create majortest/src/Makevars containing: | | ``` | PKG_CXXFLAGS = $(CXX1XSTD) | ``` | | 3. Edit rcpp_hello_world.cpp replacing: | ``` | List rcpp_hello_world() { | ``` | with | ``` | List rcpp_hello_world(CharacterVector major) { | ``` | | 4. both `devtools::load_all()` or `R CMD INSTALL majortest` then produce | the previously described compilation errors, with my compiler set to gcc | or clang. Confirmed. (I'd add a step 3.5 to manually run compileAttributes() as not everybody uses devtools and/or RStudio which does that.) | Hope that helps. Yes, opening a new ticket. We had hope Kevin had squashed this. Sadly, we were wrong. Dirk | Jack | | On 03/09/2015 09:52 PM, Dirk Eddelbuettel wrote: | > | > Jack, | > | > Which Rcpp release did you use? We just released 0.11.5. Please try that if | > you haven't yet. | > | > This is "still not reproducible" as I would have to execute a number of steps | > you have not spilled out in an easy-to-follow and unamibguous way. (Your | > description was close though.) | > | > Dirk | > | -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From zjgslxh at 163.com Tue Mar 10 14:49:18 2015 From: zjgslxh at 163.com (lxh) Date: Tue, 10 Mar 2015 21:49:18 +0800 (CST) Subject: [Rcpp-devel] =?gbk?b?wLTX1Gx4aLXE08q8/g==?= Message-ID: <1aef304a.18ae8.14c03f25f44.Coremail.zjgslxh@163.com> Hi, ######################################### > Rcpp::sourceCpp('armaPRI.cpp') Error in inDL(x, as.logical(local), as.logical(now), ...) : unable to load shared object 'C:/Users/lxh/AppData/Local/Temp/RtmpeenRl8/sourcecpp_27081dce2d4b/sourceCpp_77404.dll': LoadLibrary failure: The specified module could not be found. I get this when trying to use RcppArmadillo with my princomp computation program(armaPRI.cpp). I am sure that i have missed something which results above error message, but i have no idea about that. By the way, the program eigentrans.cpp which uses the RcppEigen package works well. The Rtools path and the R bin path are already added to my Enviroment Path: ############################################################# c:\Rtools\gcc-4.6.3\bin;c:\Rtools\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\MiKTeX 2.9\miktex\bin\;C:\Python27;C:\Python27\DLLs;C:\Python27\Scripts;C:\Program Files\ATI Technologies\ATI.ACE\Core-Static;C:\TDM-GCC-32\bin;C:\R-3.1.3\bin;D:\Program Files\RStudio\bin\pandoc ########################################################################################### armaPRI.cpp ########################################################################################### // [[Rcpp::depends(RcppArmadillo)]] #include using namespace arma; using namespace Rcpp; // [[Rcpp::export]] int PRINCOMP(){ mat A = randu(5,4); mat coeff; mat score; vec latent; vec tsquared; princomp(coeff, score, latent, tsquared, A); Rcout<>eigentrans.cpp ##################################################################################################### // [[Rcpp::depends(RcppEigen)]] #include using namespace Rcpp; //[[Rcpp::export]] List trans(NumericMatrix& AA,NumericMatrix& BB) { const Eigen::MatrixXd A(as(AA)); const Eigen::MatrixXd B(as(BB)); Eigen::MatrixXd m(A.cwiseProduct(B)); Eigen::MatrixXd n(A.array().sin()); return List::create(Named("A")=A,Named("B")=B,Named("m")=m,Named("n")=n); } ################################################################################################## > sessionInfo() R version 3.1.3 (2015-03-09) Platform: i386-w64-mingw32/i386 (32-bit) Running under: Windows 7 (build 7601) Service Pack 1 locale: [1] LC_COLLATE=Chinese (Simplified)_People's Republic of China.936 [2] LC_CTYPE=Chinese (Simplified)_People's Republic of China.936 [3] LC_MONETARY=Chinese (Simplified)_People's Republic of China.936 [4] LC_NUMERIC=C [5] LC_TIME=Chinese (Simplified)_People's Republic of China.936 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] Rcpp_0.11.5 loaded via a namespace (and not attached): [1] RcppArmadillo_0.4.650.1.1 tools_3.1.3 thanksAt 2015-03-10 20:28:24, rcpp-devel-request at lists.r-forge.r-project.org wrote:>Welcome to the Rcpp-devel at lists.r-forge.r-project.org mailing list!>>To post to this list, send your message to:>> rcpp-devel at lists.r-forge.r-project.org>>General information about the mailing list is at:>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel>>>If you ever want to unsubscribe or change your options (eg, switch to>or from digest mode, change your password, etc.), visit your>subscription page at:>> https://lists.r-forge.r-project.org/cgi-bin/mailman/options/rcpp-devel/zjgslxh%40163.com>>>You can also make such adjustments via email by sending a message to:>> Rcpp-devel-request at lists.r-forge.r-project.org>>with the word `help' in the subject or body (don't include the>quotes), and you will get back a message with instructions.>>You must know your password to change your options (including changing>the password, itself) or to unsubscribe without confirmation. It is:>> mugua123>>Normally, Mailman will remind you of your lists.r-forge.r-project.org>mailing list passwords once every month, although you can disable this>if you prefer. This reminder will also include instructions on how to>unsubscribe or change your account options. There is also a button on>your options page that will email your current password to you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From balamut2 at illinois.edu Tue Mar 10 23:23:26 2015 From: balamut2 at illinois.edu (Balamuta, James Joseph) Date: Tue, 10 Mar 2015 22:23:26 +0000 Subject: [Rcpp-devel] Linking code in src/ subfolders Message-ID: <5C591F8C0073E544A1330AA751BA3E6913ADDDD8@chimbx4.ad.uillinois.edu> Greetings and Salutations All, I'm trying to add structure to the src folder since the amount of files I have residing in the src directory could be better organized. Note: These files are made using RcppArmadillo. E.g. I'm looking to go from: src/ |-> main.cpp |-> testA.cpp |-> testA.h |-> testB.cpp |-> testB.h To a structure like: src/ |-> main.cpp |-> A |-> testA.cpp |-> testA.h |-> B |-> testB.cpp |-> testB.h Each file has a function: main.cpp => testfunction(x) testA.cpp => testfunctionA(x) testB.cpp => testfunctionB(x) To illustrate the issue, I have two repositories SrcDir: https://github.com/coatless/header_cpp_subdir_code SubdirSrc: https://github.com/coatless/header_cpp_code If I compile without trying to change the folder structure, the package is made and is able to use without issue. However, when I attempt to change the folder structure, I was getting a compiler error that identifies with a linking issue: main.o:main.cpp:(.text+0x650a): undefined reference to `testfunctionA(arma::Col const&)' I looked at CRANs documentation for a suggested fix and ended up trying to ensure a link by using the makevars file (see: http://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars) I made the following changes: The Makevars file update seemed to work as the package then compiled. (Exact changes to Makevars.win: https://github.com/coatless/header_cpp_subdir_code/blob/master/src/Makevars.win ) Herein lies the issues: 1. Any files I've written in the subdirectories are no longer exported to RcppExport.cpp even though they have a // [[Rcpp::export]] tag preceding the function declaration. 2. When I try to call the function in main.cpp (testfunction(x)), I receive: testfunction(2^(1:5)) Error in .Call("SubdirSrc_testfunction", PACKAGE = "SubdirSrc", x) : "SubdirSrc_testfunction" not available for .Call() for package "SubdirSrc" Any suggestions would be appreciated. Sincerely, JJB -------------- next part -------------- An HTML attachment was scrubbed... URL: From xian at unm.edu Tue Mar 10 23:44:18 2015 From: xian at unm.edu (Christian Gunning) Date: Tue, 10 Mar 2015 18:44:18 -0400 Subject: [Rcpp-devel] ??lxh??? Message-ID: lxh, A quick word r.e. list etiquette -- you can help others process your request by including in your email a short & informative title, and a one or two sentence description of what you're trying to accomplish. I'm not on windows, so all I can say is that it works for me. Have you verified that RcppArmadillo installed without error? The other thing you can try is use the following to observe the details of the compilation process: sourceCpp('armaPRI.cpp', verbose=TRUE) hth, Christian > Date: Tue, 10 Mar 2015 21:49:18 +0800 (CST) > From: lxh > To: rcpp-devel at lists.r-forge.r-project.org > Subject: [Rcpp-devel] ??lxh??? > Message-ID: <1aef304a.18ae8.14c03f25f44.Coremail.zjgslxh at 163.com> > Content-Type: text/plain; charset="gbk" > > Hi, > > ######################################### >> Rcpp::sourceCpp('armaPRI.cpp') > > Error in inDL(x, as.logical(local), as.logical(now), ...) : > unable to load shared object 'C:/Users/lxh/AppData/Local/Temp/RtmpeenRl8/sourcecpp_27081dce2d4b/sourceCpp_77404.dll': > LoadLibrary failure: The specified module could not be found. > I get this when trying to use RcppArmadillo with my princomp computation program(armaPRI.cpp). I am sure that i have missed something which results above error message, but i have no idea about that. > By the way, the program eigentrans.cpp which uses the RcppEigen package works well. > > The Rtools path and the R bin path are already added to my Enviroment Path: > ############################################################# > c:\Rtools\gcc-4.6.3\bin;c:\Rtools\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\MiKTeX 2.9\miktex\bin\;C:\Python27;C:\Python27\DLLs;C:\Python27\Scripts;C:\Program Files\ATI Technologies\ATI.ACE\Core-Static;C:\TDM-GCC-32\bin;C:\R-3.1.3\bin;D:\Program Files\RStudio\bin\pandoc > ########################################################################################### > armaPRI.cpp > ########################################################################################### > // [[Rcpp::depends(RcppArmadillo)]] > #include > using namespace arma; > using namespace Rcpp; > // [[Rcpp::export]] > int PRINCOMP(){ > mat A = randu(5,4); > mat coeff; > mat score; > vec latent; > vec tsquared; > princomp(coeff, score, latent, tsquared, A); > Rcout< return 0; > } > ##################################################################################################### -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! From jj.allaire at gmail.com Wed Mar 11 00:06:07 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Tue, 10 Mar 2015 19:06:07 -0400 Subject: [Rcpp-devel] Linking code in src/ subfolders In-Reply-To: <5C591F8C0073E544A1330AA751BA3E6913ADDDD8@chimbx4.ad.uillinois.edu> References: <5C591F8C0073E544A1330AA751BA3E6913ADDDD8@chimbx4.ad.uillinois.edu> Message-ID: Right now compileAttributes doesn't scan code in subdirectories. This is mostly because for it to be really seamless compileAttributes should run before every build. We therefore want to keep the total execution time <= 50ms. Since some R packages have *a lot* of code in subdirectories of src we didn't want to get into scanning recursively before every build. On Tue, Mar 10, 2015 at 6:23 PM, Balamuta, James Joseph wrote: > Greetings and Salutations All, > > I?m trying to add structure to the src folder since the amount of files I > have residing in the src directory could be better organized. Note: These > files are made using RcppArmadillo. > > > > E.g. > > > > I?m looking to go from: > > src/ > |-> main.cpp > > |-> testA.cpp > > |-> testA.h > > |-> testB.cpp > > |-> testB.h > > > > To a structure like: > > > src/ > |-> main.cpp > |-> A > > |-> testA.cpp > > |-> testA.h > > |-> B > > |-> testB.cpp > > |-> testB.h > > > > Each file has a function: > > main.cpp => testfunction(x) > > testA.cpp => testfunctionA(x) > > testB.cpp => testfunctionB(x) > > > > To illustrate the issue, I have two repositories > > > > SrcDir: https://github.com/coatless/header_cpp_subdir_code > > SubdirSrc: https://github.com/coatless/header_cpp_code > > > > If I compile without trying to change the folder structure, the package is > made and is able to use without issue. > > > > However, when I attempt to change the folder structure, I was getting a > compiler error that identifies with a linking issue: > > > > main.o:main.cpp:(.text+0x650a): undefined reference to > `testfunctionA(arma::Col const&)' > > I looked at CRANs documentation for a suggested fix and ended up trying to > ensure a link by using the makevars file (see: > http://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars) > > > I made the following changes: > > > > The Makevars file update seemed to work as the package then compiled. > > (Exact changes to Makevars.win: > https://github.com/coatless/header_cpp_subdir_code/blob/master/src/Makevars.win > ) > > > > Herein lies the issues: > > > > 1. Any files I?ve written in the subdirectories are no longer > exported to RcppExport.cpp even though they have a // [[Rcpp::export]] tag > preceding the function declaration. > > 2. When I try to call the function in main.cpp (testfunction(x)), I > receive: > > testfunction(2^(1:5)) > > Error in .Call("SubdirSrc_testfunction", PACKAGE = "SubdirSrc", x) : > > "SubdirSrc_testfunction" not available for .Call() for package "SubdirSrc" > > > > > > Any suggestions would be appreciated. > > Sincerely, > > > JJB > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From edd at debian.org Wed Mar 11 01:14:18 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Tue, 10 Mar 2015 19:14:18 -0500 Subject: [Rcpp-devel] Linking code in src/ subfolders In-Reply-To: References: <5C591F8C0073E544A1330AA751BA3E6913ADDDD8@chimbx4.ad.uillinois.edu> Message-ID: <21759.35034.365588.639529@max.nulle.part> On 10 March 2015 at 19:06, JJ Allaire wrote: | Right now compileAttributes doesn't scan code in subdirectories. This | is mostly because for it to be really seamless compileAttributes | should run before every build. We therefore want to keep the total | execution time <= 50ms. | | Since some R packages have *a lot* of code in subdirectories of src we | didn't want to get into scanning recursively before every build. And the simple solution is that if you do have a lot of code in such subdirectories, maybe you should consider writing _explicit_ interfaces rather than relying on compileAttributes() Or, conversely, write accessors in the top-level directory and compileAttributes() will take care of you. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From jordi_molins at hotmail.com Wed Mar 11 12:58:14 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 12:58:14 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost Message-ID: Hello, I know this question has been answered before (and I have searched for it), but I cannot figure out a solution: I want to use boost with Rcpp, basically to be able to use numbers with a higher precision than a double. I am comfortable using the inline package for small c++ functions, I have never used Rccp per se. I have created a skeleton with convolve, and it works. I have added a new function: // We can now use the BH package// [[Rcpp::depends(BH)]] #include #include using namespace Rcpp; // [[Rcpp::export]]int computeGCD(int a, int b) { return boost::math::gcd(a, b);} and when I try to compile, it fails (basically, the compiler does not find where boost is). I have then run in R: Sys.setenv("PKG_CXXFLAGS"="-I C:/Program~Files/boost/boost_1_57_0/") which allows me to go a bit more into the build, but the compile fails when reaching the #include above, since it does not know where boost is (boost is at C:/Program~Files/boost/boost_1_57_0/ I have not compiled it, but just extracted with 7-zip and copied and pasted the corresponding files into this subfolder). >From reading the attribute document of Dirk Eddelbuettel, it seems as if adding the comments // [[Rcpp::export]] and // [[Rcpp::depends(BH)]] would do the job. But for me, it does not work. What do I need to do? Thanks and sorry for the basic question, I have tried to read everything I could before asking this list. -------------- next part -------------- An HTML attachment was scrubbed... URL: From deter088 at umn.edu Wed Mar 11 13:05:39 2015 From: deter088 at umn.edu (Charles Determan Jr) Date: Wed, 11 Mar 2015 07:05:39 -0500 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: Message-ID: I think you want to check your DESCRIPTION file. I believe you need to have the 'LinkingTo:' option to also include BH. Regards, Charles On Wed, Mar 11, 2015 at 6:58 AM, Jordi Molins Coronado < jordi_molins at hotmail.com> wrote: > Hello, > > I know this question has been answered before (and I have searched for > it), but I cannot figure out a solution: > > I want to use boost with Rcpp, basically to be able to use numbers with a > higher precision than a double. I am comfortable using the inline package > for small c++ functions, I have never used Rccp per se. I have created a > skeleton with convolve, and it works. I have added a new function: > > // We can now use the BH package > // [[Rcpp::depends(BH)]] > > #include > #include > > using namespace Rcpp; > > // [[Rcpp::export]] > int computeGCD(int a, int b) { > return boost::math::gcd(a, b); > } > > and when I try to compile, it fails (basically, the compiler does not find > where boost is). I have then run in R: > > Sys.setenv("PKG_CXXFLAGS"="-I C:/Program~Files/boost/boost_1_57_0/") > > which allows me to go a bit more into the build, but the compile fails > when reaching the #include above, since > it does not know where boost is (boost is at C:/Program~Files/boost/boost_1_57_0/ > I have not compiled it, but just extracted with 7-zip and copied and pasted > the corresponding files into this subfolder). > > From reading the attribute document of Dirk Eddelbuettel, it seems as if > adding the comments // [[Rcpp::export]] and // [[Rcpp::depends(BH)]] > would do the job. But for me, it does not work. What do I need to do? > Thanks and sorry for the basic question, I have tried to read everything I > could before asking this list. > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordi_molins at hotmail.com Wed Mar 11 13:09:47 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 13:09:47 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: , Message-ID: Thank you very much for your answer. I have changed the DESCRIPTION file to include: LinkingTo: Rcpp, BH (before: LinkingTo: Rcpp) but it continues giving the same build error. Date: Wed, 11 Mar 2015 07:05:39 -0500 Subject: Re: [Rcpp-devel] Question about Rcpp and boost From: deter088 at umn.edu To: jordi_molins at hotmail.com CC: rcpp-devel at lists.r-forge.r-project.org I think you want to check your DESCRIPTION file. I believe you need to have the 'LinkingTo:' option to also include BH. Regards,Charles On Wed, Mar 11, 2015 at 6:58 AM, Jordi Molins Coronado wrote: Hello, I know this question has been answered before (and I have searched for it), but I cannot figure out a solution: I want to use boost with Rcpp, basically to be able to use numbers with a higher precision than a double. I am comfortable using the inline package for small c++ functions, I have never used Rccp per se. I have created a skeleton with convolve, and it works. I have added a new function: // We can now use the BH package// [[Rcpp::depends(BH)]] #include #include using namespace Rcpp; // [[Rcpp::export]]int computeGCD(int a, int b) { return boost::math::gcd(a, b);} and when I try to compile, it fails (basically, the compiler does not find where boost is). I have then run in R: Sys.setenv("PKG_CXXFLAGS"="-I C:/Program~Files/boost/boost_1_57_0/") which allows me to go a bit more into the build, but the compile fails when reaching the #include above, since it does not know where boost is (boost is at C:/Program~Files/boost/boost_1_57_0/ I have not compiled it, but just extracted with 7-zip and copied and pasted the corresponding files into this subfolder). >From reading the attribute document of Dirk Eddelbuettel, it seems as if adding the comments // [[Rcpp::export]] and // [[Rcpp::depends(BH)]] would do the job. But for me, it does not work. What do I need to do? Thanks and sorry for the basic question, I have tried to read everything I could before asking this list. _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Wed Mar 11 13:12:16 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 11 Mar 2015 07:12:16 -0500 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: Message-ID: <21760.12576.664728.785700@max.nulle.part> On 11 March 2015 at 07:05, Charles Determan Jr wrote: | I think you want to check your DESCRIPTION file.? I believe you need to have | the 'LinkingTo:' option to also include BH. Beat me to it :) That is exactly it -- "LinkingTo: BH" is what you need (and our documentation says so in many places). Also look at eg the CRAN page for BH and the list of reverse dependencies and pick a (small) package also using Rcpp -- for example my RcppAnnoy. Download its source and study its package structure as this is _exactly_ the framework you want to copy. Use the copious examples out there to your advantage. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From deter088 at umn.edu Wed Mar 11 13:21:49 2015 From: deter088 at umn.edu (Charles Determan Jr) Date: Wed, 11 Mar 2015 07:21:49 -0500 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: Message-ID: Could you provide the build error you are referring to? You shouldn't need to change the PKG_CXXFLAGS variable at all. Try restarting your environment and rebuilding. I just created a package containing your code with the BH package without error on Windows 7. On Wed, Mar 11, 2015 at 7:09 AM, Jordi Molins Coronado < jordi_molins at hotmail.com> wrote: > Thank you very much for your answer. I have changed the DESCRIPTION file > to include: > > LinkingTo: Rcpp, BH > > (before: LinkingTo: Rcpp) but it continues giving the same build error. > > ------------------------------ > Date: Wed, 11 Mar 2015 07:05:39 -0500 > Subject: Re: [Rcpp-devel] Question about Rcpp and boost > From: deter088 at umn.edu > To: jordi_molins at hotmail.com > CC: rcpp-devel at lists.r-forge.r-project.org > > > I think you want to check your DESCRIPTION file. I believe you need to > have the 'LinkingTo:' option to also include BH. > > Regards, > Charles > > On Wed, Mar 11, 2015 at 6:58 AM, Jordi Molins Coronado < > jordi_molins at hotmail.com> wrote: > > Hello, > > I know this question has been answered before (and I have searched for > it), but I cannot figure out a solution: > > I want to use boost with Rcpp, basically to be able to use numbers with a > higher precision than a double. I am comfortable using the inline package > for small c++ functions, I have never used Rccp per se. I have created a > skeleton with convolve, and it works. I have added a new function: > > // We can now use the BH package > // [[Rcpp::depends(BH)]] > > #include > #include > > using namespace Rcpp; > > // [[Rcpp::export]] > int computeGCD(int a, int b) { > return boost::math::gcd(a, b); > } > > and when I try to compile, it fails (basically, the compiler does not find > where boost is). I have then run in R: > > Sys.setenv("PKG_CXXFLAGS"="-I C:/Program~Files/boost/boost_1_57_0/") > > which allows me to go a bit more into the build, but the compile fails > when reaching the #include above, since > it does not know where boost is (boost is at C:/Program~Files/boost/boost_1_57_0/ > I have not compiled it, but just extracted with 7-zip and copied and pasted > the corresponding files into this subfolder). > > From reading the attribute document of Dirk Eddelbuettel, it seems as if > adding the comments // [[Rcpp::export]] and // [[Rcpp::depends(BH)]] > would do the job. But for me, it does not work. What do I need to do? > Thanks and sorry for the basic question, I have tried to read everything I > could before asking this list. > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > > > -- Dr. Charles Determan, PhD Integrated Biosciences -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordi_molins at hotmail.com Wed Mar 11 13:26:33 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 13:26:33 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: , , , Message-ID: I have restarted everything, and now I can already compute GCD using the boost library, thank you for your help. Just an annoying thing: when I do: sourceCpp("convolve.cpp") I get a warning, which I did not get the first time I built my project (without the boost file). Warning message:In normalizePath(path.expand(path), winslash, mustWork) : path[1]="C:/Users/Jordi/Documents/[A_FOLDER]/JungleR/JungleR/src/../inst/include": The system cannot find the path specified Do you know what can that be? I will definitely look into the examples in BH library, thanks for the recommendation. Date: Wed, 11 Mar 2015 07:21:49 -0500 Subject: Re: [Rcpp-devel] Question about Rcpp and boost From: deter088 at umn.edu To: jordi_molins at hotmail.com CC: rcpp-devel at lists.r-forge.r-project.org Could you provide the build error you are referring to? You shouldn't need to change the PKG_CXXFLAGS variable at all. Try restarting your environment and rebuilding. I just created a package containing your code with the BH package without error on Windows 7. On Wed, Mar 11, 2015 at 7:09 AM, Jordi Molins Coronado wrote: Thank you very much for your answer. I have changed the DESCRIPTION file to include: LinkingTo: Rcpp, BH (before: LinkingTo: Rcpp) but it continues giving the same build error. Date: Wed, 11 Mar 2015 07:05:39 -0500 Subject: Re: [Rcpp-devel] Question about Rcpp and boost From: deter088 at umn.edu To: jordi_molins at hotmail.com CC: rcpp-devel at lists.r-forge.r-project.org I think you want to check your DESCRIPTION file. I believe you need to have the 'LinkingTo:' option to also include BH. Regards,Charles On Wed, Mar 11, 2015 at 6:58 AM, Jordi Molins Coronado wrote: Hello, I know this question has been answered before (and I have searched for it), but I cannot figure out a solution: I want to use boost with Rcpp, basically to be able to use numbers with a higher precision than a double. I am comfortable using the inline package for small c++ functions, I have never used Rccp per se. I have created a skeleton with convolve, and it works. I have added a new function: // We can now use the BH package// [[Rcpp::depends(BH)]] #include #include using namespace Rcpp; // [[Rcpp::export]]int computeGCD(int a, int b) { return boost::math::gcd(a, b);} and when I try to compile, it fails (basically, the compiler does not find where boost is). I have then run in R: Sys.setenv("PKG_CXXFLAGS"="-I C:/Program~Files/boost/boost_1_57_0/") which allows me to go a bit more into the build, but the compile fails when reaching the #include above, since it does not know where boost is (boost is at C:/Program~Files/boost/boost_1_57_0/ I have not compiled it, but just extracted with 7-zip and copied and pasted the corresponding files into this subfolder). >From reading the attribute document of Dirk Eddelbuettel, it seems as if adding the comments // [[Rcpp::export]] and // [[Rcpp::depends(BH)]] would do the job. But for me, it does not work. What do I need to do? Thanks and sorry for the basic question, I have tried to read everything I could before asking this list. _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Dr. Charles Determan, PhD Integrated Biosciences -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Wed Mar 11 13:36:30 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 11 Mar 2015 07:36:30 -0500 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: Message-ID: <21760.14030.879453.332358@max.nulle.part> Jordi, 1) Please edit your replies. There is no need to return all the prior emails. 2) Please read the vignette 'Rcpp Attributes'. The vignette has all the answers. In short you REALLY do not want sourceCpp in a package. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From jordi_molins at hotmail.com Wed Mar 11 13:45:22 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 13:45:22 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: <21760.14030.879453.332358@max.nulle.part> References: , , , , , <21760.14030.879453.332358@max.nulle.part> Message-ID: Dirk, sorry for the newbie mistakes. Now I know I want to use compileAttributes() :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordi_molins at hotmail.com Wed Mar 11 19:16:59 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 19:16:59 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: , , , , , <21760.14030.879453.332358@max.nulle.part>, Message-ID: I have an odd problem: now I have the computeGCD function working. My next step has been to create another function doing something trivial, but adding the include for multiprecision of integers: // We can now use the BH package// [[Rcpp::depends(BH)]] #include #include #include using namespace Rcpp; // [[Rcpp::export]]int precision(int num) { int fact = 1; for(int i=num; i>1; --i) { fact *= i; } return fact*boost::math::gcd(8, 16); The build does not work, and the error message is: precisionboost.cpp:5:44: fatal error: boost/multiprecision/cpp_int.hpp: No such file or directorycompilation terminated. This is odd, since if I delete the corresponding include for multiprecision and I leave the one of math, the compilation ends without problems. So, it cannot be a LinkingTo issue. Of course, the file cpp_int.hpp is there. I have tried to find references to the use of boost multiprecision within Rcpp, but I have not found any. Any idea of what is going on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordi_molins at hotmail.com Wed Mar 11 19:34:13 2015 From: jordi_molins at hotmail.com (Jordi Molins Coronado) Date: Wed, 11 Mar 2015 19:34:13 +0100 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: , , , , , <21760.14030.879453.332358@max.nulle.part>, , Message-ID: I think I can answer my own question: #include is not in the BH package. Instead, #include is included, so if I add this #include in my file, the compile works. But apparently, I cannot include cpp_int.cpp. -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Wed Mar 11 19:56:51 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 11 Mar 2015 13:56:51 -0500 Subject: [Rcpp-devel] Question about Rcpp and boost In-Reply-To: References: <21760.14030.879453.332358@max.nulle.part> Message-ID: <21760.36851.437979.343378@max.nulle.part> On 11 March 2015 at 19:34, Jordi Molins Coronado wrote: | I think I can answer my own question: #include is not in the BH package. Instead, #include is included, so if I add this #include in my file, the | compile works. But apparently, I cannot include cpp_int.cpp. Correct, it is not (yet) in BH. But I just filed an issue for BH to include multiprecision next time I update the package. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From xian at unm.edu Wed Mar 11 23:25:41 2015 From: xian at unm.edu (Christian Gunning) Date: Wed, 11 Mar 2015 18:25:41 -0400 Subject: [Rcpp-devel] R.e. A RcppArmadillo based program cannot compile under Windows and RStudio Message-ID: No problem. One more piece of list etiquette: when writing follow-up emails, please CC the list in your response to individuals who have directly answered you. This prevents questions from being (privately) answered repeatedly. Two questions: * Can you confirm that you do, in fact, have RcppArmadillo installed by a call to library(RcppArmadillo)? * You've looked over the Rcpp FAQ carefully? Compilation on windows is, er, different. http://dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf I don't have a windows machine, so I'm not able to reproduce your problem. Perhaps someone else can use the detailed information that you've provided to identify the problem with your included example. Best, Christian On Wed, Mar 11, 2015 at 9:38 AM, ?????? wrote: > I am sorry for that not include a short & informative title in my email.? > > A RcppArmadillo based program cannot compile under Windows and RStudio ? > > In RStudio > ###### >> Rcpp::sourceCpp('armaLU.cpp',verbose = TRUE) > > Generated extern "C" functions > -------------------------------------------------------- > > > #include > // LU > int LU(); > RcppExport SEXP sourceCpp_75578_LU() { > BEGIN_RCPP > Rcpp::RObject __result; > Rcpp::RNGScope __rngScope; > __result = Rcpp::wrap(LU()); > return __result; > END_RCPP > } > > Generated R functions > ------------------------------------------------------- > > `.sourceCpp_75578_DLLInfo` <- > dyn.load('C:/Users/lxh/AppData/Local/Temp/RtmporLzPl/sourcecpp_19a8798d5d84/sourceCpp_59453.dll') > > LU <- Rcpp:::sourceCppFunction(function() {}, FALSE, > `.sourceCpp_75578_DLLInfo`, 'sourceCpp_75578_LU') > > rm(`.sourceCpp_75578_DLLInfo`) > > Building shared library > -------------------------------------------------------- > > DIR: C:/Users/lxh/AppData/Local/Temp/RtmporLzPl/sourcecpp_19a8798d5d84 > > C:/R-31~1.3/bin/i386/R CMD SHLIB -o "sourceCpp_59453.dll" "" "armaLU.cpp" > g++ -m32 -I"C:/R-31~1.3/include" -DNDEBUG > -I"C:/R-3.1.3/library/Rcpp/include" > -I"C:/R-3.1.3/library/RcppArmadillo/include" -I"D:/Rstudio" > -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O2 -Wall > -mtune=core2 -c armaLU.cpp -o armaLU.o > g++ -m32 -shared -s -static-libgcc -o sourceCpp_59453.dll tmp.def armaLU.o > -LC:/R-31~1.3/bin/i386 -lRlapack -LC:/R-31~1.3/bin/i386 -lRblas -lgfortran > -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 > -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/R-31~1.3/bin/i386 -lR > Error in inDL(x, as.logical(local), as.logical(now), ...) : > unable to load shared object > 'C:/Users/lxh/AppData/Local/Temp/RtmporLzPl/sourcecpp_19a8798d5d84/sourceCpp_59453.dll': > LoadLibrary failure: The specified module could not be found??? > > ######################################### > But the default Rcpp template can compile with Rcpp without problem , the > RcppEigen program can compile without any error also.? > > > ------------------ ???? ------------------ > ???: "Christian Gunning";; > ????: 2015?3?11?(???) ??6:44 > ???: > "rcpp-devel at lists.r-forge.r-project.org"; > ??: [Rcpp-devel] ??lxh??? > > lxh, > > A quick word r.e. list etiquette -- you can help others process your > request by including in your email a short & informative title, and a > one or two sentence description of what you're trying to accomplish. > > I'm not on windows, so all I can say is that it works for me. Have you > verified that RcppArmadillo installed without error? > > The other thing you can try is use the following to observe the > details of the compilation process: > > sourceCpp('armaPRI.cpp', verbose=TRUE) > > hth, > Christian > >> Date: Tue, 10 Mar 2015 21:49:18 +0800 (CST) >> From: lxh >> To: rcpp-devel at lists.r-forge.r-project.org >> Subject: [Rcpp-devel] ??lxh??? >> Message-ID: <1aef304a.18ae8.14c03f25f44.Coremail.zjgslxh at 163.com> >> Content-Type: text/plain; charset="gbk" >> >> Hi, >> >> ######################################### >>> Rcpp::sourceCpp('armaPRI.cpp') >> >> Error in inDL(x, as.logical(local), as.logical(now), ...) : >> unable to load shared object >> 'C:/Users/lxh/AppData/Local/Temp/RtmpeenRl8/sourcecpp_27081dce2d4b/sourceCpp_77404.dll': >> LoadLibrary failure: The specified module could not be found. >> I get this when trying to use RcppArmadillo with my princomp computation >> program(armaPRI.cpp). I am sure that i have missed something which results >> above error message, but i have no idea about that. >> By the way, the program eigentrans.cpp which uses the RcppEigen package >> works well. >> >> The Rtools path and the R bin path are already added to my Enviroment >> Path: >> ############################################################# >> >> c:\Rtools\gcc-4.6.3\bin;c:\Rtools\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\MiKTeX >> 2.9\miktex\bin\;C:\Python27;C:\Python27\DLLs;C:\Python27\Scripts;C:\Program >> Files\ATI >> Technologies\ATI.ACE\Core-Static;C:\TDM-GCC-32\bin;C:\R-3.1.3\bin;D:\Program >> Files\RStudio\bin\pandoc >> >> ########################################################################################### >> armaPRI.cpp >> >> ########################################################################################### >> // [[Rcpp::depends(RcppArmadillo)]] >> #include >> using namespace arma; >> using namespace Rcpp; >> // [[Rcpp::export]] >> int PRINCOMP(){ >> mat A = randu(5,4); >> mat coeff; >> mat score; >> vec latent; >> vec tsquared; >> princomp(coeff, score, latent, tsquared, A); >> Rcout<> return 0; >> } >> >> ##################################################################################################### > > -- > A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! From henrik.singmann at psychologie.uni-freiburg.de Thu Mar 12 20:37:07 2015 From: henrik.singmann at psychologie.uni-freiburg.de (Henrik Singmann) Date: Thu, 12 Mar 2015 20:37:07 +0100 Subject: [Rcpp-devel] R CMD check fails on latest RcppEigen using R-devel on Windows Message-ID: <5501EAE3.7040303@psychologie.uni-freiburg.de> Dear all, In the process of preparing to submit an updated version of a package of mine to CRAN, the R CMD check on my package always failed (i.e., the R instance crashed) when running the 32bit examples involving RcppEigen. On further investigating, it seems that this is already true for the version of my package on CRAN (http://cran.r-project.org/web/checks/check_results_MPTinR.html), but also for other packages involving RcppEigen such as lme4 (http://cran.r-project.org/web/checks/check_results_lme4.html) and even RcppEigen itself (http://cran.r-project.org/web/checks/check_results_RcppEigen.html) I can confirm the error shown at the CRAN check results for RcppEigen on my own machines (Win 7 64 bit and Win 8.1, both 64 bit versions) when downloading the RcppEigen source and running R CMD check using R-devel on it (see below for logs and sessionInfo). As I said, the problem only occurs on the 32 bit versions (i386) and kills the R process (with uninformative Windows error message). Can someone else using Windows confirm this behavior on the latest R-devel? Any ideas on what to do? Or should I perhaps send this to R-devel instead of here as it seems to be also related to R-devel? Btw, I am using the latest R tools and R-devel, both downloaded today. Cheers, Henrik ####### The log gives: ** running tests for arch 'i386' ... ERROR Running the tests in 'tests/doRUnit.R' failed. Last 13 lines of output: Executing test function test.as.SpMat ... done successfully. Executing test function test.as.Vec ... done successfully. Executing test function test.wrap.R ... done successfully. Executing test function test.fastLm ... ####### doRUnit.Rout.fail R Under development (unstable) (2015-03-11 r67980) -- "Unsuffered Consequences" Copyright (C) 2015 The R Foundation for Statistical Computing Platform: i386-w64-mingw32/i386 (32-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > #### doRUnit.R --- Run RUnit tests > ####------------------------------------------------------------------------ > > ### borrowed from package fUtilities in RMetrics > ### http://r-forge.r-project.org/plugins/scmsvn/viewcvs.php/pkg/fUtilities/tests/doRUnit.R?rev=1958&root=rmetrics&view=markup > > ### Originally follows Gregor Gojanc's example in CRAN package 'gdata' > ### and the corresponding section in the R Wiki: > ### http://wiki.r-project.org/rwiki/doku.php?id=developers:runit > > ### MM: Vastly changed: This should also be "runnable" for *installed* > ## package which has no ./tests/ > ## ----> put the bulk of the code e.g. in ../inst/unitTests/runTests.R : > > if(require("RUnit", quietly = TRUE)) { + pkg <- "RcppEigen" + + require( pkg, character.only=TRUE) + + path <- system.file("unitTests", package = pkg) + + stopifnot(file.exists(path), file.info(path.expand(path))$isdir) + + ## without this, we get unit test failures + Sys.setenv( R_TESTS = "" ) + + RcppEigen.unit.test.output.dir <- getwd() + + source(file.path(path, "runTests.R"), echo = TRUE) + + } else { + print( "package RUnit not available, cannot run unit tests" ) + } Loading required package: RcppEigen > pkg <- "RcppEigen" > if (!require("inline", character.only = TRUE, quietly = TRUE)) { + stop("The inline package is required to run RcppEigen unit tests") + } > if (compareVersion(packageDescription("inline")[["Version"]], + "0.3.5") < 0) { + stop("RcppEigen unit tests need at least the version 0.3. ..." ... [TRUNCATED] > if (require("RUnit", quietly = TRUE)) { + is_local <- function() { + if (exists("argv", globalenv()) && "--local" %in% argv) + .... [TRUNCATED] Executing test function test.as.MMat ... done successfully. Executing test function test.as.MSpMat ... done successfully. Executing test function test.as.MVec ... done successfully. Executing test function test.as.SpMat ... done successfully. Executing test function test.as.Vec ... done successfully. Executing test function test.wrap.R ... done successfully. Executing test function test.fastLm ... ###### a session info of my machine: > sessionInfo() R Under development (unstable) (2015-03-11 r67980) Platform: i386-w64-mingw32/i386 (32-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1 locale: [1] LC_COLLATE=German_Switzerland.1252 LC_CTYPE=German_Switzerland.1252 LC_MONETARY=German_Switzerland.1252 [4] LC_NUMERIC=C LC_TIME=German_Switzerland.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] RcppEigen_0.3.2.4.0 devtools_1.6.1 loaded via a namespace (and not attached): [1] Matrix_1.1-5 tools_3.2.0 Rcpp_0.11.5 grid_3.2.0 lattice_0.20-30 fortunes_1.5-2 -- Dr. Henrik Singmann Universit?t Z?rich, Schweiz http://singmann.org From edd at debian.org Thu Mar 12 21:04:54 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Thu, 12 Mar 2015 15:04:54 -0500 Subject: [Rcpp-devel] R CMD check fails on latest RcppEigen using R-devel on Windows In-Reply-To: <5501EAE3.7040303@psychologie.uni-freiburg.de> References: <5501EAE3.7040303@psychologie.uni-freiburg.de> Message-ID: <21761.61798.787081.545464@max.nulle.part> We heard -- see https://github.com/RcppCore/Rcpp/issues/276 It is at this point unclear if it us, or Rtools but we are aware and this will get addressed. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From edd at debian.org Thu Mar 12 21:16:16 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Thu, 12 Mar 2015 15:16:16 -0500 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: <5501E4CA.3060907@gmail.com> References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> Message-ID: <21761.62480.686704.551425@max.nulle.part> Duncan, The preferred and widely-documented address for Rcpp issues is rcpp-devel where I am forwarding this, I would appreciate keeping follow-up there. On 12 March 2015 at 15:11, Duncan Murdoch wrote: | Jack (and Dirk): | | I see this as well when I recompile Rcpp. | | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit | Windows with a program calling Rf_error from Rcpp, but not from a simple | C program (attached) that looks equivalent. This is using the new | toolchain, both to compile Rcpp and Jack's cppFunction() call below. | | I'd be happy to help with debugging this, but not for a few days: I'll | be out of the office until Wednesday next week, and I don't have 64 bit | Windows when I'm on the road. Sure. We can pick this up when you are back. The change vector appears to be on your side of the fence so we need access from your end. Few of us (on the Rcpp core group) use Windows all that much. I think I saw a blog post mentioned where someone said that with g++ 4.9.2 (though possibly a different build/configuration/...) some exception-related behaviour was improved. I am sure that something is different now, and we will try to accomodate it. Safe travels. Dirk | | Duncan Murdoch | | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: | > On 12/03/2015 1:31 PM, Jack Wasey wrote: | > > Dear Duncan, | > > | > > I hope you don't mind me emailing you directly, rather than through | > > r-devel, since it seemed a very specific problem. I had just sent an | > > email to the list when it crossed with yours saying you had released a | > > new Rtools, so I pulled my r-devel email. I have a probable R, | > > possible Rcpp bug causing a hang with Rf_error("stop") after showing | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a few | > > hours ago). | > > | > > I'm afraid my reproducible example uses Rcpp, but only for | > > compilation. I'm not adept enough to eliminate the Rcpp step quickly, | > > but I hope it will be of use anyway. | > > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") | > > doError() | > > | > > This causes a hang after showing the error message. Winbuilder checked | > > my package "icd9" without problems in 32 bit R-devel, and both 32 and | > > 64 bit R current. | > > | > > Not being sure whether this is an Rcpp or R bug (see also | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to report | > > this as a bug against R itself, but perhaps people more skilled than | > > me can do this if indeed it is so. | > | > I see the same problem. I've just tried the equivalent as a simple C | > program, and it was fine. I will try compiling Rcpp and see if that | > fixes it; I wouldn't be surprised if the binary on CRAN was built with | > the old compiler. | > | > Duncan Murdoch | > > | > > Best wishes, | > > Jack | > > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch | > > wrote: | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, adding the | > > > cygpath.exe utility. That utility converts between Windows style paths like | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It may be useful | > > > in configuration files if your external library expects to find gcc on the | > > > path, since Rtools no longer puts it there. Assuming you want to use the | > > > Rtools toolchain, you can construct the path to the gcc directory in your | > > > Makevars.win file as | > > > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin | > > > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that should | > > > already have been read.) | > > > | > > > Thanks to JJ Allaire for the prompting on this. | > > > | > > > Duncan Murdoch | > > > | > > > ______________________________________________ | > > > R-devel at r-project.org mailing list | > > > https://stat.ethz.ch/mailman/listinfo/r-devel | > | | [DELETED ATTACHMENT test.c, plain text] -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From jj.allaire at gmail.com Thu Mar 12 21:24:05 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Thu, 12 Mar 2015 16:24:05 -0400 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: <21761.62480.686704.551425@max.nulle.part> References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: Just a note that Rf_error is actually not technically allowed in Rcpp (it's a longjmp which bypasses all C++ destructors on the stack). We do it as follows: Rcpp::stop("error message") Which throws an exception which is ultimately caught by our wrapper macro (which then calls Rf_error in a context where there are no more destructors). On Thu, Mar 12, 2015 at 4:16 PM, Dirk Eddelbuettel wrote: > > Duncan, > > The preferred and widely-documented address for Rcpp issues is rcpp-devel > where I am forwarding this, I would appreciate keeping follow-up there. > > On 12 March 2015 at 15:11, Duncan Murdoch wrote: > | Jack (and Dirk): > | > | I see this as well when I recompile Rcpp. > | > | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit > | Windows with a program calling Rf_error from Rcpp, but not from a simple > | C program (attached) that looks equivalent. This is using the new > | toolchain, both to compile Rcpp and Jack's cppFunction() call below. > | > | I'd be happy to help with debugging this, but not for a few days: I'll > | be out of the office until Wednesday next week, and I don't have 64 bit > | Windows when I'm on the road. > > Sure. We can pick this up when you are back. The change vector appears to > be on your side of the fence so we need access from your end. Few of us (on > the Rcpp core group) use Windows all that much. > > I think I saw a blog post mentioned where someone said that with g++ 4.9.2 > (though possibly a different build/configuration/...) some exception-related > behaviour was improved. I am sure that something is different now, and we > will try to accomodate it. > > Safe travels. > > Dirk > > | > | Duncan Murdoch > | > | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: > | > On 12/03/2015 1:31 PM, Jack Wasey wrote: > | > > Dear Duncan, > | > > > | > > I hope you don't mind me emailing you directly, rather than through > | > > r-devel, since it seemed a very specific problem. I had just sent an > | > > email to the list when it crossed with yours saying you had released a > | > > new Rtools, so I pulled my r-devel email. I have a probable R, > | > > possible Rcpp bug causing a hang with Rf_error("stop") after showing > | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a few > | > > hours ago). > | > > > | > > I'm afraid my reproducible example uses Rcpp, but only for > | > > compilation. I'm not adept enough to eliminate the Rcpp step quickly, > | > > but I hope it will be of use anyway. > | > > > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") > | > > doError() > | > > > | > > This causes a hang after showing the error message. Winbuilder checked > | > > my package "icd9" without problems in 32 bit R-devel, and both 32 and > | > > 64 bit R current. > | > > > | > > Not being sure whether this is an Rcpp or R bug (see also > | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to report > | > > this as a bug against R itself, but perhaps people more skilled than > | > > me can do this if indeed it is so. > | > > | > I see the same problem. I've just tried the equivalent as a simple C > | > program, and it was fine. I will try compiling Rcpp and see if that > | > fixes it; I wouldn't be surprised if the binary on CRAN was built with > | > the old compiler. > | > > | > Duncan Murdoch > | > > > | > > Best wishes, > | > > Jack > | > > > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch > | > > wrote: > | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, adding the > | > > > cygpath.exe utility. That utility converts between Windows style paths like > | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It may be useful > | > > > in configuration files if your external library expects to find gcc on the > | > > > path, since Rtools no longer puts it there. Assuming you want to use the > | > > > Rtools toolchain, you can construct the path to the gcc directory in your > | > > > Makevars.win file as > | > > > > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin > | > > > > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that should > | > > > already have been read.) > | > > > > | > > > Thanks to JJ Allaire for the prompting on this. > | > > > > | > > > Duncan Murdoch > | > > > > | > > > ______________________________________________ > | > > > R-devel at r-project.org mailing list > | > > > https://stat.ethz.ch/mailman/listinfo/r-devel > | > > | > | [DELETED ATTACHMENT test.c, plain text] > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From avraham.adler at gmail.com Thu Mar 12 21:25:18 2015 From: avraham.adler at gmail.com (Avraham Adler) Date: Thu, 12 Mar 2015 16:25:18 -0400 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: I believe the new toolchain uses SJLJ and not SEH specifically for backwards compatibility. Avi On Thu, Mar 12, 2015 at 4:24 PM, JJ Allaire wrote: > Just a note that Rf_error is actually not technically allowed in Rcpp > (it's a longjmp which bypasses all C++ destructors on the stack). We > do it as follows: > > Rcpp::stop("error message") > > Which throws an exception which is ultimately caught by our wrapper > macro (which then calls Rf_error in a context where there are no more > destructors). > > > > On Thu, Mar 12, 2015 at 4:16 PM, Dirk Eddelbuettel wrote: > > > > Duncan, > > > > The preferred and widely-documented address for Rcpp issues is rcpp-devel > > where I am forwarding this, I would appreciate keeping follow-up there. > > > > On 12 March 2015 at 15:11, Duncan Murdoch wrote: > > | Jack (and Dirk): > > | > > | I see this as well when I recompile Rcpp. > > | > > | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit > > | Windows with a program calling Rf_error from Rcpp, but not from a > simple > > | C program (attached) that looks equivalent. This is using the new > > | toolchain, both to compile Rcpp and Jack's cppFunction() call below. > > | > > | I'd be happy to help with debugging this, but not for a few days: I'll > > | be out of the office until Wednesday next week, and I don't have 64 bit > > | Windows when I'm on the road. > > > > Sure. We can pick this up when you are back. The change vector appears > to > > be on your side of the fence so we need access from your end. Few of us > (on > > the Rcpp core group) use Windows all that much. > > > > I think I saw a blog post mentioned where someone said that with g++ > 4.9.2 > > (though possibly a different build/configuration/...) some > exception-related > > behaviour was improved. I am sure that something is different now, and > we > > will try to accomodate it. > > > > Safe travels. > > > > Dirk > > > > | > > | Duncan Murdoch > > | > > | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: > > | > On 12/03/2015 1:31 PM, Jack Wasey wrote: > > | > > Dear Duncan, > > | > > > > | > > I hope you don't mind me emailing you directly, rather than through > > | > > r-devel, since it seemed a very specific problem. I had just sent > an > > | > > email to the list when it crossed with yours saying you had > released a > > | > > new Rtools, so I pulled my r-devel email. I have a probable R, > > | > > possible Rcpp bug causing a hang with Rf_error("stop") after > showing > > | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a > few > > | > > hours ago). > > | > > > > | > > I'm afraid my reproducible example uses Rcpp, but only for > > | > > compilation. I'm not adept enough to eliminate the Rcpp step > quickly, > > | > > but I hope it will be of use anyway. > > | > > > > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") > > | > > doError() > > | > > > > | > > This causes a hang after showing the error message. Winbuilder > checked > > | > > my package "icd9" without problems in 32 bit R-devel, and both 32 > and > > | > > 64 bit R current. > > | > > > > | > > Not being sure whether this is an Rcpp or R bug (see also > > | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to > report > > | > > this as a bug against R itself, but perhaps people more skilled > than > > | > > me can do this if indeed it is so. > > | > > > | > I see the same problem. I've just tried the equivalent as a simple C > > | > program, and it was fine. I will try compiling Rcpp and see if that > > | > fixes it; I wouldn't be surprised if the binary on CRAN was built > with > > | > the old compiler. > > | > > > | > Duncan Murdoch > > | > > > > | > > Best wishes, > > | > > Jack > > | > > > > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch > > | > > wrote: > > | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, > adding the > > | > > > cygpath.exe utility. That utility converts between Windows > style paths like > > | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It > may be useful > > | > > > in configuration files if your external library expects to find > gcc on the > > | > > > path, since Rtools no longer puts it there. Assuming you want > to use the > > | > > > Rtools toolchain, you can construct the path to the gcc > directory in your > > | > > > Makevars.win file as > > | > > > > > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin > > | > > > > > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that > should > > | > > > already have been read.) > > | > > > > > | > > > Thanks to JJ Allaire for the prompting on this. > > | > > > > > | > > > Duncan Murdoch > > | > > > > > | > > > ______________________________________________ > > | > > > R-devel at r-project.org mailing list > > | > > > https://stat.ethz.ch/mailman/listinfo/r-devel > > | > > > | > > | [DELETED ATTACHMENT test.c, plain text] > > > > -- > > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > > _______________________________________________ > > Rcpp-devel mailing list > > Rcpp-devel at lists.r-forge.r-project.org > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jj.allaire at gmail.com Thu Mar 12 21:39:48 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Thu, 12 Mar 2015 16:39:48 -0400 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: Just noticed that Rcpp::stop also causes a hang on 64-bit R so the longjmp in the wrapper macro is having the same effect (good to be clear that the destructors being bypassed wasn't the source of the hang/crash) On Thu, Mar 12, 2015 at 4:25 PM, Avraham Adler wrote: > I believe the new toolchain uses SJLJ and not SEH specifically for backwards > compatibility. > > Avi > > On Thu, Mar 12, 2015 at 4:24 PM, JJ Allaire wrote: >> >> Just a note that Rf_error is actually not technically allowed in Rcpp >> (it's a longjmp which bypasses all C++ destructors on the stack). We >> do it as follows: >> >> Rcpp::stop("error message") >> >> Which throws an exception which is ultimately caught by our wrapper >> macro (which then calls Rf_error in a context where there are no more >> destructors). >> >> >> >> On Thu, Mar 12, 2015 at 4:16 PM, Dirk Eddelbuettel wrote: >> > >> > Duncan, >> > >> > The preferred and widely-documented address for Rcpp issues is >> > rcpp-devel >> > where I am forwarding this, I would appreciate keeping follow-up there. >> > >> > On 12 March 2015 at 15:11, Duncan Murdoch wrote: >> > | Jack (and Dirk): >> > | >> > | I see this as well when I recompile Rcpp. >> > | >> > | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit >> > | Windows with a program calling Rf_error from Rcpp, but not from a >> > simple >> > | C program (attached) that looks equivalent. This is using the new >> > | toolchain, both to compile Rcpp and Jack's cppFunction() call below. >> > | >> > | I'd be happy to help with debugging this, but not for a few days: I'll >> > | be out of the office until Wednesday next week, and I don't have 64 >> > bit >> > | Windows when I'm on the road. >> > >> > Sure. We can pick this up when you are back. The change vector appears >> > to >> > be on your side of the fence so we need access from your end. Few of us >> > (on >> > the Rcpp core group) use Windows all that much. >> > >> > I think I saw a blog post mentioned where someone said that with g++ >> > 4.9.2 >> > (though possibly a different build/configuration/...) some >> > exception-related >> > behaviour was improved. I am sure that something is different now, and >> > we >> > will try to accomodate it. >> > >> > Safe travels. >> > >> > Dirk >> > >> > | >> > | Duncan Murdoch >> > | >> > | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: >> > | > On 12/03/2015 1:31 PM, Jack Wasey wrote: >> > | > > Dear Duncan, >> > | > > >> > | > > I hope you don't mind me emailing you directly, rather than >> > through >> > | > > r-devel, since it seemed a very specific problem. I had just sent >> > an >> > | > > email to the list when it crossed with yours saying you had >> > released a >> > | > > new Rtools, so I pulled my r-devel email. I have a probable R, >> > | > > possible Rcpp bug causing a hang with Rf_error("stop") after >> > showing >> > | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a >> > few >> > | > > hours ago). >> > | > > >> > | > > I'm afraid my reproducible example uses Rcpp, but only for >> > | > > compilation. I'm not adept enough to eliminate the Rcpp step >> > quickly, >> > | > > but I hope it will be of use anyway. >> > | > > >> > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") >> > | > > doError() >> > | > > >> > | > > This causes a hang after showing the error message. Winbuilder >> > checked >> > | > > my package "icd9" without problems in 32 bit R-devel, and both 32 >> > and >> > | > > 64 bit R current. >> > | > > >> > | > > Not being sure whether this is an Rcpp or R bug (see also >> > | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to >> > report >> > | > > this as a bug against R itself, but perhaps people more skilled >> > than >> > | > > me can do this if indeed it is so. >> > | > >> > | > I see the same problem. I've just tried the equivalent as a simple >> > C >> > | > program, and it was fine. I will try compiling Rcpp and see if that >> > | > fixes it; I wouldn't be surprised if the binary on CRAN was built >> > with >> > | > the old compiler. >> > | > >> > | > Duncan Murdoch >> > | > > >> > | > > Best wishes, >> > | > > Jack >> > | > > >> > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch >> > | > > wrote: >> > | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, >> > adding the >> > | > > > cygpath.exe utility. That utility converts between Windows >> > style paths like >> > | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It >> > may be useful >> > | > > > in configuration files if your external library expects to find >> > gcc on the >> > | > > > path, since Rtools no longer puts it there. Assuming you want >> > to use the >> > | > > > Rtools toolchain, you can construct the path to the gcc >> > directory in your >> > | > > > Makevars.win file as >> > | > > > >> > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin >> > | > > > >> > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that >> > should >> > | > > > already have been read.) >> > | > > > >> > | > > > Thanks to JJ Allaire for the prompting on this. >> > | > > > >> > | > > > Duncan Murdoch >> > | > > > >> > | > > > ______________________________________________ >> > | > > > R-devel at r-project.org mailing list >> > | > > > https://stat.ethz.ch/mailman/listinfo/r-devel >> > | > >> > | >> > | [DELETED ATTACHMENT test.c, plain text] >> > >> > -- >> > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org >> > _______________________________________________ >> > Rcpp-devel mailing list >> > Rcpp-devel at lists.r-forge.r-project.org >> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > From jj.allaire at gmail.com Thu Mar 12 21:51:46 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Thu, 12 Mar 2015 16:51:46 -0400 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: This is a minimum case to reproduce the crash (no Rcpp involved *however* a C++ destructor being executed seems to be what triggers the crash): #include #include #include extern "C" SEXP foo(SEXP bar) { std::string str; Rf_error("foobar"); return bar; } If I remove the "std::string str" line then it doesn't crash. So it's something related to the interaction between longjmp and C++ destructors... On Thu, Mar 12, 2015 at 4:39 PM, JJ Allaire wrote: > Just noticed that Rcpp::stop also causes a hang on 64-bit R so the > longjmp in the wrapper macro is having the same effect (good to be > clear that the destructors being bypassed wasn't the source of the > hang/crash) > > > On Thu, Mar 12, 2015 at 4:25 PM, Avraham Adler wrote: >> I believe the new toolchain uses SJLJ and not SEH specifically for backwards >> compatibility. >> >> Avi >> >> On Thu, Mar 12, 2015 at 4:24 PM, JJ Allaire wrote: >>> >>> Just a note that Rf_error is actually not technically allowed in Rcpp >>> (it's a longjmp which bypasses all C++ destructors on the stack). We >>> do it as follows: >>> >>> Rcpp::stop("error message") >>> >>> Which throws an exception which is ultimately caught by our wrapper >>> macro (which then calls Rf_error in a context where there are no more >>> destructors). >>> >>> >>> >>> On Thu, Mar 12, 2015 at 4:16 PM, Dirk Eddelbuettel wrote: >>> > >>> > Duncan, >>> > >>> > The preferred and widely-documented address for Rcpp issues is >>> > rcpp-devel >>> > where I am forwarding this, I would appreciate keeping follow-up there. >>> > >>> > On 12 March 2015 at 15:11, Duncan Murdoch wrote: >>> > | Jack (and Dirk): >>> > | >>> > | I see this as well when I recompile Rcpp. >>> > | >>> > | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit >>> > | Windows with a program calling Rf_error from Rcpp, but not from a >>> > simple >>> > | C program (attached) that looks equivalent. This is using the new >>> > | toolchain, both to compile Rcpp and Jack's cppFunction() call below. >>> > | >>> > | I'd be happy to help with debugging this, but not for a few days: I'll >>> > | be out of the office until Wednesday next week, and I don't have 64 >>> > bit >>> > | Windows when I'm on the road. >>> > >>> > Sure. We can pick this up when you are back. The change vector appears >>> > to >>> > be on your side of the fence so we need access from your end. Few of us >>> > (on >>> > the Rcpp core group) use Windows all that much. >>> > >>> > I think I saw a blog post mentioned where someone said that with g++ >>> > 4.9.2 >>> > (though possibly a different build/configuration/...) some >>> > exception-related >>> > behaviour was improved. I am sure that something is different now, and >>> > we >>> > will try to accomodate it. >>> > >>> > Safe travels. >>> > >>> > Dirk >>> > >>> > | >>> > | Duncan Murdoch >>> > | >>> > | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: >>> > | > On 12/03/2015 1:31 PM, Jack Wasey wrote: >>> > | > > Dear Duncan, >>> > | > > >>> > | > > I hope you don't mind me emailing you directly, rather than >>> > through >>> > | > > r-devel, since it seemed a very specific problem. I had just sent >>> > an >>> > | > > email to the list when it crossed with yours saying you had >>> > released a >>> > | > > new Rtools, so I pulled my r-devel email. I have a probable R, >>> > | > > possible Rcpp bug causing a hang with Rf_error("stop") after >>> > showing >>> > | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a >>> > few >>> > | > > hours ago). >>> > | > > >>> > | > > I'm afraid my reproducible example uses Rcpp, but only for >>> > | > > compilation. I'm not adept enough to eliminate the Rcpp step >>> > quickly, >>> > | > > but I hope it will be of use anyway. >>> > | > > >>> > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") >>> > | > > doError() >>> > | > > >>> > | > > This causes a hang after showing the error message. Winbuilder >>> > checked >>> > | > > my package "icd9" without problems in 32 bit R-devel, and both 32 >>> > and >>> > | > > 64 bit R current. >>> > | > > >>> > | > > Not being sure whether this is an Rcpp or R bug (see also >>> > | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to >>> > report >>> > | > > this as a bug against R itself, but perhaps people more skilled >>> > than >>> > | > > me can do this if indeed it is so. >>> > | > >>> > | > I see the same problem. I've just tried the equivalent as a simple >>> > C >>> > | > program, and it was fine. I will try compiling Rcpp and see if that >>> > | > fixes it; I wouldn't be surprised if the binary on CRAN was built >>> > with >>> > | > the old compiler. >>> > | > >>> > | > Duncan Murdoch >>> > | > > >>> > | > > Best wishes, >>> > | > > Jack >>> > | > > >>> > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch >>> > | > > wrote: >>> > | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, >>> > adding the >>> > | > > > cygpath.exe utility. That utility converts between Windows >>> > style paths like >>> > | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It >>> > may be useful >>> > | > > > in configuration files if your external library expects to find >>> > gcc on the >>> > | > > > path, since Rtools no longer puts it there. Assuming you want >>> > to use the >>> > | > > > Rtools toolchain, you can construct the path to the gcc >>> > directory in your >>> > | > > > Makevars.win file as >>> > | > > > >>> > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin >>> > | > > > >>> > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that >>> > should >>> > | > > > already have been read.) >>> > | > > > >>> > | > > > Thanks to JJ Allaire for the prompting on this. >>> > | > > > >>> > | > > > Duncan Murdoch >>> > | > > > >>> > | > > > ______________________________________________ >>> > | > > > R-devel at r-project.org mailing list >>> > | > > > https://stat.ethz.ch/mailman/listinfo/r-devel >>> > | > >>> > | >>> > | [DELETED ATTACHMENT test.c, plain text] >>> > >>> > -- >>> > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org >>> > _______________________________________________ >>> > Rcpp-devel mailing list >>> > Rcpp-devel at lists.r-forge.r-project.org >>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >>> _______________________________________________ >>> Rcpp-devel mailing list >>> Rcpp-devel at lists.r-forge.r-project.org >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> >> From jj.allaire at gmail.com Thu Mar 12 21:59:24 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Thu, 12 Mar 2015 16:59:24 -0400 Subject: [Rcpp-devel] Rcpp bug (was: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)) In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: This also does not crash (because the destructor has already run before the longjmp): #include #include #include extern "C" SEXP foo(SEXP bar) { { std::string str; } Rf_error("foobar"); return bar; } Kevin and I are investigating further now. On Thu, Mar 12, 2015 at 4:51 PM, JJ Allaire wrote: > This is a minimum case to reproduce the crash (no Rcpp involved > *however* a C++ destructor being executed seems to be what triggers > the crash): > > #include > #include > > #include > > extern "C" SEXP foo(SEXP bar) { > > std::string str; > > Rf_error("foobar"); > return bar; > } > > If I remove the "std::string str" line then it doesn't crash. > > So it's something related to the interaction between longjmp and C++ > destructors... > > On Thu, Mar 12, 2015 at 4:39 PM, JJ Allaire wrote: >> Just noticed that Rcpp::stop also causes a hang on 64-bit R so the >> longjmp in the wrapper macro is having the same effect (good to be >> clear that the destructors being bypassed wasn't the source of the >> hang/crash) >> >> >> On Thu, Mar 12, 2015 at 4:25 PM, Avraham Adler wrote: >>> I believe the new toolchain uses SJLJ and not SEH specifically for backwards >>> compatibility. >>> >>> Avi >>> >>> On Thu, Mar 12, 2015 at 4:24 PM, JJ Allaire wrote: >>>> >>>> Just a note that Rf_error is actually not technically allowed in Rcpp >>>> (it's a longjmp which bypasses all C++ destructors on the stack). We >>>> do it as follows: >>>> >>>> Rcpp::stop("error message") >>>> >>>> Which throws an exception which is ultimately caught by our wrapper >>>> macro (which then calls Rf_error in a context where there are no more >>>> destructors). >>>> >>>> >>>> >>>> On Thu, Mar 12, 2015 at 4:16 PM, Dirk Eddelbuettel wrote: >>>> > >>>> > Duncan, >>>> > >>>> > The preferred and widely-documented address for Rcpp issues is >>>> > rcpp-devel >>>> > where I am forwarding this, I would appreciate keeping follow-up there. >>>> > >>>> > On 12 March 2015 at 15:11, Duncan Murdoch wrote: >>>> > | Jack (and Dirk): >>>> > | >>>> > | I see this as well when I recompile Rcpp. >>>> > | >>>> > | Dirk: We're seeing the Rgui console on Windows lock up in 64 bit >>>> > | Windows with a program calling Rf_error from Rcpp, but not from a >>>> > simple >>>> > | C program (attached) that looks equivalent. This is using the new >>>> > | toolchain, both to compile Rcpp and Jack's cppFunction() call below. >>>> > | >>>> > | I'd be happy to help with debugging this, but not for a few days: I'll >>>> > | be out of the office until Wednesday next week, and I don't have 64 >>>> > bit >>>> > | Windows when I'm on the road. >>>> > >>>> > Sure. We can pick this up when you are back. The change vector appears >>>> > to >>>> > be on your side of the fence so we need access from your end. Few of us >>>> > (on >>>> > the Rcpp core group) use Windows all that much. >>>> > >>>> > I think I saw a blog post mentioned where someone said that with g++ >>>> > 4.9.2 >>>> > (though possibly a different build/configuration/...) some >>>> > exception-related >>>> > behaviour was improved. I am sure that something is different now, and >>>> > we >>>> > will try to accomodate it. >>>> > >>>> > Safe travels. >>>> > >>>> > Dirk >>>> > >>>> > | >>>> > | Duncan Murdoch >>>> > | >>>> > | On 12/03/2015 2:54 PM, Duncan Murdoch wrote: >>>> > | > On 12/03/2015 1:31 PM, Jack Wasey wrote: >>>> > | > > Dear Duncan, >>>> > | > > >>>> > | > > I hope you don't mind me emailing you directly, rather than >>>> > through >>>> > | > > r-devel, since it seemed a very specific problem. I had just sent >>>> > an >>>> > | > > email to the list when it crossed with yours saying you had >>>> > released a >>>> > | > > new Rtools, so I pulled my r-devel email. I have a probable R, >>>> > | > > possible Rcpp bug causing a hang with Rf_error("stop") after >>>> > showing >>>> > | > > the error message, but only with 64 bit Rtools 3.3 (downloaded a >>>> > few >>>> > | > > hours ago). >>>> > | > > >>>> > | > > I'm afraid my reproducible example uses Rcpp, but only for >>>> > | > > compilation. I'm not adept enough to eliminate the Rcpp step >>>> > quickly, >>>> > | > > but I hope it will be of use anyway. >>>> > | > > >>>> > | > > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") >>>> > | > > doError() >>>> > | > > >>>> > | > > This causes a hang after showing the error message. Winbuilder >>>> > checked >>>> > | > > my package "icd9" without problems in 32 bit R-devel, and both 32 >>>> > and >>>> > | > > 64 bit R current. >>>> > | > > >>>> > | > > Not being sure whether this is an Rcpp or R bug (see also >>>> > | > > https://github.com/RcppCore/Rcpp/issues/276), I hesitated to >>>> > report >>>> > | > > this as a bug against R itself, but perhaps people more skilled >>>> > than >>>> > | > > me can do this if indeed it is so. >>>> > | > >>>> > | > I see the same problem. I've just tried the equivalent as a simple >>>> > C >>>> > | > program, and it was fine. I will try compiling Rcpp and see if that >>>> > | > fixes it; I wouldn't be surprised if the binary on CRAN was built >>>> > with >>>> > | > the old compiler. >>>> > | > >>>> > | > Duncan Murdoch >>>> > | > > >>>> > | > > Best wishes, >>>> > | > > Jack >>>> > | > > >>>> > | > > On Thu, Mar 12, 2015 at 1:17 PM, Duncan Murdoch >>>> > | > > wrote: >>>> > | > > > I've just uploaded a minor update (3.3.0.1957) to Rtools33, >>>> > adding the >>>> > | > > > cygpath.exe utility. That utility converts between Windows >>>> > style paths like >>>> > | > > > D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools. It >>>> > may be useful >>>> > | > > > in configuration files if your external library expects to find >>>> > gcc on the >>>> > | > > > path, since Rtools no longer puts it there. Assuming you want >>>> > to use the >>>> > | > > > Rtools toolchain, you can construct the path to the gcc >>>> > directory in your >>>> > | > > > Makevars.win file as >>>> > | > > > >>>> > | > > > $(cygpath $(RTOOLS))gcc492_$(WIN)/bin >>>> > | > > > >>>> > | > > > (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that >>>> > should >>>> > | > > > already have been read.) >>>> > | > > > >>>> > | > > > Thanks to JJ Allaire for the prompting on this. >>>> > | > > > >>>> > | > > > Duncan Murdoch >>>> > | > > > >>>> > | > > > ______________________________________________ >>>> > | > > > R-devel at r-project.org mailing list >>>> > | > > > https://stat.ethz.ch/mailman/listinfo/r-devel >>>> > | > >>>> > | >>>> > | [DELETED ATTACHMENT test.c, plain text] >>>> > >>>> > -- >>>> > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org >>>> > _______________________________________________ >>>> > Rcpp-devel mailing list >>>> > Rcpp-devel at lists.r-forge.r-project.org >>>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >>>> _______________________________________________ >>>> Rcpp-devel mailing list >>>> Rcpp-devel at lists.r-forge.r-project.org >>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >>> >>> From jeroen.ooms at stat.ucla.edu Thu Mar 12 22:25:28 2015 From: jeroen.ooms at stat.ucla.edu (Jeroen Ooms) Date: Thu, 12 Mar 2015 14:25:28 -0700 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools Message-ID: This is probably not exclusively Rcpp problem, but I could use some debugging help from Rcpp experts. When building V8 on windows with the new tool chain, the build succeeds and the package can be loaded, but when trying to use it on i386, it crashes (it works fine on x64). Other packages that link to external c++ libraries (e.g. RQuantlib) might suffer from this as well. To reproduce, install latest rtools and r-devel on windows and run: install.packages("V8", type = "source") library(V8) ct <- new_context() The R.exe process crashes without any error message or crashdump, which makes this very difficult to debug. I suspect that the problem is that the new Rtools has been built with some different settings resulting in incompatible binaries, but I can't find the problem. Any suggestions on where to start debugging this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jj.allaire at gmail.com Thu Mar 12 22:40:57 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Thu, 12 Mar 2015 17:40:57 -0400 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools In-Reply-To: References: Message-ID: I had an issue like this in RcppParallel that was solved by rebuilding Rcpp from source using the very latest R-devel and Rtools 3.3. Have you tried that? J.J. On Thu, Mar 12, 2015 at 5:25 PM, Jeroen Ooms wrote: > This is probably not exclusively Rcpp problem, but I could use some > debugging help from Rcpp experts. > > When building V8 on windows with the new tool chain, the build succeeds and > the package can be loaded, but when trying to use it on i386, it crashes (it > works fine on x64). Other packages that link to external c++ libraries (e.g. > RQuantlib) might suffer from this as well. > > To reproduce, install latest rtools and r-devel on windows and run: > > install.packages("V8", type = "source") > library(V8) > ct <- new_context() > > The R.exe process crashes without any error message or crashdump, which > makes this very difficult to debug. I suspect that the problem is that the > new Rtools has been built with some different settings resulting in > incompatible binaries, but I can't find the problem. > > Any suggestions on where to start debugging this? > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From jeroen.ooms at stat.ucla.edu Thu Mar 12 22:51:41 2015 From: jeroen.ooms at stat.ucla.edu (Jeroen Ooms) Date: Thu, 12 Mar 2015 14:51:41 -0700 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools In-Reply-To: References: Message-ID: On Thu, Mar 12, 2015 at 2:40 PM, JJ Allaire wrote: > I had an issue like this in RcppParallel that was solved by rebuilding > Rcpp from source using the very latest R-devel and Rtools 3.3. Have > you tried that? > Yes, that doesn't seem to help. I have a suspicion now. I have seen this behavior before when linking libraries on windows that were built with different threading models. Based on gcc -v, the old rtools (and hence my v8 static libs) use win32 threading, whereas the new rtools has been built with --enable-threads=posix. -------------- next part -------------- An HTML attachment was scrubbed... URL: From murdoch.duncan at gmail.com Thu Mar 12 23:49:21 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Thu, 12 Mar 2015 18:49:21 -0400 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools In-Reply-To: References: Message-ID: <550217F1.1040900@gmail.com> On 12/03/2015 5:51 PM, Jeroen Ooms wrote: > On Thu, Mar 12, 2015 at 2:40 PM, JJ Allaire > wrote: > > I had an issue like this in RcppParallel that was solved by rebuilding > Rcpp from source using the very latest R-devel and Rtools 3.3. Have > you tried that? > > > Yes, that doesn't seem to help. > > I have a suspicion now. I have seen this behavior before when linking > libraries on windows that were built with different threading models. > Based on gcc -v, the old rtools (and hence my v8 static libs) use win32 > threading, whereas the new rtools has been built with > --enable-threads=posix. The scripts are on CRAN, if you want to try modifying them. However, if the scripts aren't satisfactory very soon, I'll just go back to the previous Rtools. Duncan From avraham.adler at gmail.com Fri Mar 13 00:13:18 2015 From: avraham.adler at gmail.com (Avraham Adler) Date: Thu, 12 Mar 2015 19:13:18 -0400 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools In-Reply-To: <550217F1.1040900@gmail.com> References: <550217F1.1040900@gmail.com> Message-ID: Why would posix be enabled for a Windows target? Avi On Thu, Mar 12, 2015 at 6:49 PM, Duncan Murdoch wrote: > On 12/03/2015 5:51 PM, Jeroen Ooms wrote: > > On Thu, Mar 12, 2015 at 2:40 PM, JJ Allaire > > wrote: > > > > I had an issue like this in RcppParallel that was solved by > rebuilding > > Rcpp from source using the very latest R-devel and Rtools 3.3. Have > > you tried that? > > > > > > Yes, that doesn't seem to help. > > > > I have a suspicion now. I have seen this behavior before when linking > > libraries on windows that were built with different threading models. > > Based on gcc -v, the old rtools (and hence my v8 static libs) use win32 > > threading, whereas the new rtools has been built with > > --enable-threads=posix. > > The scripts are on CRAN, if you want to try modifying them. However, if > the scripts aren't satisfactory very soon, I'll just go back to the > previous Rtools. > > Duncan > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From murdoch.duncan at gmail.com Fri Mar 13 00:52:34 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Thu, 12 Mar 2015 19:52:34 -0400 Subject: [Rcpp-devel] New Windows toolchain In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> Message-ID: <550226C2.10304@gmail.com> I've told some of you privately that I'm basically not going to be working on this again before next Wednesday, due to travel. The 3.2.0 release process starts on Monday, and I will be travelling for all but two weeks before the release, so there is very little time left for changes. So here's what I plan to do: By the end of next week I will decide on what toolchain to include in Rtools. This will be based largely on what I hear from this group, because the main advantage of the new toolchain is C++ support. I think there are 2 likely choices, with a third possible. We could use the Rtools33 toolchain that is currently on CRAN. This appears likely to require changes by C++ users. Alternatively, we could revert to the previous toolchain, based on gcc 4.6.3. If we do revert, we will be able to change for R 3.2.1, which is unscheduled, but likely to appear in June, judging by past history. The 3rd possibility is that someone other than me makes small changes to the toolchain (the scripts are on CRAN in bin/windows/Rtools/scripts), and enough people test the resulting chain so we have general agreement that the modification works. It needs to be someone other than me, because I need to try it out in my own tests, and that means I need to have it in hand by Wednesday, if I'm going to decide by Friday. I will not build this into Rtools before next Friday, so you'll need to distribute this by some other means. Duncan Murdoch From kevinushey at gmail.com Fri Mar 13 03:42:49 2015 From: kevinushey at gmail.com (Kevin Ushey) Date: Thu, 12 Mar 2015 19:42:49 -0700 Subject: [Rcpp-devel] New Windows toolchain In-Reply-To: <550226C2.10304@gmail.com> References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> <550226C2.10304@gmail.com> Message-ID: Hi Duncan, First off -- a _huge_ thanks for all the time and effort you've put into updating the toolchain on Windows. We understand that, effectively, your (and R-core)'s stake in updating the toolchain on Windows is quite a bit smaller than ours, since C++ support is not a governing priority in the maintenance and development of R. Those of us on the C++ / Rcpp side are very excited at the prospect of being able to develop R packages which leverage C++11 features and also work across the board on all the platforms used by R, and so the prospect of the toolchain being updated on Windows is very exciting. At the moment, our hope is that a combination of patches to Rcpp, and potentially a few small patches to the Rtools scripts (which we are happy to provide and test ourselves), will show that the updated toolchain is stable enough for use with R 3.2.0. A summary of the issue for anyone else on the list who hasn't followed the other threads follows: --- Many Rcpp client packages / C++-containing packages erroneously make calls directly to `Rf_error()`. Calling `Rf_error()` directly from C++ can result in undefined behaviour -- this is because calling `Rf_error()` will result in a `longjmp`, which bypasses the destructors of any objects on the stack (thereby potentially leaking memory, but in general, causing undefined behaviour). The simplest reproducible example (without Rcpp) demonstrating this problem, from JJ, is this: #include #include #include extern "C" SEXP ouch() { std::string str; Rf_error("ouch"); return R_NilValue; } When calling `ouch()`, the object `str` is effectively leaked, and its destructor is never called, due to the longjmp invoked from `Rf_error()`. With the older Rtools toolchain, packages generally 'got away' with doing this -- perhaps certain objects were leaked, or destructors weren't run, but often this wasn't a show-stopper. This is no longer the case with the new toolchain -- here, our 'luck runs out' and we instead get stuck with an infinite loop / a crash, with (at least as I see) msvcrt attempting to repeatedly throw and unwind an exception following the `longjmp`. One proposal fix would for us on the Rcpp side is to simply provide our own `#define` that masks `Rf_error()` and instead calls R's `Rf_error()` in a safe way (as `Rcpp::stop()` does) -- this is likely to resolve the issue for 99% of the Rcpp client packages. Packages that call `Rf_error()` from C++ code otherwise would require patches; perhaps we could assist in providing patches for those packages if necessary as well. It is also possible that we could build the Rtools toolchain in such a way that the old behaviour is preserved -- this is somewhat less desirable, since we're still letting packages get away with doing the 'wrong' thing, but it is certainly better than 'breaking the world' as we do now. With some luck, this could be achieved with some small modifications to how the toolchain is configured on build, but we are definitely shooting in the dark there as we really have no idea what flags we might want to set. Many thanks, Kevin On Thu, Mar 12, 2015 at 4:52 PM, Duncan Murdoch wrote: > I've told some of you privately that I'm basically not going to be > working on this again before next Wednesday, due to travel. The 3.2.0 > release process starts on Monday, and I will be travelling for all but > two weeks before the release, so there is very little time left for changes. > > So here's what I plan to do: > > By the end of next week I will decide on what toolchain to include in > Rtools. This will be based largely on what I hear from this group, > because the main advantage of the new toolchain is C++ support. > > I think there are 2 likely choices, with a third possible. We could use > the Rtools33 toolchain that is currently on CRAN. This appears likely > to require changes by C++ users. Alternatively, we could revert to the > previous toolchain, based on gcc 4.6.3. If we do revert, we will be > able to change for R 3.2.1, which is unscheduled, but likely to appear > in June, judging by past history. > > The 3rd possibility is that someone other than me makes small changes to > the toolchain (the scripts are on CRAN in bin/windows/Rtools/scripts), > and enough people test the resulting chain so we have general agreement > that the modification works. It needs to be someone other than me, > because I need to try it out in my own tests, and that means I need to > have it in hand by Wednesday, if I'm going to decide by Friday. I will > not build this into Rtools before next Friday, so you'll need to > distribute this by some other means. > > Duncan Murdoch > > > > > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From henrik.singmann at psychologie.uni-freiburg.de Fri Mar 13 11:22:57 2015 From: henrik.singmann at psychologie.uni-freiburg.de (Henrik Singmann) Date: Fri, 13 Mar 2015 11:22:57 +0100 Subject: [Rcpp-devel] New Windows toolchain In-Reply-To: References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> <550226C2.10304@gmail.com> Message-ID: <000f01d05d77$ad20be40$07623ac0$@psychologie.uni-freiburg.de> Hi Kevin, Sorry to step in here but I would like to draw your attention to my mail from yesterday pointing to the fact that there seem to be two issues right now. 1. The issue you mentioned which can be easily reproduced by running the following to lines in R-devel 64bit on Windows with the following lines: Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") doError() 2. The issue I mentioned in my previous mail regarding RcppEigen which can be reproduced by installing RcppEigen from source on 32bit R-devel and running the following line from the unit tests to crash R: ##### code ##### install.packages("RcppEigen", type = "source") require(RcppEigen) data(trees, package="datasets") .Call("fastLm",cbind(1, log(trees$Girth)),log(trees$Volume), 2L,PACKAGE="RcppEigen") ##### end code ##### It seems to even be enough to install RcppEigen from source on 64-bit R-devel and then load/attach the package in 32bit R-devel to crash R. To say it more clearly, the difference between the two issues is that while the first does not appear on 32bit R-devel, the second does, and the other way round for 64-bit R-devel. I am sorry I cannot provide a more detailed analysis what the bug in issue 2 actually triggers, but it is perhaps interesting to see that the following line does not trigger it (in which only the last argument to fastLM was changed): .Call("fastLm",cbind(1, log(trees$Girth)),log(trees$Volume), 1L,PACKAGE="RcppEigen") I hope this issue can also be considered as I will refrain from submitting a (much needed) update of my package to CRAN as long as the check fails like this on 32bit Windows R-devel. Unless of course someone knows that the wrath of CRAN will not be triggered by this error (which I doubt). Cheers, Henrik --- Dr. Henrik Singmann PostDoc University of Z?rich, Switzerland http://singmann.org -----Urspr?ngliche Nachricht----- Von: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto:rcpp-devel-bounces at lists.r-forge.r-project.org] Im Auftrag von Kevin Ushey Gesendet: Freitag, 13. M?rz 2015 03:43 An: Duncan Murdoch Cc: rcpp-devel Betreff: Re: [Rcpp-devel] New Windows toolchain Hi Duncan, First off -- a _huge_ thanks for all the time and effort you've put into updating the toolchain on Windows. We understand that, effectively, your (and R-core)'s stake in updating the toolchain on Windows is quite a bit smaller than ours, since C++ support is not a governing priority in the maintenance and development of R. Those of us on the C++ / Rcpp side are very excited at the prospect of being able to develop R packages which leverage C++11 features and also work across the board on all the platforms used by R, and so the prospect of the toolchain being updated on Windows is very exciting. At the moment, our hope is that a combination of patches to Rcpp, and potentially a few small patches to the Rtools scripts (which we are happy to provide and test ourselves), will show that the updated toolchain is stable enough for use with R 3.2.0. A summary of the issue for anyone else on the list who hasn't followed the other threads follows: --- Many Rcpp client packages / C++-containing packages erroneously make calls directly to `Rf_error()`. Calling `Rf_error()` directly from C++ can result in undefined behaviour -- this is because calling `Rf_error()` will result in a `longjmp`, which bypasses the destructors of any objects on the stack (thereby potentially leaking memory, but in general, causing undefined behaviour). The simplest reproducible example (without Rcpp) demonstrating this problem, from JJ, is this: #include #include #include extern "C" SEXP ouch() { std::string str; Rf_error("ouch"); return R_NilValue; } When calling `ouch()`, the object `str` is effectively leaked, and its destructor is never called, due to the longjmp invoked from `Rf_error()`. With the older Rtools toolchain, packages generally 'got away' with doing this -- perhaps certain objects were leaked, or destructors weren't run, but often this wasn't a show-stopper. This is no longer the case with the new toolchain -- here, our 'luck runs out' and we instead get stuck with an infinite loop / a crash, with (at least as I see) msvcrt attempting to repeatedly throw and unwind an exception following the `longjmp`. One proposal fix would for us on the Rcpp side is to simply provide our own `#define` that masks `Rf_error()` and instead calls R's `Rf_error()` in a safe way (as `Rcpp::stop()` does) -- this is likely to resolve the issue for 99% of the Rcpp client packages. Packages that call `Rf_error()` from C++ code otherwise would require patches; perhaps we could assist in providing patches for those packages if necessary as well. It is also possible that we could build the Rtools toolchain in such a way that the old behaviour is preserved -- this is somewhat less desirable, since we're still letting packages get away with doing the 'wrong' thing, but it is certainly better than 'breaking the world' as we do now. With some luck, this could be achieved with some small modifications to how the toolchain is configured on build, but we are definitely shooting in the dark there as we really have no idea what flags we might want to set. Many thanks, Kevin On Thu, Mar 12, 2015 at 4:52 PM, Duncan Murdoch wrote: > I've told some of you privately that I'm basically not going to be > working on this again before next Wednesday, due to travel. The 3.2.0 > release process starts on Monday, and I will be travelling for all but > two weeks before the release, so there is very little time left for changes. > > So here's what I plan to do: > > By the end of next week I will decide on what toolchain to include in > Rtools. This will be based largely on what I hear from this group, > because the main advantage of the new toolchain is C++ support. > > I think there are 2 likely choices, with a third possible. We could > use the Rtools33 toolchain that is currently on CRAN. This appears > likely to require changes by C++ users. Alternatively, we could > revert to the previous toolchain, based on gcc 4.6.3. If we do > revert, we will be able to change for R 3.2.1, which is unscheduled, > but likely to appear in June, judging by past history. > > The 3rd possibility is that someone other than me makes small changes > to the toolchain (the scripts are on CRAN in > bin/windows/Rtools/scripts), and enough people test the resulting > chain so we have general agreement that the modification works. It > needs to be someone other than me, because I need to try it out in my > own tests, and that means I need to have it in hand by Wednesday, if > I'm going to decide by Friday. I will not build this into Rtools > before next Friday, so you'll need to distribute this by some other means. > > Duncan Murdoch > > > > > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-deve > l _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From jj.allaire at gmail.com Fri Mar 13 11:29:15 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Fri, 13 Mar 2015 06:29:15 -0400 Subject: [Rcpp-devel] New Windows toolchain In-Reply-To: <000f01d05d77$ad20be40$07623ac0$@psychologie.uni-freiburg.de> References: <2019630797.513208.1426100997515.JavaMail.root@fredhutch.org> <550099AC.8010309@gmail.com> <5501CA23.2050000@gmail.com> <5501E103.9080902@gmail.com> <5501E4CA.3060907@gmail.com> <21761.62480.686704.551425@max.nulle.part> <550226C2.10304@gmail.com> <000f01d05d77$ad20be40$07623ac0$@psychologie.uni-freiburg.de> Message-ID: We think the second issue may be caused by Rtools linking against posix threads rather than win32 threads (which it did in previous versions). We will attempt to create a patched version of Rtools today that can be used to verify this. J.J. On Fri, Mar 13, 2015 at 6:22 AM, Henrik Singmann wrote: > Hi Kevin, > > Sorry to step in here but I would like to draw your attention to my mail > from yesterday pointing to the fact that there seem to be two issues right > now. > > 1. The issue you mentioned which can be easily reproduced by running the > following to lines in R-devel 64bit on Windows with the following lines: > Rcpp::cppFunction("void doError() { Rf_error(\"stopping\"); }") > doError() > > 2. The issue I mentioned in my previous mail regarding RcppEigen which can > be reproduced by installing RcppEigen from source on 32bit R-devel and > running the following line from the unit tests to crash R: > > ##### code ##### > install.packages("RcppEigen", type = "source") > require(RcppEigen) > data(trees, package="datasets") > .Call("fastLm",cbind(1, log(trees$Girth)),log(trees$Volume), > 2L,PACKAGE="RcppEigen") > ##### end code ##### > > It seems to even be enough to install RcppEigen from source on 64-bit > R-devel and then load/attach the package in 32bit R-devel to crash R. > > > To say it more clearly, the difference between the two issues is that while > the first does not appear on 32bit R-devel, the second does, and the other > way round for 64-bit R-devel. > > I am sorry I cannot provide a more detailed analysis what the bug in issue 2 > actually triggers, but it is perhaps interesting to see that the following > line does not trigger it (in which only the last argument to fastLM was > changed): > .Call("fastLm",cbind(1, log(trees$Girth)),log(trees$Volume), > 1L,PACKAGE="RcppEigen") > > > I hope this issue can also be considered as I will refrain from submitting a > (much needed) update of my package to CRAN as long as the check fails like > this on 32bit Windows R-devel. Unless of course someone knows that the wrath > of CRAN will not be triggered by this error (which I doubt). > > Cheers, > Henrik > > --- > Dr. Henrik Singmann > PostDoc > University of Z?rich, Switzerland > http://singmann.org > > -----Urspr?ngliche Nachricht----- > Von: rcpp-devel-bounces at lists.r-forge.r-project.org > [mailto:rcpp-devel-bounces at lists.r-forge.r-project.org] Im Auftrag von Kevin > Ushey > Gesendet: Freitag, 13. M?rz 2015 03:43 > An: Duncan Murdoch > Cc: rcpp-devel > Betreff: Re: [Rcpp-devel] New Windows toolchain > > Hi Duncan, > > First off -- a _huge_ thanks for all the time and effort you've put into > updating the toolchain on Windows. We understand that, effectively, your > (and R-core)'s stake in updating the toolchain on Windows is quite a bit > smaller than ours, since C++ support is not a governing priority in the > maintenance and development of R. Those of us on the C++ / Rcpp side are > very excited at the prospect of being able to develop R packages which > leverage C++11 features and also work across the board on all the platforms > used by R, and so the prospect of the toolchain being updated on Windows is > very exciting. > > At the moment, our hope is that a combination of patches to Rcpp, and > potentially a few small patches to the Rtools scripts (which we are happy to > provide and test ourselves), will show that the updated toolchain is stable > enough for use with R 3.2.0. > > A summary of the issue for anyone else on the list who hasn't followed the > other threads follows: > > --- > > Many Rcpp client packages / C++-containing packages erroneously make calls > directly to `Rf_error()`. Calling `Rf_error()` directly from C++ can result > in undefined behaviour -- this is because calling `Rf_error()` will result > in a `longjmp`, which bypasses the destructors of any objects on the stack > (thereby potentially leaking memory, but in general, causing undefined > behaviour). The simplest reproducible example (without Rcpp) demonstrating > this problem, from JJ, is this: > > #include > #include > > #include > > extern "C" SEXP ouch() { > std::string str; > Rf_error("ouch"); > return R_NilValue; > } > > When calling `ouch()`, the object `str` is effectively leaked, and its > destructor is never called, due to the longjmp invoked from `Rf_error()`. > > With the older Rtools toolchain, packages generally 'got away' with doing > this -- perhaps certain objects were leaked, or destructors weren't run, but > often this wasn't a show-stopper. This is no longer the case with the new > toolchain -- here, our 'luck runs out' and we instead get stuck with an > infinite loop / a crash, with (at least as I > see) msvcrt attempting to repeatedly throw and unwind an exception following > the `longjmp`. > > One proposal fix would for us on the Rcpp side is to simply provide our own > `#define` that masks `Rf_error()` and instead calls R's `Rf_error()` in a > safe way (as `Rcpp::stop()` does) -- this is likely to resolve the issue for > 99% of the Rcpp client packages. Packages that call `Rf_error()` from C++ > code otherwise would require patches; perhaps we could assist in providing > patches for those packages if necessary as well. > > It is also possible that we could build the Rtools toolchain in such a way > that the old behaviour is preserved -- this is somewhat less desirable, > since we're still letting packages get away with doing the 'wrong' thing, > but it is certainly better than 'breaking the world' as we do now. With some > luck, this could be achieved with some small modifications to how the > toolchain is configured on build, but we are definitely shooting in the dark > there as we really have no idea what flags we might want to set. > > Many thanks, > Kevin > > On Thu, Mar 12, 2015 at 4:52 PM, Duncan Murdoch > wrote: >> I've told some of you privately that I'm basically not going to be >> working on this again before next Wednesday, due to travel. The 3.2.0 >> release process starts on Monday, and I will be travelling for all but >> two weeks before the release, so there is very little time left for > changes. >> >> So here's what I plan to do: >> >> By the end of next week I will decide on what toolchain to include in >> Rtools. This will be based largely on what I hear from this group, >> because the main advantage of the new toolchain is C++ support. >> >> I think there are 2 likely choices, with a third possible. We could >> use the Rtools33 toolchain that is currently on CRAN. This appears >> likely to require changes by C++ users. Alternatively, we could >> revert to the previous toolchain, based on gcc 4.6.3. If we do >> revert, we will be able to change for R 3.2.1, which is unscheduled, >> but likely to appear in June, judging by past history. >> >> The 3rd possibility is that someone other than me makes small changes >> to the toolchain (the scripts are on CRAN in >> bin/windows/Rtools/scripts), and enough people test the resulting >> chain so we have general agreement that the modification works. It >> needs to be someone other than me, because I need to try it out in my >> own tests, and that means I need to have it in hand by Wednesday, if >> I'm going to decide by Friday. I will not build this into Rtools >> before next Friday, so you'll need to distribute this by some other means. >> >> Duncan Murdoch >> >> >> >> >> >> >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-deve >> l > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From jj.allaire at gmail.com Fri Mar 13 12:31:50 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Fri, 13 Mar 2015 07:31:50 -0400 Subject: [Rcpp-devel] V8 crashes on 32bit windows when built with new Rtools In-Reply-To: References: <550217F1.1040900@gmail.com> Message-ID: It's not posix per-se, it's "posix threads" (of which there is a win32 port available in mingw). This is done sometimes to ease porting of software from posix to windows. An update: Kevin submitted a patch to use win32 threads for Rtools (restoring the previous behavior) and Duncan is rebuilding Rtools now with this change. Once we have this update in hand we can test again. On Thu, Mar 12, 2015 at 7:13 PM, Avraham Adler wrote: > Why would posix be enabled for a Windows target? > > Avi > > On Thu, Mar 12, 2015 at 6:49 PM, Duncan Murdoch > wrote: >> >> On 12/03/2015 5:51 PM, Jeroen Ooms wrote: >> > On Thu, Mar 12, 2015 at 2:40 PM, JJ Allaire > > > wrote: >> > >> > I had an issue like this in RcppParallel that was solved by >> > rebuilding >> > Rcpp from source using the very latest R-devel and Rtools 3.3. Have >> > you tried that? >> > >> > >> > Yes, that doesn't seem to help. >> > >> > I have a suspicion now. I have seen this behavior before when linking >> > libraries on windows that were built with different threading models. >> > Based on gcc -v, the old rtools (and hence my v8 static libs) use win32 >> > threading, whereas the new rtools has been built with >> > --enable-threads=posix. >> >> The scripts are on CRAN, if you want to try modifying them. However, if >> the scripts aren't satisfactory very soon, I'll just go back to the >> previous Rtools. >> >> Duncan >> >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > From jack at jackwasey.com Fri Mar 13 17:49:31 2015 From: jack at jackwasey.com (Jack Wasey) Date: Fri, 13 Mar 2015 12:49:31 -0400 Subject: [Rcpp-devel] stop and Rf_error cause the same problem Message-ID: <5503151B.8020902@jackwasey.com> I've seen it said that calling Rf_error is the cause of the new hang with Rtools33, quoting my example in (closed) github issue https://github.com/RcppCore/Rcpp/issues/276 Just to be clear, the calling Rcpp::stop hung in exactly the same way, but I did at that time also demonstrate the bug using Rf_error. Hope that clarifies things a little, Jack From jj.allaire at gmail.com Fri Mar 13 18:08:06 2015 From: jj.allaire at gmail.com (JJ Allaire) Date: Fri, 13 Mar 2015 13:08:06 -0400 Subject: [Rcpp-devel] stop and Rf_error cause the same problem In-Reply-To: <5503151B.8020902@jackwasey.com> References: <5503151B.8020902@jackwasey.com> Message-ID: Yup, we're aware of the stop issue as well (it has the same root cause). J.J. On Fri, Mar 13, 2015 at 12:49 PM, Jack Wasey wrote: > I've seen it said that calling Rf_error is the cause of the new hang with > Rtools33, quoting my example in (closed) github issue > https://github.com/RcppCore/Rcpp/issues/276 > > Just to be clear, the calling Rcpp::stop hung in exactly the same way, but I > did at that time also demonstrate the bug using Rf_error. > > Hope that clarifies things a little, > Jack > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From bhaskarvk at gmail.com Sun Mar 15 20:48:30 2015 From: bhaskarvk at gmail.com (Bhaskar V. Karambelkar) Date: Sun, 15 Mar 2015 15:48:30 -0400 Subject: [Rcpp-devel] How to elevate privilege from within Rcpp? Message-ID: I need to call a 3rd party library function with root privileges. I'm not sure how to accomplish this from within Rcpp. If I was writing an executable, I could call seteuid to elevate privileges before calling the external function, and set the suid bit on the executable. But given that this needs to be done from within a Rcpp Library, I'm not sure how to do this. The last thing I want to do is run R as root or set suid on the R binary. Any help / pointers would be greatly appreciated. thanks Bhaskar -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Sun Mar 15 21:14:57 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Sun, 15 Mar 2015 15:14:57 -0500 Subject: [Rcpp-devel] How to elevate privilege from within Rcpp? In-Reply-To: References: Message-ID: <21765.59457.789381.284036@max.nulle.part> On 15 March 2015 at 15:48, Bhaskar V. Karambelkar wrote: | I need to call a 3rd party library function with root privileges. | I'm not sure how to accomplish this from within Rcpp. I dont think you can ... | If I was writing an executable, I could call seteuid to elevate privileges | before calling the external function, and set the suid bit on the executable. | But given that this needs to be done from within a Rcpp Library, I'm not sure | how to do this. The last thing I want to do is run R as root or set suid on the | R binary. ... unless you run R with suid bits, which you shouldn't as you rightly point out. Rcpp is standard C/C++ code. So the wisdom of 'man seteuid' still applies: seteuid() sets the effective user ID of the calling process. Unprivileged user processes may only set the effective user ID to the real user ID, the effective user ID or the saved set-user-ID. | Any help / pointers would be greatly appreciated. You may have to call a small self-contained executable which may then have a suid bit set. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From bhaskarvk at gmail.com Sun Mar 15 21:26:38 2015 From: bhaskarvk at gmail.com (Bhaskar V. Karambelkar) Date: Sun, 15 Mar 2015 16:26:38 -0400 Subject: [Rcpp-devel] How to elevate privilege from within Rcpp? In-Reply-To: <21765.59457.789381.284036@max.nulle.part> References: <21765.59457.789381.284036@max.nulle.part> Message-ID: Thanks Dirk, I was afraid of that, but glad that you confirmed it. In the mean time I've figured out a way that may not require root :) thanks again Bhaskar On Sun, Mar 15, 2015 at 4:14 PM, Dirk Eddelbuettel wrote: > > On 15 March 2015 at 15:48, Bhaskar V. Karambelkar wrote: > | I need to call a 3rd party library function with root privileges. > | I'm not sure how to accomplish this from within Rcpp. > > I dont think you can ... > > | If I was writing an executable, I could call seteuid to elevate > privileges > | before calling the external function, and set the suid bit on the > executable. > | But given that this needs to be done from within a Rcpp Library, I'm not > sure > | how to do this. The last thing I want to do is run R as root or set suid > on the > | R binary. > > ... unless you run R with suid bits, which you shouldn't as you rightly > point out. > > Rcpp is standard C/C++ code. So the wisdom of 'man seteuid' still applies: > > seteuid() sets the effective user ID of the calling process. > Unprivileged > user processes may only set the effective user ID to the real user ID, > the > effective user ID or the saved set-user-ID. > > | Any help / pointers would be greatly appreciated. > > You may have to call a small self-contained executable which may then have > a > suid bit set. > > Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From edd at debian.org Sun Mar 15 21:34:52 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Sun, 15 Mar 2015 15:34:52 -0500 Subject: [Rcpp-devel] How to elevate privilege from within Rcpp? In-Reply-To: References: <21765.59457.789381.284036@max.nulle.part> Message-ID: <21765.60652.833997.954356@max.nulle.part> On 15 March 2015 at 16:26, Bhaskar V. Karambelkar wrote: | I was afraid of that, but glad that you confirmed it. In the mean time I've | figured out a way that may not require root :) Hah! If there is a generalizable trick worth sharing then please don't hold back and tell us. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From bhaskarvk at gmail.com Sun Mar 15 22:29:45 2015 From: bhaskarvk at gmail.com (Bhaskar V. Karambelkar) Date: Sun, 15 Mar 2015 17:29:45 -0400 Subject: [Rcpp-devel] How to elevate privilege from within Rcpp? In-Reply-To: <21765.60652.833997.954356@max.nulle.part> References: <21765.59457.789381.284036@max.nulle.part> <21765.60652.833997.954356@max.nulle.part> Message-ID: Nope I'm afraid no generalization trick. Basically I needed to open a RAW socket which requires root access, but reading up on socket man page, I see that now I can open a ICMP socket which is a new addition, and doesn't require root and yet do what I want to do. So just something that works for my particular case. thanks Bhaskar On Sun, Mar 15, 2015 at 4:34 PM, Dirk Eddelbuettel wrote: > > On 15 March 2015 at 16:26, Bhaskar V. Karambelkar wrote: > | I was afraid of that, but glad that you confirmed it. In the mean time > I've > | figured out a way that may not require root :) > > Hah! If there is a generalizable trick worth sharing then please don't > hold > back and tell us. > > Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtenenba at fredhutch.org Mon Mar 16 20:08:00 2015 From: dtenenba at fredhutch.org (Dan Tenenbaum) Date: Mon, 16 Mar 2015 12:08:00 -0700 (PDT) Subject: [Rcpp-devel] Report on building Bioconductor with the new toolchain In-Reply-To: <1993527922.602409.1426532005695.JavaMail.root@fredhutch.org> Message-ID: <1087852121.603219.1426532880090.JavaMail.root@fredhutch.org> Hi, Pardon me if this is off-topic but it looks from recent discussion like several people here are working on helping Duncan with the new toolchain and I thought some reports about issues with it might be welcome/helpful. If not, please ignore this email or suggest a more appropriate place for it (since Duncan is traveling, I thought R-devel might be a less appropriate place). Bioconductor has been running with the new toolchain for a few days. The main issues seem to be: - If a package links against another C/C++ library, that library needs to be rebuilt from source using the compilers in the new Rtools. This can be fairly painful for some windows libraries. In one case (protobuf) the appropriate version of ranlib was not run against the 64-bit static library; I ended up doing this by hand. - A package (Rgraphviz) failed because of a line in Makevars.win: gcc.exe: error: unrecognized command line option '-WI,--enable-auto-import' removing the line: PKG_LIBS += -WI,--enable-auto-import fixed the problem There are several other packages that have problems related to the new toolchain, but I haven't figured out exactly what they are yet. Our build report (http://bioconductor.org/checkResults/devel/bioc-LATEST/) has more information. Here are the specific packages (the report contains a reference to their source in svn; you can check them out with the username readonly and password readonly): http://bioconductor.org/checkResults/devel/bioc-LATEST/affxparser/moscato2-install.html http://bioconductor.org/checkResults/devel/bioc-LATEST/AffyTiling/moscato2-buildsrc.html http://bioconductor.org/checkResults/devel/bioc-LATEST/DNAcopy/moscato2-install.html http://bioconductor.org/checkResults/devel/bioc-LATEST/GeneSelectMMD/moscato2-buildsrc.html http://bioconductor.org/checkResults/devel/bioc-LATEST/Rdisop/moscato2-install.html Let me know if you have any questions about this. Thanks, Dan From edd at debian.org Tue Mar 17 03:29:15 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Mon, 16 Mar 2015 21:29:15 -0500 Subject: [Rcpp-devel] RcppCore repos at GitHub -- New checkout advised Message-ID: <21767.37243.441824.786356@max.nulle.part> All, On the weekend, and while planning to do some minor fixes for an old issue in the the repos for Rcpp, RcppArmadillo and RcppEigen I managed to actually do more damage. So if you have currently checked-out version, the new pull will likely inflate the total commit count by a factor of two. The repos themselves are absolutely fine (following a reset) but you will probably have to reclone these three. I am truly sorry for the inconvience -- this was not what was planned. But the important part is that the code (and commit history) are all fine, but again -- if you have checkout of Rcpp, RcppArmadillo and/or RcppEigen you want to move those out of the way and change over. Sorry, Dirk PS This resets time stamps. If you're like me and you like file timestamps, something like the following is a quick fix -- run it from inside RcppEigen relatiev to a 'parked' copy of the old repo under rcppeigneBORKED ## dry run to test find . ! -path "./.git*" ! -name "\." -exec echo "touch -r ../rcppeigenBORKED/{} {}" \; ## actual reset'ing via touch find . ! -path "./.git*" ! -name "\." -exec touch -r ../rcppeigenBORKED/{} {} \; Adjust names and paths accordingly for the other two. -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From fumb05 at web.de Tue Mar 17 15:26:28 2015 From: fumb05 at web.de (Marius Wirths) Date: Tue, 17 Mar 2015 15:26:28 +0100 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. Message-ID: An HTML attachment was scrubbed... URL: From plummerm at iarc.fr Tue Mar 17 15:37:14 2015 From: plummerm at iarc.fr (Martyn Plummer) Date: Tue, 17 Mar 2015 14:37:14 +0000 Subject: [Rcpp-devel] Report on building Bioconductor with the new toolchain In-Reply-To: <1087852121.603219.1426532880090.JavaMail.root@fredhutch.org> References: <1087852121.603219.1426532880090.JavaMail.root@fredhutch.org> Message-ID: <1426603034.18970.29.camel@iarc.fr> On Mon, 2015-03-16 at 12:08 -0700, Dan Tenenbaum wrote: > Hi, > > Pardon me if this is off-topic but it looks from recent discussion > like several people here are working on helping Duncan with the new > toolchain and I thought some reports about issues with it might be > welcome/helpful. If not, please ignore this email or suggest a more > appropriate place for it (since Duncan is traveling, I thought R-devel > might be a less appropriate place). > > Bioconductor has been running with the new toolchain for a few days. > The main issues seem to be: > > - If a package links against another C/C++ library, that library needs > to be rebuilt from source using the compilers in the new Rtools. This > can be fairly painful for some windows libraries. In one case > (protobuf) the appropriate version of ranlib was not run against the > 64-bit static library; I ended up doing this by hand. That sounds like a show stopper to me. We certainly don't want to have to rebuild third party libraries. But two things are not clear to me 1) Is it just libraries using C++ that are affected or also libraries using only C? 2) What happens if you don't rebuild the library? > - A package (Rgraphviz) failed because of a line in Makevars.win: > gcc.exe: error: unrecognized command line option > '-WI,--enable-auto-import' > removing the line: > PKG_LIBS += -WI,--enable-auto-import > fixed the problem That one is just a typo. To pass an option (e.g. --enable-auto-import) to the linker one should use -Wl not -WI. So it is good that the new tool chain is picking this up. Martyn > There are several other packages that have problems related to the new > toolchain, but I haven't figured out exactly what they are yet. Our > build report (http://bioconductor.org/checkResults/devel/bioc-LATEST/) > has more information. Here are the specific packages (the report > contains a reference to their source in svn; you can check them out > with the username readonly and password readonly): > > http://bioconductor.org/checkResults/devel/bioc-LATEST/affxparser/moscato2-install.html > http://bioconductor.org/checkResults/devel/bioc-LATEST/AffyTiling/moscato2-buildsrc.html > http://bioconductor.org/checkResults/devel/bioc-LATEST/DNAcopy/moscato2-install.html > http://bioconductor.org/checkResults/devel/bioc-LATEST/GeneSelectMMD/moscato2-buildsrc.html > http://bioconductor.org/checkResults/devel/bioc-LATEST/Rdisop/moscato2-install.html > > > Let me know if you have any questions about this. > > Thanks, > Dan > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ----------------------------------------------------------------------- This message and its attachments are strictly confidential. If you are not the intended recipient of this message, please immediately notify the sender and delete it. Since its integrity cannot be guaranteed, its content cannot involve the sender's responsibility. Any misuse, any disclosure or publication of its content, either whole or partial, is prohibited, exception made of formally approved use ----------------------------------------------------------------------- From edd at debian.org Tue Mar 17 15:42:43 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Tue, 17 Mar 2015 09:42:43 -0500 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: References: Message-ID: <21768.15715.881001.794719@max.nulle.part> Hi Marius, On 17 March 2015 at 15:26, Marius Wirths wrote: | Hello everybody, | | this is my very first post on this mailing list so if I have made any mistakes | please feel free to point me to it. | | I have written a package for R. This package internally uses Rcpp-Modules. Cool. So do quite a few other packages on CRAN. | However my package got rejected from CRAN due to the fact that my package can | not be loaded under Solaris. So, i set up a virtual machine with Solaris 10 to | find the error in my code. For now it appears to me, that it is not possible to | use Rcpp-Modules under Solaris using Oracle Studio 12 compilers (which are used | by CRAN as far as i know). I just glanced at two of mine. Both are fine on r-patched-solaris-x86 but fail on r-patched-solaris-sparc due to a compiler issue when dealing with Boost headers via the BH package. I have more package with Modules on CRAN but don't have time now to check all of them. | I created a minimal example through: | | "Rcpp.package.skeleton("somename", module=TRUE)" | | I removed every file from the "src"-folder except the "Num.cpp" file and | reduced the content of the "zzz.R" file located in the "R" folder to the single | line "loadModule("NumEx", TRUE)". | Compiling this minimal example with the latest version of R using the Oracle | Studio 12 compilers results in the output given below. Also i have used the | following enviroment variables (which are used by CRAN too): [...] | Output generated through "R CMD INSTALL ./somename" : | | # R CMD INSTALL ./somename [...] | I have no idea how to solve this problem. Using the "Rcpp.package.skeleton" | without "module=TRUE" compiles just fine (except for one warning) and can be | loaded: [...] | If you need any further informations to help me please feel free to ask. It is a tricky situation. I don't quite know how to say this -- nobody actively working with Rcpp has access to Solaris or interest in it (maybe Martyn Plummer excepted). So right now you are effectively being held hostage over a build failure on an architecture nobody uses. You could stop using Modules. Or you could add a LinkingTo: BH as your package would fail like mine. At least that masks the other issue ;-) Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From edd at debian.org Tue Mar 17 16:03:04 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Tue, 17 Mar 2015 10:03:04 -0500 Subject: [Rcpp-devel] Report on building Bioconductor with the new toolchain In-Reply-To: <1426603034.18970.29.camel@iarc.fr> References: <1087852121.603219.1426532880090.JavaMail.root@fredhutch.org> <1426603034.18970.29.camel@iarc.fr> Message-ID: <21768.16936.541363.364394@max.nulle.part> On 17 March 2015 at 14:37, Martyn Plummer wrote: | That sounds like a show stopper to me. AFAIK there are a number of balls still high in the air. I heard from JJ and Kevin who are working hard with Duncan on this. I am sure they will summarize the situation here in due course. But for now I'd say the dust has very much not settled. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From dtenenba at fredhutch.org Tue Mar 17 16:29:46 2015 From: dtenenba at fredhutch.org (Dan Tenenbaum) Date: Tue, 17 Mar 2015 08:29:46 -0700 (PDT) Subject: [Rcpp-devel] Report on building Bioconductor with the new toolchain In-Reply-To: <1426603034.18970.29.camel@iarc.fr> Message-ID: <2035137562.619607.1426606186210.JavaMail.root@fredhutch.org> ----- Original Message ----- > From: "Martyn Plummer" > To: dtenenba at fredhutch.org > Cc: rcpp-devel at lists.r-forge.r-project.org > Sent: Tuesday, March 17, 2015 7:37:14 AM > Subject: Re: [Rcpp-devel] Report on building Bioconductor with the new toolchain > > On Mon, 2015-03-16 at 12:08 -0700, Dan Tenenbaum wrote: > > Hi, > > > > Pardon me if this is off-topic but it looks from recent discussion > > like several people here are working on helping Duncan with the new > > toolchain and I thought some reports about issues with it might be > > welcome/helpful. If not, please ignore this email or suggest a more > > appropriate place for it (since Duncan is traveling, I thought > > R-devel > > might be a less appropriate place). > > > > Bioconductor has been running with the new toolchain for a few > > days. > > The main issues seem to be: > > > > - If a package links against another C/C++ library, that library > > needs > > to be rebuilt from source using the compilers in the new Rtools. > > This > > can be fairly painful for some windows libraries. In one case > > (protobuf) the appropriate version of ranlib was not run against > > the > > 64-bit static library; I ended up doing this by hand. > > That sounds like a show stopper to me. We certainly don't want to > have > to rebuild third party libraries. Jeroen replied to me off-list and said this was being worked on and was likely the result of using different flags from the previous version of rtools. However, I note that the third party libraries I'm dealing with were not necessarily built with the old Rtools compilers; in some cases I downloaded pre-built binaries. They were built with mingw though. > But two things are not clear to me > 1) Is it just libraries using C++ that are affected or also libraries > using only C? So far it seems like just C++ (protocol buffers, openbabel, proteome wizard, ROOT, etc..) > 2) What happens if you don't rebuild the library? Usually something like this: ** testing if installed package can be loaded Error : .onLoad failed in loadNamespace() for 'ChemmineOB', details: call: inDL(x, as.logical(local), as.logical(now), ...) error: unable to load shared object 'E:/biocbld/bbs-3.1-bioc/R/library/ChemmineOB/libs/i386/ChemmineOB.dll': LoadLibrary failure: Invalid access to memory location. > > > - A package (Rgraphviz) failed because of a line in Makevars.win: > > gcc.exe: error: unrecognized command line option > > '-WI,--enable-auto-import' > > removing the line: > > PKG_LIBS += -WI,--enable-auto-import > > fixed the problem > > That one is just a typo. To pass an option (e.g. > --enable-auto-import) > to the linker one should use -Wl not -WI. So it is good that the new > tool chain is picking this up. Good to know, thanks. Dan > > Martyn > > > There are several other packages that have problems related to the > > new > > toolchain, but I haven't figured out exactly what they are yet. Our > > build report > > (http://bioconductor.org/checkResults/devel/bioc-LATEST/) > > has more information. Here are the specific packages (the report > > contains a reference to their source in svn; you can check them out > > with the username readonly and password readonly): > > > > http://bioconductor.org/checkResults/devel/bioc-LATEST/affxparser/moscato2-install.html > > http://bioconductor.org/checkResults/devel/bioc-LATEST/AffyTiling/moscato2-buildsrc.html > > http://bioconductor.org/checkResults/devel/bioc-LATEST/DNAcopy/moscato2-install.html > > http://bioconductor.org/checkResults/devel/bioc-LATEST/GeneSelectMMD/moscato2-buildsrc.html > > http://bioconductor.org/checkResults/devel/bioc-LATEST/Rdisop/moscato2-install.html > > > > > > Let me know if you have any questions about this. > > > > Thanks, > > Dan > > > > _______________________________________________ > > Rcpp-devel mailing list > > Rcpp-devel at lists.r-forge.r-project.org > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > ----------------------------------------------------------------------- > This message and its attachments are strictly confidential. If you > are > not the intended recipient of this message, please immediately notify > the sender and delete it. Since its integrity cannot be guaranteed, > its content cannot involve the sender's responsibility. Any misuse, > any disclosure or publication of its content, either whole or > partial, > is prohibited, exception made of formally approved use > ----------------------------------------------------------------------- > From edd at debian.org Tue Mar 17 23:18:08 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Tue, 17 Mar 2015 17:18:08 -0500 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: <21768.15715.881001.794719@max.nulle.part> References: <21768.15715.881001.794719@max.nulle.part> Message-ID: <21768.43040.451611.157409@max.nulle.part> On 17 March 2015 at 09:42, Dirk Eddelbuettel wrote: | I just glanced at two of mine. Both are fine on r-patched-solaris-x86 but | fail on r-patched-solaris-sparc due to a compiler issue when dealing with | Boost headers via the BH package. I have more package with Modules on CRAN | but don't have time now to check all of them. A counter-example is RcppCNPy. It uses Rcpp Modules, yet tests clean on Solaris and everywhere apart from snowleopard: http://cran.r-project.org/web/checks/check_results_RcppCNPy.html Now, Rcpp Modules are quite expressive and powerful, but also a little finicky at times. It may well be that you use a feature I do not use in RcppCNPy. But you could try this and see how it fares in your VM. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From plummerm at iarc.fr Wed Mar 18 10:45:43 2015 From: plummerm at iarc.fr (Martyn Plummer) Date: Wed, 18 Mar 2015 09:45:43 +0000 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: <21768.15715.881001.794719@max.nulle.part> References: <21768.15715.881001.794719@max.nulle.part> Message-ID: <1426671943.18065.10.camel@iarc.fr> On Tue, 2015-03-17 at 09:42 -0500, Dirk Eddelbuettel wrote: > Hi Marius, > > On 17 March 2015 at 15:26, Marius Wirths wrote: > | Hello everybody, > | > | this is my very first post on this mailing list so if I have made any mistakes > | please feel free to point me to it. > | > | I have written a package for R. This package internally uses Rcpp-Modules. > > Cool. So do quite a few other packages on CRAN. > > | However my package got rejected from CRAN due to the fact that my package can > | not be loaded under Solaris. So, i set up a virtual machine with Solaris 10 to > | find the error in my code. For now it appears to me, that it is not possible to > | use Rcpp-Modules under Solaris using Oracle Studio 12 compilers (which are used > | by CRAN as far as i know). > > I just glanced at two of mine. Both are fine on r-patched-solaris-x86 but > fail on r-patched-solaris-sparc due to a compiler issue when dealing with > Boost headers via the BH package. I have more package with Modules on CRAN > but don't have time now to check all of them. > > | I created a minimal example through: > | > | "Rcpp.package.skeleton("somename", module=TRUE)" > | > | I removed every file from the "src"-folder except the "Num.cpp" file and > | reduced the content of the "zzz.R" file located in the "R" folder to the single > | line "loadModule("NumEx", TRUE)". > | Compiling this minimal example with the latest version of R using the Oracle > | Studio 12 compilers results in the output given below. Also i have used the > | following enviroment variables (which are used by CRAN too): > > [...] > > | Output generated through "R CMD INSTALL ./somename" : > | > | # R CMD INSTALL ./somename > > [...] > > | I have no idea how to solve this problem. Using the "Rcpp.package.skeleton" > | without "module=TRUE" compiles just fine (except for one warning) and can be > | loaded: > > [...] > > | If you need any further informations to help me please feel free to ask. > > > It is a tricky situation. I don't quite know how to say this -- nobody > actively working with Rcpp has access to Solaris or interest in it (maybe > Martyn Plummer excepted). So right now you are effectively being held > hostage over a build failure on an architecture nobody uses. Well that is one interpretation. Another one is that as a testing platform Solaris exposes weaknesses in C++ code that are not found by other platforms due to its fastidious interpretation of the C++98 standard. For example, Solaris Studio distinguishes between functions with C and C ++ linkage. In Marius's test package this generates multiple warnings and may be the cause of the missing symbol, which demangles as: Rcpp::standard_delete_finalizer(__type_0*) As you can see, __type_0 is just a placeholder, so something went wrong with the linkage. Marius, if you mail me your test package I will see what it gives me. Martyn > You could stop using Modules. > > Or you could add a LinkingTo: BH as your package would fail like mine. At > least that masks the other issue ;-) > > Dirk > ----------------------------------------------------------------------- This message and its attachments are strictly confidential. If you are not the intended recipient of this message, please immediately notify the sender and delete it. Since its integrity cannot be guaranteed, its content cannot involve the sender's responsibility. Any misuse, any disclosure or publication of its content, either whole or partial, is prohibited, exception made of formally approved use ----------------------------------------------------------------------- From edd at debian.org Wed Mar 18 12:19:59 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 18 Mar 2015 06:19:59 -0500 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: <1426671943.18065.10.camel@iarc.fr> References: <21768.15715.881001.794719@max.nulle.part> <1426671943.18065.10.camel@iarc.fr> Message-ID: <21769.24415.707623.561766@max.nulle.part> On 18 March 2015 at 09:45, Martyn Plummer wrote: | Well that is one interpretation. Another one is that as a testing | platform Solaris exposes weaknesses in C++ code that are not found by | other platforms due to its fastidious interpretation of the C++98 | standard. Indeed. Which makes it even more sad that CRAN only has a budget of about one minute for tests. So effectively a large number of tests are ... simply turned off at CRAN. We do not get the feedback from another compiler you (correctly, at least "in theory") highlight here as we are being told to suppress the tests. Not ideal. | Marius, if you mail me your test package I will see what it gives me. Appreciate that. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From fumb05 at web.de Wed Mar 18 13:03:26 2015 From: fumb05 at web.de (Marius Wirths) Date: Wed, 18 Mar 2015 13:03:26 +0100 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: <21769.24415.707623.561766@max.nulle.part> References: <21768.15715.881001.794719@max.nulle.part> <1426671943.18065.10.camel@iarc.fr>, <21769.24415.707623.561766@max.nulle.part> Message-ID: An HTML attachment was scrubbed... URL: From edd at debian.org Wed Mar 18 13:21:08 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 18 Mar 2015 07:21:08 -0500 Subject: [Rcpp-devel] Using "Rcpp Modules" under Solaris fails to load. In-Reply-To: References: <21768.15715.881001.794719@max.nulle.part> <1426671943.18065.10.camel@iarc.fr> <21769.24415.707623.561766@max.nulle.part> Message-ID: <21769.28084.274533.446458@max.nulle.part> On 18 March 2015 at 13:03, Marius Wirths wrote: | enviroment. Dirk, your information, that there are indeed many packages using | modules without any issues was an important information for me. So i think, it | must be something wrong with my code. Maybe something is missing in my | "Makevars"-file or something like that. I will investigate your package Maybe. Maybe not. Could just be that that some facets of Modules pass on Slowlaris whereas other trigger an error. As I mentioned to Martyn, we stopped running all tests years ago as we (and everybody else) were told (with some reason, CRAN being volunteers and all...) to not run as many tests. You can try Rcpp itself by setting the env.var. RunAllRcppTests="yes" which is checked for in tests/doRUnit.R -- or just alter DESCRIPTION to have a dev version number, ie 0.11.5.1 which also tickles it (as we only ever ship release versions format a.b.c with three components). Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From murdoch.duncan at gmail.com Wed Mar 18 14:27:19 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Wed, 18 Mar 2015 09:27:19 -0400 Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 Message-ID: <55097D37.3040105@gmail.com> To anyone following the Windows toolchain saga: The gcc 4.9.2 toolchain that is currently in Rtools33 has too many incompatibilities with existing code, so we won't be using it in the R 3.2.0 build. I will soon be uploading to CRAN a new version of Rtools33 that is very similar to Rtools32, containing gcc 4.6.3. We are continuing to work on the new toolchain, and hope to have it ready before R 3.2.1 is released. The known problems are as follows: - C++ code should not call Rf_error(), as it uses longjmp, and the behaviour of longjmp is undefined in C++ when destructors need to be called. However, a number of packages do call Rf_error, and in gcc 4.6.3, they get away with it. In our candidate 4.9.2 build, they crashed. If we can't work around this, I'll suggest that we test for the presence of Rf_error in C++ code, and start issuing warnings or errors when it is seen. But before we do that, we need a solid replacement. - There are some other crashes that appear to be unrelated, also with C++ code. - There are some subtle differences in arithmetic that result in tests failing. These may be due to bugs in MinGW-w64 code, or may be unavoidable. Duncan Murdoch From murdoch.duncan at gmail.com Wed Mar 18 14:27:26 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Wed, 18 Mar 2015 09:27:26 -0400 Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 Message-ID: <55097D3E.7020606@gmail.com> To anyone following the Windows toolchain saga: The gcc 4.9.2 toolchain that is currently in Rtools33 has too many incompatibilities with existing code, so we won't be using it in the R 3.2.0 build. I will soon be uploading to CRAN a new version of Rtools33 that is very similar to Rtools32, containing gcc 4.6.3. We are continuing to work on the new toolchain, and hope to have it ready before R 3.2.1 is released. The known problems are as follows: - C++ code should not call Rf_error(), as it uses longjmp, and the behaviour of longjmp is undefined in C++ when destructors need to be called. However, a number of packages do call Rf_error, and in gcc 4.6.3, they get away with it. In our candidate 4.9.2 build, they crashed. If we can't work around this, I'll suggest that we test for the presence of Rf_error in C++ code, and start issuing warnings or errors when it is seen. But before we do that, we need a solid replacement. - There are some other crashes that appear to be unrelated, also with C++ code. - There are some subtle differences in arithmetic that result in tests failing. These may be due to bugs in MinGW-w64 code, or may be unavoidable. Duncan Murdoch From edd at debian.org Wed Mar 18 15:04:49 2015 From: edd at debian.org (Dirk Eddelbuettel) Date: Wed, 18 Mar 2015 09:04:49 -0500 Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 In-Reply-To: <55097D3E.7020606@gmail.com> References: <55097D3E.7020606@gmail.com> Message-ID: <21769.34305.173670.719458@max.nulle.part> Duncan (and everybody else working on it behind the curtains), Thanks for the update. All the work is truly appreciated, and it is really too bad that we turned to C++ testing so late in the process. But ensuring release quality is paramount, so withholding g++ 4.9.* on Windows til these issues are sorted out is the correct approach. On 18 March 2015 at 09:27, Duncan Murdoch wrote: | called. However, a number of packages do call Rf_error, and in gcc | 4.6.3, they get away with it. In our candidate 4.9.2 build, they For the record, "we also got away with it" using every other know compiler and configuration. We would have addressed this earlier if it had been known. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org From dtenenba at fredhutch.org Wed Mar 18 15:55:06 2015 From: dtenenba at fredhutch.org (Dan Tenenbaum) Date: Wed, 18 Mar 2015 07:55:06 -0700 (PDT) Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 In-Reply-To: <55097D3E.7020606@gmail.com> Message-ID: <1298686136.647031.1426690506425.JavaMail.root@fredhutch.org> Duncan, ----- Original Message ----- > From: "Duncan Murdoch" > To: "R-devel at r-project.org" , rcpp-devel at r-forge.wu-wien.ac.at > Sent: Wednesday, March 18, 2015 6:27:26 AM > Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 > > To anyone following the Windows toolchain saga: > > The gcc 4.9.2 toolchain that is currently in Rtools33 has too many > incompatibilities with existing code, so we won't be using it in the > R > 3.2.0 build. I will soon be uploading to CRAN a new version of > Rtools33 > that is very similar to Rtools32, containing gcc 4.6.3. > > We are continuing to work on the new toolchain, and hope to have it > ready before R 3.2.1 is released. Thanks very much for your work on this. > > The known problems are as follows: > > - C++ code should not call Rf_error(), as it uses longjmp, and the > behaviour of longjmp is undefined in C++ when destructors need to be > called. However, a number of packages do call Rf_error, and in gcc > 4.6.3, they get away with it. In our candidate 4.9.2 build, they > crashed. If we can't work around this, I'll suggest that we test for > the presence of Rf_error in C++ code, and start issuing warnings or > errors when it is seen. But before we do that, we need a solid > replacement. > > - There are some other crashes that appear to be unrelated, also > with > C++ code. > > - There are some subtle differences in arithmetic that result in > tests > failing. These may be due to bugs in MinGW-w64 code, > or may be unavoidable. Is it not considered a "known problem" that C++ libraries linked against by R packages need to be rebuilt with g++ 4.9.2 in order for the R packages to install/load? Thanks again, Dan > > Duncan Murdoch > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > From plummerm at iarc.fr Wed Mar 18 16:33:30 2015 From: plummerm at iarc.fr (Martyn Plummer) Date: Wed, 18 Mar 2015 15:33:30 +0000 Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 In-Reply-To: <1298686136.647031.1426690506425.JavaMail.root@fredhutch.org> References: <1298686136.647031.1426690506425.JavaMail.root@fredhutch.org> Message-ID: <1426692809.18065.14.camel@iarc.fr> On Wed, 2015-03-18 at 07:55 -0700, Dan Tenenbaum wrote: > Is it not considered a "known problem" that C++ libraries linked > against by R packages need to be rebuilt with g++ 4.9.2 in order for > the R packages to install/load? This could well be due to incompatible thread models (win32 vs posix). See the thread "V8 crashes..." on the Rcpp-devel mailing list. We have not yet had a chance to test the gcc 4.9.2 toolchain built with win32 threads. Martyn > Thanks again, > Dan ----------------------------------------------------------------------- This message and its attachments are strictly confidential. If you are not the intended recipient of this message, please immediately notify the sender and delete it. Since its integrity cannot be guaranteed, its content cannot involve the sender's responsibility. Any misuse, any disclosure or publication of its content, either whole or partial, is prohibited, exception made of formally approved use ----------------------------------------------------------------------- From murdoch.duncan at gmail.com Wed Mar 18 17:29:40 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Wed, 18 Mar 2015 12:29:40 -0400 Subject: [Rcpp-devel] Windows gcc toolchain for R 3.2.0 In-Reply-To: <1426692809.18065.14.camel@iarc.fr> References: <1298686136.647031.1426690506425.JavaMail.root@fredhutch.org> <1426692809.18065.14.camel@iarc.fr> Message-ID: <5509A7F4.7090408@gmail.com> On 18/03/2015 11:33 AM, Martyn Plummer wrote: > On Wed, 2015-03-18 at 07:55 -0700, Dan Tenenbaum wrote: > > Is it not considered a "known problem" that C++ libraries linked > > against by R packages need to be rebuilt with g++ 4.9.2 in order for > > the R packages to install/load? > > This could well be due to incompatible thread models (win32 vs posix). > See the thread "V8 crashes..." on the Rcpp-devel mailing list. We have > not yet had a chance to test the gcc 4.9.2 toolchain built with win32 > threads. I believe the RStudio folks have tried it, and it was not sufficient to fix everything; time just ran out for finding what else was wrong. Duncan Murdoch From murdoch.duncan at gmail.com Wed Mar 18 19:56:23 2015 From: murdoch.duncan at gmail.com (Duncan Murdoch) Date: Wed, 18 Mar 2015 14:56:23 -0400 Subject: [Rcpp-devel] Windows gcc 4.9.2 toolchain Message-ID: <5509CA57.4030708@gmail.com> To Windows Rcpp users: I'm now back online, and looking for test cases for the new gcc 4.9.2 toolchain. I'll be posting them to a page that's currently on CRAN at bin/windows/Rtools/scripts. R 3.2.0 will not use this toolchain, it will use the old one. Rtools33 no longer has the new toolchain, but you can download it in two tarballs from that CRAN page. Currently R-devel is using the old toolchain, but once we have fixes for all the known problems, I'll switch it back to the new one. Since I haven't been following the discussion on this list for the last 5 days, it will take a while to get all the known problems posted; if you've found one and it's not there yet, please send me a link to simple reproduction instructions. Duncan Murdoch From ejb6 at cornell.edu Tue Mar 24 15:53:50 2015 From: ejb6 at cornell.edu (Elliot Joel Bernstein) Date: Tue, 24 Mar 2015 10:53:50 -0400 Subject: [Rcpp-devel] Using Rcpp and MATLAB MAT-File API to Read MAT Files Message-ID: I'm trying to use the MATLAB Mat-File API to read data from MAT files into R. (I've tried using the R.matlab package, but the matrices I'm loading can be very large, and that method was too slow. In addition, some of the files I'm loading are v7.3 MAT files, which are not supported by the package.) I wrote the following simple function to start testing. It works fine if I compile without the Rcpp parts in Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but when I run in R it crashes my R session without displaying "Test". #include #include "mat.h" using namespace Rcpp; // [[Rcpp::export]] NumericVector rcpp_mat_load() { MATFile* mf = matOpen("z:/temp/test.mat", "r"); printf("Test\n"); double val = 10; NumericVector y = NumericVector::create(val); return y; } I am using R version 3.0.3 on Windows 7 with version 0.11.2 of Rcpp. The input file, test.mat, is a very small MAT file that I'm just using for testing. I am happy to provide any other details that would be helpful. I would appreciate any suggestions for getting this to work. Thanks! - Elliot -------------- next part -------------- An HTML attachment was scrubbed... URL: From xian at unm.edu Wed Mar 25 13:08:19 2015 From: xian at unm.edu (Christian Gunning) Date: Wed, 25 Mar 2015 08:08:19 -0400 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files Message-ID: Can you provide a link to the mat.h file referenced in the source (which I assume is the same as the API mentioned below)? Also, see Dirk's comments about compilers here: http://stackoverflow.com/questions/10723165/using-visual-c-with-r and here: dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf Best, Christian > I'm trying to use the MATLAB Mat-File API to read data from MAT files into > R. (I've tried using the R.matlab package, but the matrices I'm loading can > be very large, and that method was too slow. In addition, some of the files > I'm loading are v7.3 MAT files, which are not supported by the package.) I > wrote the following simple function to start testing. It works fine if I > compile without the Rcpp parts in Visual Studio (i.e. I can call matOpen > and get a nonzero pointer), but when I run in R it crashes my R session > without displaying "Test". > > #include > #include "mat.h" > > using namespace Rcpp; > > // [[Rcpp::export]] > NumericVector rcpp_mat_load() { > > MATFile* mf = matOpen("z:/temp/test.mat", "r"); > printf("Test\n"); > double val = 10; > NumericVector y = NumericVector::create(val); > return y; > } -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! From DSmith at nexidia.com Wed Mar 25 13:36:47 2015 From: DSmith at nexidia.com (Dale Smith) Date: Wed, 25 Mar 2015 12:36:47 +0000 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: References: Message-ID: mat.h is a header file which comes with MATLAB. I don't think the license allows redistribution, and you would not be able to compile without the MATLAB libraries. An alternative may be Octave since it will read .mat files. Perhaps someone can email the name of the header file used from the Octave distribution? http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave This also offers a viable way to test this type of code for those who don't have MATLAB. Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 ext 4008 | fax: +1 404 495 7221 | nexidia.com -----Original Message----- From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto:rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian Gunning Sent: Wednesday, March 25, 2015 8:08 AM To: rcpp-devel at lists.r-forge.r-project.org Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files Can you provide a link to the mat.h file referenced in the source (which I assume is the same as the API mentioned below)? Also, see Dirk's comments about compilers here: http://stackoverflow.com/questions/10723165/using-visual-c-with-r and here: dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf Best, Christian > I'm trying to use the MATLAB Mat-File API to read data from MAT files > into R. (I've tried using the R.matlab package, but the matrices I'm > loading can be very large, and that method was too slow. In addition, > some of the files I'm loading are v7.3 MAT files, which are not > supported by the package.) I wrote the following simple function to > start testing. It works fine if I compile without the Rcpp parts in > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but > when I run in R it crashes my R session without displaying "Test". > > #include > #include "mat.h" > > using namespace Rcpp; > > // [[Rcpp::export]] > NumericVector rcpp_mat_load() { > > MATFile* mf = matOpen("z:/temp/test.mat", "r"); > printf("Test\n"); > double val = 10; > NumericVector y = NumericVector::create(val); > return y; > } -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From ejb6 at cornell.edu Wed Mar 25 14:28:25 2015 From: ejb6 at cornell.edu (Elliot Joel Bernstein) Date: Wed, 25 Mar 2015 09:28:25 -0400 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: References: Message-ID: Dale - Thank you for the suggestion. I have MATLAB and the MATLAB libraries, and I am not looking to redistribute anything. I just have a lot of data in MAT files that I would like to be able to load analyze in R. I am able to compile and link against the MATLAB libraries, but the code crashes when I run it. I looked into the Octave FAQ. It looks like they don't handle v7.3 files either, and I don't think they provide an API that I could access from R/C++. - Elliot On Wed, Mar 25, 2015 at 8:36 AM, Dale Smith wrote: > mat.h is a header file which comes with MATLAB. I don't think the license > allows redistribution, and you would not be able to compile without the > MATLAB libraries. > > An alternative may be Octave since it will read .mat files. Perhaps > someone can email the name of the header file used from the Octave > distribution? > > > http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave > > This also offers a viable way to test this type of code for those who > don't have MATLAB. > > Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 > ext 4008 | fax: +1 404 495 7221 | nexidia.com > > > -----Original Message----- > From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: > rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian > Gunning > Sent: Wednesday, March 25, 2015 8:08 AM > To: rcpp-devel at lists.r-forge.r-project.org > Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT > Files > > Can you provide a link to the mat.h file referenced in the source (which I > assume is the same as the API mentioned below)? > > Also, see Dirk's comments about compilers here: > http://stackoverflow.com/questions/10723165/using-visual-c-with-r > and here: > dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf > > Best, > Christian > > > I'm trying to use the MATLAB Mat-File API to read data from MAT files > > into R. (I've tried using the R.matlab package, but the matrices I'm > > loading can be very large, and that method was too slow. In addition, > > some of the files I'm loading are v7.3 MAT files, which are not > > supported by the package.) I wrote the following simple function to > > start testing. It works fine if I compile without the Rcpp parts in > > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but > > when I run in R it crashes my R session without displaying "Test". > > > > #include > > #include "mat.h" > > > > using namespace Rcpp; > > > > // [[Rcpp::export]] > > NumericVector rcpp_mat_load() { > > > > MATFile* mf = matOpen("z:/temp/test.mat", "r"); > > printf("Test\n"); > > double val = 10; > > NumericVector y = NumericVector::create(val); > > return y; > > } > > -- > A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbuonagurio at exponent.com Wed Mar 25 16:09:57 2015 From: jbuonagurio at exponent.com (John Buonagurio) Date: Wed, 25 Mar 2015 15:09:57 +0000 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: References: Message-ID: <00F9E64B563E7C45818C85C1B7FCC726A77B753F@SFMB02.exponent.com> Elliot, How are you building the executable? Did you make sure to include all dependencies in the search path (run depends.exe against libmat.dll and libmx.dll)? Did you make sure to set the appropriate architecture e.g. -m64 in CFLAGS? I don't have access to the MAT-File libraries to test this myself. Start simple and take Rcpp out of the equation. Call matOpen from a simple C function with type void. Does it work with dyn.load() and .C()? If you are *only* working with v7.3 files which are based on HDF5, you may be able to avoid this issue and access them using rhdf5: source("http://bioconductor.org/biocLite.R") biocLite("rhdf5") library(rhdf5) h5dump("file.mat", load=FALSE) John From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto:rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Elliot Joel Bernstein Sent: Wednesday, March 25, 2015 9:28 AM To: Dale Smith Cc: xian at unm.edu; rcpp-devel at lists.r-forge.r-project.org Subject: Re: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files Dale - Thank you for the suggestion. I have MATLAB and the MATLAB libraries, and I am not looking to redistribute anything. I just have a lot of data in MAT files that I would like to be able to load analyze in R. I am able to compile and link against the MATLAB libraries, but the code crashes when I run it. I looked into the Octave FAQ. It looks like they don't handle v7.3 files either, and I don't think they provide an API that I could access from R/C++. - Elliot On Wed, Mar 25, 2015 at 8:36 AM, Dale Smith wrote: mat.h is a header file which comes with MATLAB. I don't think the license allows redistribution, and you would not be able to compile without the MATLAB libraries. An alternative may be Octave since it will read .mat files. Perhaps someone can email the name of the header file used from the Octave distribution? http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave This also offers a viable way to test this type of code for those who don't have MATLAB. Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 ext 4008 | fax: +1 404 495 7221 | nexidia.com -----Original Message----- From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto:rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian Gunning Sent: Wednesday, March 25, 2015 8:08 AM To: rcpp-devel at lists.r-forge.r-project.org Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files Can you provide a link to the mat.h file referenced in the source (which I assume is the same as the API mentioned below)? Also, see Dirk's comments about compilers here: http://stackoverflow.com/questions/10723165/using-visual-c-with-r and here: dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf Best, Christian > I'm trying to use the MATLAB Mat-File API to read data from MAT files > into R. (I've tried using the R.matlab package, but the matrices I'm > loading can be very large, and that method was too slow. In addition, > some of the files I'm loading are v7.3 MAT files, which are not > supported by the package.) I wrote the following simple function to > start testing. It works fine if I compile without the Rcpp parts in > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but > when I run in R it crashes my R session without displaying "Test". > > #include > #include "mat.h" > > using namespace Rcpp; > > // [[Rcpp::export]] > NumericVector rcpp_mat_load() { > >? ? ?MATFile* mf = matOpen("z:/temp/test.mat", "r"); >? ? ?printf("Test\n"); >? ? ?double val = 10; >? ? ?NumericVector y = NumericVector::create(val); >? ? ?return y; > } -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From ejb6 at cornell.edu Wed Mar 25 17:26:57 2015 From: ejb6 at cornell.edu (Elliot Joel Bernstein) Date: Wed, 25 Mar 2015 12:26:57 -0400 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: <00F9E64B563E7C45818C85C1B7FCC726A77B753F@SFMB02.exponent.com> References: <00F9E64B563E7C45818C85C1B7FCC726A77B753F@SFMB02.exponent.com> Message-ID: John - Thanks for the suggestions. I tried taking not only Rcpp but R out of the loop, and just compiling from the command line with MinGW. That crashed as well, so I think there's probably a compatibility issue with MATLAB's API and MinGW. I'll contact Mathworks tech support to try and get some help with that. Regarding v7.3...my files aren't all v7.3, but I could definitely save them all that way, which might be the easiest solution. I tried what you suggested with some of my real data files, which contain two variables each: a matrix and a struct. I get the following warnings: > data <- h5dump("z:/temp/test73.mat", bit64conversion='double') Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced by NA's Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced by NA's Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced by NA's The data look OK, as far as I can tell. (The matrix is 411x30927, so it's not like I can check every element, but I don't see anything obviously wrong.) In addition to the two variables I was expecting, the result has two additional elements, #refs# and #subsystem. I'm not actually sure how to see what they contain, since the names include the comment character. Can I safely ignore those warnings? Thanks. - Elliot On Wed, Mar 25, 2015 at 11:09 AM, John Buonagurio wrote: > Elliot, > > How are you building the executable? Did you make sure to include all > dependencies in the search path (run depends.exe against libmat.dll and > libmx.dll)? Did you make sure to set the appropriate architecture e.g. -m64 > in CFLAGS? I don't have access to the MAT-File libraries to test this > myself. > > Start simple and take Rcpp out of the equation. Call matOpen from a simple > C function with type void. Does it work with dyn.load() and .C()? > > If you are *only* working with v7.3 files which are based on HDF5, you may > be able to avoid this issue and access them using rhdf5: > > source("http://bioconductor.org/biocLite.R") > biocLite("rhdf5") > library(rhdf5) > h5dump("file.mat", load=FALSE) > > John > > From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: > rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Elliot Joel > Bernstein > Sent: Wednesday, March 25, 2015 9:28 AM > To: Dale Smith > Cc: xian at unm.edu; rcpp-devel at lists.r-forge.r-project.org > Subject: Re: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read > MAT Files > > Dale - > > Thank you for the suggestion. I have MATLAB and the MATLAB libraries, and > I am not looking to redistribute anything. I just have a lot of data in MAT > files that I would like to be able to load analyze in R. I am able to > compile and link against the MATLAB libraries, but the code crashes when I > run it. I looked into the Octave FAQ. It looks like they don't handle v7.3 > files either, and I don't think they provide an API that I could access > from R/C++. > > - Elliot > > On Wed, Mar 25, 2015 at 8:36 AM, Dale Smith wrote: > mat.h is a header file which comes with MATLAB. I don't think the license > allows redistribution, and you would not be able to compile without the > MATLAB libraries. > > An alternative may be Octave since it will read .mat files. Perhaps > someone can email the name of the header file used from the Octave > distribution? > > > http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave > > This also offers a viable way to test this type of code for those who > don't have MATLAB. > > Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 > ext 4008 | fax: +1 404 495 7221 | nexidia.com > > > -----Original Message----- > From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: > rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian > Gunning > Sent: Wednesday, March 25, 2015 8:08 AM > To: rcpp-devel at lists.r-forge.r-project.org > Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT > Files > > Can you provide a link to the mat.h file referenced in the source (which I > assume is the same as the API mentioned below)? > > Also, see Dirk's comments about compilers here: > http://stackoverflow.com/questions/10723165/using-visual-c-with-r > and here: > dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf > > Best, > Christian > > > I'm trying to use the MATLAB Mat-File API to read data from MAT files > > into R. (I've tried using the R.matlab package, but the matrices I'm > > loading can be very large, and that method was too slow. In addition, > > some of the files I'm loading are v7.3 MAT files, which are not > > supported by the package.) I wrote the following simple function to > > start testing. It works fine if I compile without the Rcpp parts in > > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but > > when I run in R it crashes my R session without displaying "Test". > > > > #include > > #include "mat.h" > > > > using namespace Rcpp; > > > > // [[Rcpp::export]] > > NumericVector rcpp_mat_load() { > > > > MATFile* mf = matOpen("z:/temp/test.mat", "r"); > > printf("Test\n"); > > double val = 10; > > NumericVector y = NumericVector::create(val); > > return y; > > } > > -- > A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ejb6 at cornell.edu Wed Mar 25 17:57:51 2015 From: ejb6 at cornell.edu (Elliot Joel Bernstein) Date: Wed, 25 Mar 2015 12:57:51 -0400 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: References: <00F9E64B563E7C45818C85C1B7FCC726A77B753F@SFMB02.exponent.com> Message-ID: John - One quick follow-up about the warnings: I think they have to do with loading cell arrays. Basically, all the actual data ends up in the #refs# field, in what seems like an arbitrary order, and the field that should contain the data is all NAs. Do you know if there's any way to get around that? I think all the cell arrays being saved contain only strings, so I can change things in MATLAB to save them as char arrays if necessary. Thanks. - Elliot On Wed, Mar 25, 2015 at 12:26 PM, Elliot Joel Bernstein wrote: > John - > > Thanks for the suggestions. I tried taking not only Rcpp but R out of the > loop, and just compiling from the command line with MinGW. That crashed as > well, so I think there's probably a compatibility issue with MATLAB's API > and MinGW. I'll contact Mathworks tech support to try and get some help > with that. > > Regarding v7.3...my files aren't all v7.3, but I could definitely save > them all that way, which might be the easiest solution. I tried what you > suggested with some of my real data files, which contain two variables > each: a matrix and a struct. I get the following warnings: > > > data <- h5dump("z:/temp/test73.mat", bit64conversion='double') > Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced > by NA's > Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced > by NA's > Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced > by NA's > > The data look OK, as far as I can tell. (The matrix is 411x30927, so it's > not like I can check every element, but I don't see anything obviously > wrong.) In addition to the two variables I was expecting, the result has > two additional elements, #refs# and #subsystem. I'm not actually sure how > to see what they contain, since the names include the comment character. > Can I safely ignore those warnings? > > Thanks. > > - Elliot > > > On Wed, Mar 25, 2015 at 11:09 AM, John Buonagurio < > jbuonagurio at exponent.com> wrote: > >> Elliot, >> >> How are you building the executable? Did you make sure to include all >> dependencies in the search path (run depends.exe against libmat.dll and >> libmx.dll)? Did you make sure to set the appropriate architecture e.g. -m64 >> in CFLAGS? I don't have access to the MAT-File libraries to test this >> myself. >> >> Start simple and take Rcpp out of the equation. Call matOpen from a >> simple C function with type void. Does it work with dyn.load() and .C()? >> >> If you are *only* working with v7.3 files which are based on HDF5, you >> may be able to avoid this issue and access them using rhdf5: >> >> source("http://bioconductor.org/biocLite.R") >> biocLite("rhdf5") >> library(rhdf5) >> h5dump("file.mat", load=FALSE) >> >> John >> >> From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: >> rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Elliot Joel >> Bernstein >> Sent: Wednesday, March 25, 2015 9:28 AM >> To: Dale Smith >> Cc: xian at unm.edu; rcpp-devel at lists.r-forge.r-project.org >> Subject: Re: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read >> MAT Files >> >> Dale - >> >> Thank you for the suggestion. I have MATLAB and the MATLAB libraries, and >> I am not looking to redistribute anything. I just have a lot of data in MAT >> files that I would like to be able to load analyze in R. I am able to >> compile and link against the MATLAB libraries, but the code crashes when I >> run it. I looked into the Octave FAQ. It looks like they don't handle v7.3 >> files either, and I don't think they provide an API that I could access >> from R/C++. >> >> - Elliot >> >> On Wed, Mar 25, 2015 at 8:36 AM, Dale Smith wrote: >> mat.h is a header file which comes with MATLAB. I don't think the license >> allows redistribution, and you would not be able to compile without the >> MATLAB libraries. >> >> An alternative may be Octave since it will read .mat files. Perhaps >> someone can email the name of the header file used from the Octave >> distribution? >> >> >> http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave >> >> This also offers a viable way to test this type of code for those who >> don't have MATLAB. >> >> Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 >> ext 4008 | fax: +1 404 495 7221 | nexidia.com >> >> >> -----Original Message----- >> From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: >> rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian >> Gunning >> Sent: Wednesday, March 25, 2015 8:08 AM >> To: rcpp-devel at lists.r-forge.r-project.org >> Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT >> Files >> >> Can you provide a link to the mat.h file referenced in the source (which >> I assume is the same as the API mentioned below)? >> >> Also, see Dirk's comments about compilers here: >> http://stackoverflow.com/questions/10723165/using-visual-c-with-r >> and here: >> dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf >> >> Best, >> Christian >> >> > I'm trying to use the MATLAB Mat-File API to read data from MAT files >> > into R. (I've tried using the R.matlab package, but the matrices I'm >> > loading can be very large, and that method was too slow. In addition, >> > some of the files I'm loading are v7.3 MAT files, which are not >> > supported by the package.) I wrote the following simple function to >> > start testing. It works fine if I compile without the Rcpp parts in >> > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but >> > when I run in R it crashes my R session without displaying "Test". >> > >> > #include >> > #include "mat.h" >> > >> > using namespace Rcpp; >> > >> > // [[Rcpp::export]] >> > NumericVector rcpp_mat_load() { >> > >> > MATFile* mf = matOpen("z:/temp/test.mat", "r"); >> > printf("Test\n"); >> > double val = 10; >> > NumericVector y = NumericVector::create(val); >> > return y; >> > } >> >> -- >> A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ejb6 at cornell.edu Thu Mar 26 16:52:12 2015 From: ejb6 at cornell.edu (Elliot Joel Bernstein) Date: Thu, 26 Mar 2015 11:52:12 -0400 Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read MAT Files In-Reply-To: References: <00F9E64B563E7C45818C85C1B7FCC726A77B753F@SFMB02.exponent.com> Message-ID: It turns out this wasn't really a Rcpp issue, but I want to post the solution, in case someone stumbles on this thread while trying to use the MAT-file API with MinGW. It turns out I was just linking against the wrong libraries. My original library directory was C:/MATLAB2013a/extern/lib/win64/microsoft, but the correct one is C:/MATLAB2013a/bin/win64. (Where "C:/MATLAB2013a" should be replaced by the root of your MATLAB installation.) - Elliot On Wed, Mar 25, 2015 at 12:57 PM, Elliot Joel Bernstein wrote: > John - > > One quick follow-up about the warnings: I think they have to do with > loading cell arrays. Basically, all the actual data ends up in the #refs# > field, in what seems like an arbitrary order, and the field that should > contain the data is all NAs. Do you know if there's any way to get around > that? I think all the cell arrays being saved contain only strings, so I > can change things in MATLAB to save them as char arrays if necessary. > > Thanks. > > - Elliot > > On Wed, Mar 25, 2015 at 12:26 PM, Elliot Joel Bernstein > wrote: > >> John - >> >> Thanks for the suggestions. I tried taking not only Rcpp but R out of the >> loop, and just compiling from the command line with MinGW. That crashed as >> well, so I think there's probably a compatibility issue with MATLAB's API >> and MinGW. I'll contact Mathworks tech support to try and get some help >> with that. >> >> Regarding v7.3...my files aren't all v7.3, but I could definitely save >> them all that way, which might be the easiest solution. I tried what you >> suggested with some of my real data files, which contain two variables >> each: a matrix and a struct. I get the following warnings: >> >> > data <- h5dump("z:/temp/test73.mat", bit64conversion='double') >> Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced >> by NA's >> Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced >> by NA's >> Warning: h5read for type 'REFERENCE' not yet implemented. Values replaced >> by NA's >> >> The data look OK, as far as I can tell. (The matrix is 411x30927, so it's >> not like I can check every element, but I don't see anything obviously >> wrong.) In addition to the two variables I was expecting, the result has >> two additional elements, #refs# and #subsystem. I'm not actually sure how >> to see what they contain, since the names include the comment character. >> Can I safely ignore those warnings? >> >> Thanks. >> >> - Elliot >> >> >> On Wed, Mar 25, 2015 at 11:09 AM, John Buonagurio < >> jbuonagurio at exponent.com> wrote: >> >>> Elliot, >>> >>> How are you building the executable? Did you make sure to include all >>> dependencies in the search path (run depends.exe against libmat.dll and >>> libmx.dll)? Did you make sure to set the appropriate architecture e.g. -m64 >>> in CFLAGS? I don't have access to the MAT-File libraries to test this >>> myself. >>> >>> Start simple and take Rcpp out of the equation. Call matOpen from a >>> simple C function with type void. Does it work with dyn.load() and .C()? >>> >>> If you are *only* working with v7.3 files which are based on HDF5, you >>> may be able to avoid this issue and access them using rhdf5: >>> >>> source("http://bioconductor.org/biocLite.R") >>> biocLite("rhdf5") >>> library(rhdf5) >>> h5dump("file.mat", load=FALSE) >>> >>> John >>> >>> From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: >>> rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Elliot >>> Joel Bernstein >>> Sent: Wednesday, March 25, 2015 9:28 AM >>> To: Dale Smith >>> Cc: xian at unm.edu; rcpp-devel at lists.r-forge.r-project.org >>> Subject: Re: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to >>> Read MAT Files >>> >>> Dale - >>> >>> Thank you for the suggestion. I have MATLAB and the MATLAB libraries, >>> and I am not looking to redistribute anything. I just have a lot of data in >>> MAT files that I would like to be able to load analyze in R. I am able to >>> compile and link against the MATLAB libraries, but the code crashes when I >>> run it. I looked into the Octave FAQ. It looks like they don't handle v7.3 >>> files either, and I don't think they provide an API that I could access >>> from R/C++. >>> >>> - Elliot >>> >>> On Wed, Mar 25, 2015 at 8:36 AM, Dale Smith wrote: >>> mat.h is a header file which comes with MATLAB. I don't think the >>> license allows redistribution, and you would not be able to compile without >>> the MATLAB libraries. >>> >>> An alternative may be Octave since it will read .mat files. Perhaps >>> someone can email the name of the header file used from the Octave >>> distribution? >>> >>> >>> http://stackoverflow.com/questions/9915658/how-do-you-open-mat-files-in-octave >>> >>> This also offers a viable way to test this type of code for those who >>> don't have MATLAB. >>> >>> Dale Smith, Ph.D. | Data Scientist | nexidia | office: +1 404 495 7220 >>> ext 4008 | fax: +1 404 495 7221 | nexidia.com >>> >>> >>> -----Original Message----- >>> From: rcpp-devel-bounces at lists.r-forge.r-project.org [mailto: >>> rcpp-devel-bounces at lists.r-forge.r-project.org] On Behalf Of Christian >>> Gunning >>> Sent: Wednesday, March 25, 2015 8:08 AM >>> To: rcpp-devel at lists.r-forge.r-project.org >>> Subject: [Rcpp-devel] R.e. Using Rcpp and MATLAB MAT-File API to Read >>> MAT Files >>> >>> Can you provide a link to the mat.h file referenced in the source (which >>> I assume is the same as the API mentioned below)? >>> >>> Also, see Dirk's comments about compilers here: >>> http://stackoverflow.com/questions/10723165/using-visual-c-with-r >>> and here: >>> dirk.eddelbuettel.com/code/rcpp/Rcpp-FAQ.pdf >>> >>> Best, >>> Christian >>> >>> > I'm trying to use the MATLAB Mat-File API to read data from MAT files >>> > into R. (I've tried using the R.matlab package, but the matrices I'm >>> > loading can be very large, and that method was too slow. In addition, >>> > some of the files I'm loading are v7.3 MAT files, which are not >>> > supported by the package.) I wrote the following simple function to >>> > start testing. It works fine if I compile without the Rcpp parts in >>> > Visual Studio (i.e. I can call matOpen and get a nonzero pointer), but >>> > when I run in R it crashes my R session without displaying "Test". >>> > >>> > #include >>> > #include "mat.h" >>> > >>> > using namespace Rcpp; >>> > >>> > // [[Rcpp::export]] >>> > NumericVector rcpp_mat_load() { >>> > >>> > MATFile* mf = matOpen("z:/temp/test.mat", "r"); >>> > printf("Test\n"); >>> > double val = 10; >>> > NumericVector y = NumericVector::create(val); >>> > return y; >>> > } >>> >>> -- >>> A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! >>> _______________________________________________ >>> Rcpp-devel mailing list >>> Rcpp-devel at lists.r-forge.r-project.org >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >>> _______________________________________________ >>> Rcpp-devel mailing list >>> Rcpp-devel at lists.r-forge.r-project.org >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uragami at hotmail.com Mon Mar 30 11:05:53 2015 From: uragami at hotmail.com (ogami musashi) Date: Mon, 30 Mar 2015 11:05:53 +0200 Subject: [Rcpp-devel] Accessing list of list members to permute them Message-ID: Hello I'm a Rcpp newbie, so sorry if the question is trivial. I have an R object which is a list of 1000 elements. Each elements is a result from a discrete wavelet transform (from package wmtsa) with 5 elements. One of them have a 50000 rows and 16 colums. The first level (1000) are in fact random signals of 50 000 days. On each of them i ran a DWT that separated each signal into 16 components (of decreasing frequency) for each day thus resulting in a 50000 days * 16 components matrix for each of the 1000 signals. I would like to sort it like that: for each component (a list of 16 elements) having for each day (50000 rows) the 1000 signals (1000 columns) here are the objects in R: dwtlist: 1000 elements * 5 elements* (50000*16) matrix sortdwtlist:16 elements*(50000*1000) matrix the code in R: for(i in 1:16){ for(j in 1:50000){ for (k in 1:1000){ sortdwtlist[[i]][j,k]<-dwtlist[[k]][[1]][j,i] } } } This takes about 27 minutes to complete. I thus tried to put in C++ using Rcpp (i'm using sourceCpp): #include using namespace Rcpp; // [[Rcpp::export]] List rearrangelist(List x){ int ni=15; int nj=49999; int nk=999; List output; for (int i= 0 ; i On Mon, Mar 30, 2015 at 6:00 AM, rcpp-devel-request at lists.r-forge.r-project.org < rcpp-devel-request at lists.r-forge.r-project.org> wrote: > > > // [[Rcpp::export]] > > List rearrangelist(List x){ > int ni=15; > int nj=49999; > int nk=999; > List output; > > for (int i= 0 ; i for (int j=0 ; j for (int k=0 ; j output[i](j,k)=x[k][0](j,i); > } > } > } > return output; > } First thing, you probably want to walk before you run. Get comfortable returning a list of vectors before you start chaining a large number of indexes together. Second, practice getting your indexes from the objects that you're operating on: size_t ni = x.size(); And you probably want to fill your return list like this. List foo; // do stuff foo.push_back(result); -xian > -- A man, a plan, a cat, a ham, a yak, a yam, a hat, a canal ? Panama! -------------- next part -------------- An HTML attachment was scrubbed... URL: From romain at r-enthusiasts.com Mon Mar 30 13:09:27 2015 From: romain at r-enthusiasts.com (=?utf-8?Q?Romain_Fran=C3=A7ois?=) Date: Mon, 30 Mar 2015 13:09:27 +0200 Subject: [Rcpp-devel] Accessing list of list members to permute them In-Reply-To: References: Message-ID: <565BFF6C-45C1-453F-BF07-DC60E73F73B0@r-enthusiasts.com> Hi, Perhaps you can use ListOf instead of List, as output[i] does not know it is a matrix, so the operator()(int, int) don?t make sense. You?d need (untested, ont sure you can nest layers of ListOf) ListOf< NumericMatrix > x ListOf< ListOf > output ; Otherwise, perhaps something like the more noisy code: NumericMatrix output_i = output[i] ; List x_k = x[k] ; NumericMatrix x_k_0 = x_k[0] ; output_i(j,k) = x_k_0(j,i) ; Also, you need to actually creates the matrices in output. something like : List output(ni) ; for (int i= 0 ; i Le 30 mars 2015 ? 11:05, ogami musashi a ?crit : > > Hello > > I'm a Rcpp newbie, so sorry if the question is trivial. > > I have an R object which is a list of 1000 elements. Each elements is a result from a discrete wavelet transform (from package wmtsa) with 5 elements. One of them have a 50000 rows and 16 colums. > > The first level (1000) are in fact random signals of 50 000 days. On each of them i ran a DWT that separated each signal into 16 components (of decreasing frequency) for each day thus resulting in a 50000 days * 16 components matrix for each of the 1000 signals. > > I would like to sort it like that: for each component (a list of 16 elements) having for each day (50000 rows) the 1000 signals (1000 columns) > > > here are the objects in R: > > dwtlist: 1000 elements * 5 elements* (50000*16) matrix > sortdwtlist:16 elements*(50000*1000) matrix > > the code in R: > > for(i in 1:16){ > > for(j in 1:50000){ > > for (k in 1:1000){ > > sortdwtlist[[i]][j,k]<-dwtlist[[k]][[1]][j,i] > } > } > } > > > This takes about 27 minutes to complete. > > I thus tried to put in C++ using Rcpp (i'm using sourceCpp): > > #include > using namespace Rcpp; > > // [[Rcpp::export]] > > List rearrangelist(List x){ > int ni=15; > int nj=49999; > int nk=999; > List output; > > for (int i= 0 ; i for (int j=0 ; j for (int k=0 ; j output[i](j,k)=x[k][0](j,i); > } > } > } > return output; > } > > > But it doesn't work. error stops at the line of permutation, so i'm sure there's something with the types inside the list. > Can someone help me? > > Thank you! > > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From krthornt at uci.edu Tue Mar 31 00:19:58 2015 From: krthornt at uci.edu (Kevin Thornton) Date: Mon, 30 Mar 2015 15:19:58 -0700 Subject: [Rcpp-devel] Problem with IntegerVector::value_type in recent Rcpp Message-ID: Hi, I've come across an issue when compiling some code with Rcpp >= 0.11.4 (using R 3.1.3 with either gcc or clang on Linux, or clang on OS X). The code that reproduces the issue is here: https://gist.github.com/molpopgen/b3bda09590172044ff84 Specifically, the last function is where I'm running into trouble. I also tried the same ideas using C++98 (function objects instead of lambda expressions), and got the same results. The short version is that Rcpp::IntegerVector::value_type appears to lose some info about value_type's const-ness when a const IntegerVector is passed to an Rcpp function. I went back an tried older version of Rcpp and verified that the code from the above link compiles with version from 0.11.0 through 0.11.3. Starting with 0.11.4, the last function in that piece of code will no longer compile, and appears to think that a non-const int & is the type that it is looking for. I tried the other STL-like typedefs that one may expect ( reference, const_reference ), and they do not exist for Rcpp vector types. Any thoughts? Is this intended? Should I not write functions taking const references to Rcpp types? Best, Kevin ___________________________ Kevin Thornton Associate Professor Ecology and Evolutionary Biology University of California, Irvine http://www.molpopgen.org http://github.com/molpopgen http://github.com/ThorntonLab From romain at r-enthusiasts.com Tue Mar 31 10:04:46 2015 From: romain at r-enthusiasts.com (=?utf-8?Q?Romain_Fran=C3=A7ois?=) Date: Tue, 31 Mar 2015 10:04:46 +0200 Subject: [Rcpp-devel] Problem with IntegerVector::value_type in recent Rcpp In-Reply-To: References: Message-ID: <00297EEC-3D69-4B33-A480-5399F92DF862@r-enthusiasts.com> Hi, That?s because VTYPE::value_type is traits::r_vector_proxy::type, i.e. template struct r_vector_proxy{ typedef typename storage_type::type& type ; } ; so VTYPE::value_type is int& You can either use stored_type, i.e. std::for_each( v.begin(), v.end(),[]( const VTYPE::stored_type& ) ... Or do a little dance with the reference qualifier: std::for_each( v.begin(), v.end(),[]( const std::remove_reference::type& __n ) ... Both of which are not very satisfying. FWIW, in Rcpp11/Rcpp14, VTYPE::value_type is defined like this: typedef typename traits::storage_type::type value_type ; so IntegerVector::value_type is int, and therefore you get what you rightfully expect. Romain > Le 31 mars 2015 ? 00:19, Kevin Thornton a ?crit : > > Hi, > > I've come across an issue when compiling some code with Rcpp >= 0.11.4 (using R 3.1.3 with either gcc or clang on Linux, or clang on OS X). > > The code that reproduces the issue is here: https://gist.github.com/molpopgen/b3bda09590172044ff84 > > Specifically, the last function is where I'm running into trouble. I also tried the same ideas using C++98 (function objects instead of lambda expressions), and got the same results. > > The short version is that Rcpp::IntegerVector::value_type appears to lose some info about value_type's const-ness when a const IntegerVector is passed to an Rcpp function. I went back an tried older version of Rcpp and verified that the code from the above link compiles with version from 0.11.0 through 0.11.3. Starting with 0.11.4, the last function in that piece of code will no longer compile, and appears to think that a non-const int & is the type that it is looking for. > > I tried the other STL-like typedefs that one may expect ( reference, const_reference ), and they do not exist for Rcpp vector types. > > Any thoughts? Is this intended? Should I not write functions taking const references to Rcpp types? > > Best, > > Kevin > > ___________________________ > Kevin Thornton > Associate Professor > Ecology and Evolutionary Biology > University of California, Irvine > http://www.molpopgen.org > http://github.com/molpopgen > http://github.com/ThorntonLab > > > > > > > _______________________________________________ > Rcpp-devel mailing list > Rcpp-devel at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel From krthornt at uci.edu Tue Mar 31 17:39:10 2015 From: krthornt at uci.edu (Kevin Thornton) Date: Tue, 31 Mar 2015 08:39:10 -0700 Subject: [Rcpp-devel] Problem with IntegerVector::value_type in recent Rcpp In-Reply-To: <00297EEC-3D69-4B33-A480-5399F92DF862@r-enthusiasts.com> References: <00297EEC-3D69-4B33-A480-5399F92DF862@r-enthusiasts.com> Message-ID: <5B6B3A0C-C999-4C0E-B239-C671202A7760@uci.edu> Thanks for the explanation, Romain. As a follow-up, is there a reason why value_type is defined in terms of a reference? In the mean time, I'll use remove_reference. -Kevin > On Mar 31, 2015, at 1:04 AM, Romain Fran?ois wrote: > > Hi, > > That?s because VTYPE::value_type is traits::r_vector_proxy::type, i.e. > > template > struct r_vector_proxy{ > typedef typename storage_type::type& type ; > } ; > > so VTYPE::value_type is int& > > You can either use stored_type, i.e. > > std::for_each( v.begin(), v.end(),[]( const VTYPE::stored_type& ) ... > > > Or do a little dance with the reference qualifier: > > std::for_each( v.begin(), v.end(),[]( const std::remove_reference::type& __n ) ... > > Both of which are not very satisfying. > > > FWIW, in Rcpp11/Rcpp14, VTYPE::value_type is defined like this: > > typedef typename traits::storage_type::type value_type ; > > so IntegerVector::value_type is int, and therefore you get what you rightfully expect. > > > Romain > > >> Le 31 mars 2015 ? 00:19, Kevin Thornton a ?crit : >> >> Hi, >> >> I've come across an issue when compiling some code with Rcpp >= 0.11.4 (using R 3.1.3 with either gcc or clang on Linux, or clang on OS X). >> >> The code that reproduces the issue is here: https://gist.github.com/molpopgen/b3bda09590172044ff84 >> >> Specifically, the last function is where I'm running into trouble. I also tried the same ideas using C++98 (function objects instead of lambda expressions), and got the same results. >> >> The short version is that Rcpp::IntegerVector::value_type appears to lose some info about value_type's const-ness when a const IntegerVector is passed to an Rcpp function. I went back an tried older version of Rcpp and verified that the code from the above link compiles with version from 0.11.0 through 0.11.3. Starting with 0.11.4, the last function in that piece of code will no longer compile, and appears to think that a non-const int & is the type that it is looking for. >> >> I tried the other STL-like typedefs that one may expect ( reference, const_reference ), and they do not exist for Rcpp vector types. >> >> Any thoughts? Is this intended? Should I not write functions taking const references to Rcpp types? >> >> Best, >> >> Kevin >> >> ___________________________ >> Kevin Thornton >> Associate Professor >> Ecology and Evolutionary Biology >> University of California, Irvine >> http://www.molpopgen.org >> http://github.com/molpopgen >> http://github.com/ThorntonLab >> >> >> >> >> >> >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > ___________________________ Kevin Thornton Associate Professor Ecology and Evolutionary Biology University of California, Irvine http://www.molpopgen.org http://github.com/molpopgen http://github.com/ThorntonLab From romain at r-enthusiasts.com Tue Mar 31 19:25:50 2015 From: romain at r-enthusiasts.com (=?utf-8?Q?Romain_Fran=C3=A7ois?=) Date: Tue, 31 Mar 2015 19:25:50 +0200 Subject: [Rcpp-devel] Problem with IntegerVector::value_type in recent Rcpp In-Reply-To: <5B6B3A0C-C999-4C0E-B239-C671202A7760@uci.edu> References: <00297EEC-3D69-4B33-A480-5399F92DF862@r-enthusiasts.com> <5B6B3A0C-C999-4C0E-B239-C671202A7760@uci.edu> Message-ID: Probably just my incompetence at the time of writing it. Would be trivial enough to fix in Rcpp for anyone with the proper skills and willingness. I only have 1.05 of those 2. Romain > Le 31 mars 2015 ? 17:39, Kevin Thornton a ?crit : > > Thanks for the explanation, Romain. > > As a follow-up, is there a reason why value_type is defined in terms of a reference? In the mean time, I'll use remove_reference. > > -Kevin > > >> On Mar 31, 2015, at 1:04 AM, Romain Fran?ois wrote: >> >> Hi, >> >> That?s because VTYPE::value_type is traits::r_vector_proxy::type, i.e. >> >> template >> struct r_vector_proxy{ >> typedef typename storage_type::type& type ; >> } ; >> >> so VTYPE::value_type is int& >> >> You can either use stored_type, i.e. >> >> std::for_each( v.begin(), v.end(),[]( const VTYPE::stored_type& ) ... >> >> >> Or do a little dance with the reference qualifier: >> >> std::for_each( v.begin(), v.end(),[]( const std::remove_reference::type& __n ) ... >> >> Both of which are not very satisfying. >> >> >> FWIW, in Rcpp11/Rcpp14, VTYPE::value_type is defined like this: >> >> typedef typename traits::storage_type::type value_type ; >> >> so IntegerVector::value_type is int, and therefore you get what you rightfully expect. >> >> >> Romain >> >> >>> Le 31 mars 2015 ? 00:19, Kevin Thornton a ?crit : >>> >>> Hi, >>> >>> I've come across an issue when compiling some code with Rcpp >= 0.11.4 (using R 3.1.3 with either gcc or clang on Linux, or clang on OS X). >>> >>> The code that reproduces the issue is here: https://gist.github.com/molpopgen/b3bda09590172044ff84 >>> >>> Specifically, the last function is where I'm running into trouble. I also tried the same ideas using C++98 (function objects instead of lambda expressions), and got the same results. >>> >>> The short version is that Rcpp::IntegerVector::value_type appears to lose some info about value_type's const-ness when a const IntegerVector is passed to an Rcpp function. I went back an tried older version of Rcpp and verified that the code from the above link compiles with version from 0.11.0 through 0.11.3. Starting with 0.11.4, the last function in that piece of code will no longer compile, and appears to think that a non-const int & is the type that it is looking for. >>> >>> I tried the other STL-like typedefs that one may expect ( reference, const_reference ), and they do not exist for Rcpp vector types. >>> >>> Any thoughts? Is this intended? Should I not write functions taking const references to Rcpp types? >>> >>> Best, >>> >>> Kevin >>> >>> ___________________________ >>> Kevin Thornton >>> Associate Professor >>> Ecology and Evolutionary Biology >>> University of California, Irvine >>> http://www.molpopgen.org >>> http://github.com/molpopgen >>> http://github.com/ThorntonLab >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Rcpp-devel mailing list >>> Rcpp-devel at lists.r-forge.r-project.org >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> > > ___________________________ > Kevin Thornton > Associate Professor > Ecology and Evolutionary Biology > University of California, Irvine > http://www.molpopgen.org > http://github.com/molpopgen > http://github.com/ThorntonLab > > > > > >