[Rcpp-devel] How to manage testing code for Rcpp based project?

[Dirk Eddelbuettel]

On 21 August 2012 at 20:55, Peng Yu wrote:
| Hi Dirk,
| 
| > Before you look at RcppArmadillo, look at Rcpp itself:
| >
| >    - 22 unit test files
| 
| Are you referring to these files as test files.
| 
| ~/Documents/src/Rsrc/cran/Rcpp_0.9.13/inst/unitTests$ ls runit.*.R |wc -l
| 22
| 
| >    - 349 unit test functions
| 
| And here is how you get the number 349?

I have a simple (and possibly only approximative script) that counts for me:

   edd at max:~/svn/rcpp/pkg$ cat ../scripts/countTests.sh 
   #!/bin/bash

   cd Rcpp/inst/unitTests/
   echo -n "Functions: "
   egrep "test.* function\(" runit.*R | wc -l
   echo -n "Tests    : "
   egrep "check(Equals|Exception|True)" runit.*R  | wc -l

Improvements are as always welcome.
 
| /Documents/src/Rsrc/cran/Rcpp_0.9.13/inst/unitTests$ grep function
| runit.*.R|grep '\<test\.' | wc -l
| 349
| 
| >    - 770 unit tests
| 
| How do you get the above number?

See above,

| I don't quite what these directories are for. They seems to be some
| packages as well. Would you please help me understand how they are
| used for testing?
| 
| ~/Documents/src/Rsrc/cran/Rcpp_0.9.13/inst/unitTests$ ls -goltrd
| RcppTestA/ testRcpp*
| drwxr-xr-x 8 272 Jun 28 23:19 testRcppModule
| drwxr-xr-x 8 272 Jun 28 23:19 testRcppClass
| drwxr-xr-x 7 238 Jun 28 23:19 RcppTestA/

Read the sources of the unit tests.  We also test using Rcpp Modules etc.
 
| > It is far from perfect. In fact, the CRAN gatekeepers now complain that
| > testing takes too long (!!) and have forced us to disable a number of these
| > tests so that the testing part takes no longer than about one minute.
| >
| > R itself has three competing unit test frameworks. We went with RUnit as it
| > permits
| >
| >    - testing when developing (and I have a simple script below)
| >
| >    - testing via R CMD check, and
| >
| >    - testing once installed.
| >
| > We added a solution to have a vignette summarize the tests.
| 
| I assume that you mean Rcpp-unitTests.pdf?
| 
| 
| How does Rcpp-unitTests.txt get generated automatically during package
| build? It looks to me that it has to be manually generated?

Look at the Makefile in inst/doc/

| ~/Documents/src/Rsrc/cran/Rcpp_0.9.13$ ackall Rcpp-unitTests.txt
| inst/doc/Rcpp-unitTests.Rnw
| 25:results <- "unitTests-results/Rcpp-unitTests.txt"
| 
| man/RcppUnitTests.Rd
| 19:txt <- system.file( "doc", "unitTests-results",
| "Rcpp-unitTests.txt", package = "Rcpp" )
| 
| MD5
| 67:bee860827a4518d5fc3bbb8cc2cb4897
| *inst/doc/unitTests-results/Rcpp-unitTests.txt
| 
| 
| > RUnit works, but the setup is not the easiest. If someone has ideas for
| > better solutions, by all means implement those.  Unit testing has helped us
| > many times, and I consider it to be a very important part of package
| > development.
| 
| How (or in what aspect) RUnit is better than devtools?

I am not sure how you get from the paragraph I wrote to the question you
ask.  I use RUnit as it works for me.  

Dirk
 
| -- 
| Regards,
| Peng

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com