[Rcpp-devel] Rcpp module visible after check but not after install
Dirk Eddelbuettel
edd at debian.org
Thu Nov 9 18:30:52 CET 2017
On 9 November 2017 at 11:34, Christopher Genovese wrote:
| Sorry, let me be clear. I've done it both ways with the same basic
| result (for #2). Specifically, doing the R CMD CHECK, R CMD BUILD,
| R CMD INSTALL sequence gives an installed package with the
| the two classes invisible. So I don't think this is a devtools issue.
|
| This is rather frustrating, I agree, especially as it worked previously.
Agreed. But some things also changed in R ... and I am not entirely what the
actual cause was for the change you see.
| Is there any documentation (preferably short of going through the Rcpp
| modules
| code) on the process by which these symbols get exported into R. The
| symbols are in the .so file, but for instance, where/when would the
| function R_init_falconnr function, which is created by
| Rcpp::compileAttributes
| and seems to load the modules, be called?
Well, I also find that combining Modules and compileAttributes() can be
finicky. If output for this was was generated, you could place it e.g. in
src/init.c -- R calls the resulting function on package load. But, and that
is a big 'but' you don't have to unless you say .registration=TRUE in
useDynLib() in NAMESPACE. And I seem to recall I actually wrote the
src/init.c for RcppAnnoy "by hand" as the generator in R itself does not know
about Modules. We may need to add a tool here.
And again, note that you do not need src/init.c -- you only need it if you
set .registration=TRUE. So as first step, I'd say 'do not do that'. I know
it sounds silly -- but if I were you I'd start from the working Modules
example eg in the Rcpp unittests and would then re-add my code. Slowly, and
step by step. That way you will know which change breaks it.
And by all I means I don't claim this is ideal -- if we learn something along
the way we can try to make Modules better. It remains a useful yet somewhat
obscure and underdocumented tool.
Dirk
|
| Thanks, Chris
|
|
| On Thu, Nov 9, 2017 at 9:59 AM, Dirk Eddelbuettel <edd at debian.org> wrote:
|
| >
| > On 9 November 2017 at 09:32, Christopher Genovese wrote:
| > | Thanks, Dirk.
| > |
| > | What exactly do you mean by 1. and 2. ? Per 'Writing R Extensions' we
| > are
| > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then
| > install
| > | > it
| > | > via 'R CMD INSTALL'.
| > | >
| > | > Are you doing that?
| > | >
| > |
| > | Yes. I usually use devtools, but I've done it directly as well.
| >
| > Take this up with the devtools developers then, please.
| >
| > I operate with R command, and possibly very thin littler wrappers such as
| > build.r and rcc.r to, respectively, build and check.
| >
| > | In #1, for instance: I go to the package source directory, start R,
| > | do devtools::build and devtools::check. This builds and compiles
| > | the package. After library() at this point, the module components
| > | are visible, e.g., LshParameterSetter.
| > |
| > | In #2, for instance, I do devtools::install() or
| > devtools::install_github()
| > | or similar.
| > | Quit R, go to some other directory, restart R, and call library(). The R
| > | entry
| > | points are visible, but none of the module classes.
| > |
| > | I'm unclear as to what in the installation/export process can explain the
| > | difference
| > | between #1 and #2.
| >
| > Again, I have no idea what devtools does here. Other people may.
| >
| > The Rcpp documentation does not cover devtools so you are on your own.
| >
| > | Before emailing yesterday, I had tried removing the
| > RCPP_EXPOSED_CLASS()'s,
| > | but then the code does not compile. There is a static_assert error in
| > | wrap_dispatch_unknown_iterable in Rcpp's wrap.h (line 520). I did not
| > see an
| > | easy way to fix that. (I vaguely recall that I only included them
| > | originally because
| > | I had to.)
| >
| > I think you need those. I just glance at the different variants on the
| > train
| > in, they are mostly equivalent and just layers of each other.
| >
| > | I will look RcppAnnoy and see if I can find a way around using these
| > | macros.
| >
| > Sounds good.
| >
| > Modules can be frustrating. I just tried with some larger code at work,
| > and
| > it doesn't load :-/ Something else may get in the way but hard to tell.
| >
| > Rcpp Attributes is just fine.
| >
| > Dirk
| >
| > | Thanks again, Chris
| > |
| > |
| > |
| > |
| > | On Thu, Nov 9, 2017 at 7:55 AM, Dirk Eddelbuettel <edd at debian.org>
| > wrote:
| > |
| > | >
| > | > On 9 November 2017 at 01:54, Christopher Genovese wrote:
| > | > | Hello,
| > | > |
| > | > | I have a package under development that uses Rcpp modules to expose
| > | > | some C++ classes. I hadn't touched the package for some months (close
| > | > | to a year). At that time the package compiled and installed without
| > | > problem
| > | > | and worked well. I could access the classes as expected. Recently,
| > having
| > | > | updated Rcpp and R in the meantime, I reinstalled the package with
| > | > | the following result:
| > | > |
| > | > | 1. If building and loading from within the package directory, all
| > | > works
| > | > | fine.
| > | > | 2. When installing the package, either from github, locally, or
| > | > | otherwise,
| > | > | the modules are *not* visible.
| > | >
| > | > What exactly do you mean by 1. and 2. ? Per 'Writing R Extensions' we
| > are
| > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then
| > install
| > | > it
| > | > via 'R CMD INSTALL'.
| > | >
| > | > Are you doing that?
| > | >
| > | > | The package passes devtools:check() and compiles fine.
| > | >
| > | > Well if it passes check you should be fine.
| > | >
| > | > | For instance, for one of those classes, LshParameterSetter, I do
| > | > |
| > | > |
| > | > | RCPP_EXPOSED_CLASS(LshParameterSetter)
| > | > |
| > | > | RCPP_MODULE(mod_params) {
| > | > | class_<LshParameterSetter>("LshParameterSetter")
| > | > |
| > | > | .constructor<int,int>()
| > | > |
| > | > | .method("withDefaults", &LshParameterSetter::withDefaults,
| > | > | "Fill with defaults")
| > | > | .method("distance", &LshParameterSetter::distance,
| > | > | "Set distance metric")
| > | > | .method("numHashFunctions", &LshParameterSetter::
| > numHashFunctions,
| > | > | "Set numbers of hash functions and tables")
| > | > | .method("numHashTables", &LshParameterSetter::numHashTables,
| > | > | "Set numbers of hash functions and tables")
| > | > | .method("storage", &LshParameterSetter::storage,
| > | > | "Set LSH Hash Table Storage type")
| > | > | .method("family", &LshParameterSetter::family,
| > | > | "Set LSH Family")
| > | > | .method("rotations", &LshParameterSetter::rotations,
| > | > | "Number of rotations")
| > | > | .method("asList", &LshParameterSetter::asList,
| > | > | "List of parameter values by name")
| > | > | ;
| > | > | }
| > | > |
| > | > | and similarly for the other classes. This module is loaded in the R
| > code
| > | > | with
| > | > | Rcpp::loadModule("mod_params", TRUE)
| > | > | and similarly for the other modules.
| > | > |
| > | > | The installed package has the .rdb, .rdx, and .so file, so it's hard
| > to
| > | > see
| > | > | if something has been lost in the installation. There's no
| > indication
| > | > what
| > | > | is wrong, but after loading the installed package, a class like
| > | > | LshParameterSetter
| > | > | is not defined. (Loading from within the package source directory it
| > is,
| > | >
| > | > What do you mean by 'loading from within the package source directory'
| > ?
| > | >
| > | > | showing
| > | > |
| > | > | > > LshParameterSetter
| > | > | > C++ class 'LshParameterSetter' <0x7fde72fb1940>
| > | > | > Constructors:
| > | > | > LshParameterSetter(int, int)
| > | > | >
| > | > | > Fields: ...
| > | > | >
| > | > | as it should be.) I've done this with simple skeletons and not had a
| > | > | similar problem. But I don't see a qualitative difference between
| > these
| > | > | cases.
| > | > |
| > | > | The full package code is at https://github.com/genovese/falconnr
| > | > | and the session info is below.
| > | >
| > | > Thanks for posting the link. Not a small package ...
| > | >
| > | > | I'd appreciate any pointers on this, as the problem is unclear to me,
| > | > | especially since it worked without problem last year. I'm happy to
| > | > | provide any useful information that I might have omitted here.
| > | >
| > | > Not sure what is going on there. Package clearly builds. Testing
| > complains
| > | > some but hey, it is not a CRAN package yet...
| > | >
| > | > You have
| > | >
| > | > falconnr::LshTable
| > | >
| > | > visible, but (what you probably want) only un-exported:
| > | >
| > | > falconnr:::LshNnTable
| > | > falconnr:::LshParameterSetter
| > | >
| > | > I have been using Modules for about as long as we've had them, but I
| > have
| > | > always been a little mystified by the various UPPERCASE_MACROS
| > exporting
| > | > things. Looking at what was probably my first Modules package,
| > RcppBDT, I
| > | > seem to "simply" do
| > | >
| > | > RCPP_MODULE(bdtDdMod) {
| > | > Rcpp::class_<bdtDd>("bdtDd")
| > | > // ... stuff omitted
| > | >
| > | > and nothing else for the four or so different modules there. That
| > worked.
| > | > I
| > | > think you may rely on a helper macro which may have gotten stale as the
| > | > _underlying R behaviour_ may have changed. I think I use one or two
| > of the
| > | > exporter macros in other places (ie RcppAnnoy) so you could try that.
| > | >
| > | > In general, I always recommend to simplify as much as you can.
| > | >
| > | > Hope this helps, Dirk
| > | >
| > | > | Thanks for your help, Chris
| > | > |
| > | > | > sessionInfo()
| > | > | R version 3.3.2 (2016-10-31)
| > | > | Platform: x86_64-apple-darwin13.4.0 (64-bit)
| > | > | Running under: macOS Sierra 10.12.6
| > | > |
| > | > | locale:
| > | > | [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
| > | > |
| > | > | attached base packages:
| > | > | [1] grid stats graphics grDevices utils datasets
| > methods
| > | > | base
| > | > |
| > | > | other attached packages:
| > | > | [1] falconnr_0.0.0.9001 MASS_7.3-45 lattice_0.20-34
| > | > |
| > | > | loaded via a namespace (and not attached):
| > | > | [1] Rcpp_0.12.13 codetools_0.2-15
| > | > | _______________________________________________
| > | > | Rcpp-devel mailing list
| > | > | Rcpp-devel at lists.r-forge.r-project.org
| > | > | https://lists.r-forge.r-project.org/cgi-bin/mailman/
| > listinfo/rcpp-devel
| > | >
| > | > --
| > | > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| > | >
| >
| > --
| > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| >
--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
More information about the Rcpp-devel
mailing list