[Rcpp-devel] Rcpp module visible after check but not after install
Christopher Genovese
genovese at cmu.edu
Thu Nov 9 19:41:00 CET 2017
Thanks again, Dirk. Great suggestions; I will certainly try that
approach.
I definitely agree that modules are very useful, and I very much
appreciate all your (and your team's) efforts in bringing this to the
community.
Regards, Chris
On Thu, Nov 9, 2017 at 12:30 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>
> On 9 November 2017 at 11:34, Christopher Genovese wrote:
> | Sorry, let me be clear. I've done it both ways with the same basic
> | result (for #2). Specifically, doing the R CMD CHECK, R CMD BUILD,
> | R CMD INSTALL sequence gives an installed package with the
> | the two classes invisible. So I don't think this is a devtools issue.
> |
> | This is rather frustrating, I agree, especially as it worked previously.
>
> Agreed. But some things also changed in R ... and I am not entirely what
> the
> actual cause was for the change you see.
>
> | Is there any documentation (preferably short of going through the Rcpp
> | modules
> | code) on the process by which these symbols get exported into R. The
> | symbols are in the .so file, but for instance, where/when would the
> | function R_init_falconnr function, which is created by
> | Rcpp::compileAttributes
> | and seems to load the modules, be called?
>
> Well, I also find that combining Modules and compileAttributes() can be
> finicky. If output for this was was generated, you could place it e.g. in
> src/init.c -- R calls the resulting function on package load. But, and
> that
> is a big 'but' you don't have to unless you say .registration=TRUE in
> useDynLib() in NAMESPACE. And I seem to recall I actually wrote the
> src/init.c for RcppAnnoy "by hand" as the generator in R itself does not
> know
> about Modules. We may need to add a tool here.
>
> And again, note that you do not need src/init.c -- you only need it if you
> set .registration=TRUE. So as first step, I'd say 'do not do that'. I
> know
> it sounds silly -- but if I were you I'd start from the working Modules
> example eg in the Rcpp unittests and would then re-add my code. Slowly,
> and
> step by step. That way you will know which change breaks it.
>
> And by all I means I don't claim this is ideal -- if we learn something
> along
> the way we can try to make Modules better. It remains a useful yet
> somewhat
> obscure and underdocumented tool.
>
> Dirk
>
> |
> | Thanks, Chris
> |
> |
> | On Thu, Nov 9, 2017 at 9:59 AM, Dirk Eddelbuettel <edd at debian.org>
> wrote:
> |
> | >
> | > On 9 November 2017 at 09:32, Christopher Genovese wrote:
> | > | Thanks, Dirk.
> | > |
> | > | What exactly do you mean by 1. and 2. ? Per 'Writing R Extensions'
> we
> | > are
> | > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then
> | > install
> | > | > it
> | > | > via 'R CMD INSTALL'.
> | > | >
> | > | > Are you doing that?
> | > | >
> | > |
> | > | Yes. I usually use devtools, but I've done it directly as well.
> | >
> | > Take this up with the devtools developers then, please.
> | >
> | > I operate with R command, and possibly very thin littler wrappers such
> as
> | > build.r and rcc.r to, respectively, build and check.
> | >
> | > | In #1, for instance: I go to the package source directory, start R,
> | > | do devtools::build and devtools::check. This builds and compiles
> | > | the package. After library() at this point, the module components
> | > | are visible, e.g., LshParameterSetter.
> | > |
> | > | In #2, for instance, I do devtools::install() or
> | > devtools::install_github()
> | > | or similar.
> | > | Quit R, go to some other directory, restart R, and call library().
> The R
> | > | entry
> | > | points are visible, but none of the module classes.
> | > |
> | > | I'm unclear as to what in the installation/export process can
> explain the
> | > | difference
> | > | between #1 and #2.
> | >
> | > Again, I have no idea what devtools does here. Other people may.
> | >
> | > The Rcpp documentation does not cover devtools so you are on your own.
> | >
> | > | Before emailing yesterday, I had tried removing the
> | > RCPP_EXPOSED_CLASS()'s,
> | > | but then the code does not compile. There is a static_assert error in
> | > | wrap_dispatch_unknown_iterable in Rcpp's wrap.h (line 520). I did not
> | > see an
> | > | easy way to fix that. (I vaguely recall that I only included them
> | > | originally because
> | > | I had to.)
> | >
> | > I think you need those. I just glance at the different variants on the
> | > train
> | > in, they are mostly equivalent and just layers of each other.
> | >
> | > | I will look RcppAnnoy and see if I can find a way around using these
> | > | macros.
> | >
> | > Sounds good.
> | >
> | > Modules can be frustrating. I just tried with some larger code at
> work,
> | > and
> | > it doesn't load :-/ Something else may get in the way but hard to
> tell.
> | >
> | > Rcpp Attributes is just fine.
> | >
> | > Dirk
> | >
> | > | Thanks again, Chris
> | > |
> | > |
> | > |
> | > |
> | > | On Thu, Nov 9, 2017 at 7:55 AM, Dirk Eddelbuettel <edd at debian.org>
> | > wrote:
> | > |
> | > | >
> | > | > On 9 November 2017 at 01:54, Christopher Genovese wrote:
> | > | > | Hello,
> | > | > |
> | > | > | I have a package under development that uses Rcpp modules to
> expose
> | > | > | some C++ classes. I hadn't touched the package for some months
> (close
> | > | > | to a year). At that time the package compiled and installed
> without
> | > | > problem
> | > | > | and worked well. I could access the classes as expected.
> Recently,
> | > having
> | > | > | updated Rcpp and R in the meantime, I reinstalled the package
> with
> | > | > | the following result:
> | > | > |
> | > | > | 1. If building and loading from within the package
> directory, all
> | > | > works
> | > | > | fine.
> | > | > | 2. When installing the package, either from github, locally,
> or
> | > | > | otherwise,
> | > | > | the modules are *not* visible.
> | > | >
> | > | > What exactly do you mean by 1. and 2. ? Per 'Writing R
> Extensions' we
> | > are
> | > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then
> | > install
> | > | > it
> | > | > via 'R CMD INSTALL'.
> | > | >
> | > | > Are you doing that?
> | > | >
> | > | > | The package passes devtools:check() and compiles fine.
> | > | >
> | > | > Well if it passes check you should be fine.
> | > | >
> | > | > | For instance, for one of those classes, LshParameterSetter, I do
> | > | > |
> | > | > |
> | > | > | RCPP_EXPOSED_CLASS(LshParameterSetter)
> | > | > |
> | > | > | RCPP_MODULE(mod_params) {
> | > | > | class_<LshParameterSetter>("LshParameterSetter")
> | > | > |
> | > | > | .constructor<int,int>()
> | > | > |
> | > | > | .method("withDefaults", &LshParameterSetter::withDefaults,
> | > | > | "Fill with defaults")
> | > | > | .method("distance", &LshParameterSetter::distance,
> | > | > | "Set distance metric")
> | > | > | .method("numHashFunctions", &LshParameterSetter::
> | > numHashFunctions,
> | > | > | "Set numbers of hash functions and tables")
> | > | > | .method("numHashTables", &LshParameterSetter::
> numHashTables,
> | > | > | "Set numbers of hash functions and tables")
> | > | > | .method("storage", &LshParameterSetter::storage,
> | > | > | "Set LSH Hash Table Storage type")
> | > | > | .method("family", &LshParameterSetter::family,
> | > | > | "Set LSH Family")
> | > | > | .method("rotations", &LshParameterSetter::rotations,
> | > | > | "Number of rotations")
> | > | > | .method("asList", &LshParameterSetter::asList,
> | > | > | "List of parameter values by name")
> | > | > | ;
> | > | > | }
> | > | > |
> | > | > | and similarly for the other classes. This module is loaded in
> the R
> | > code
> | > | > | with
> | > | > | Rcpp::loadModule("mod_params", TRUE)
> | > | > | and similarly for the other modules.
> | > | > |
> | > | > | The installed package has the .rdb, .rdx, and .so file, so it's
> hard
> | > to
> | > | > see
> | > | > | if something has been lost in the installation. There's no
> | > indication
> | > | > what
> | > | > | is wrong, but after loading the installed package, a class like
> | > | > | LshParameterSetter
> | > | > | is not defined. (Loading from within the package source
> directory it
> | > is,
> | > | >
> | > | > What do you mean by 'loading from within the package source
> directory'
> | > ?
> | > | >
> | > | > | showing
> | > | > |
> | > | > | > > LshParameterSetter
> | > | > | > C++ class 'LshParameterSetter' <0x7fde72fb1940>
> | > | > | > Constructors:
> | > | > | > LshParameterSetter(int, int)
> | > | > | >
> | > | > | > Fields: ...
> | > | > | >
> | > | > | as it should be.) I've done this with simple skeletons and not
> had a
> | > | > | similar problem. But I don't see a qualitative difference between
> | > these
> | > | > | cases.
> | > | > |
> | > | > | The full package code is at https://github.com/genovese/
> falconnr
> | > | > | and the session info is below.
> | > | >
> | > | > Thanks for posting the link. Not a small package ...
> | > | >
> | > | > | I'd appreciate any pointers on this, as the problem is unclear
> to me,
> | > | > | especially since it worked without problem last year. I'm happy
> to
> | > | > | provide any useful information that I might have omitted here.
> | > | >
> | > | > Not sure what is going on there. Package clearly builds. Testing
> | > complains
> | > | > some but hey, it is not a CRAN package yet...
> | > | >
> | > | > You have
> | > | >
> | > | > falconnr::LshTable
> | > | >
> | > | > visible, but (what you probably want) only un-exported:
> | > | >
> | > | > falconnr:::LshNnTable
> | > | > falconnr:::LshParameterSetter
> | > | >
> | > | > I have been using Modules for about as long as we've had them, but
> I
> | > have
> | > | > always been a little mystified by the various UPPERCASE_MACROS
> | > exporting
> | > | > things. Looking at what was probably my first Modules package,
> | > RcppBDT, I
> | > | > seem to "simply" do
> | > | >
> | > | > RCPP_MODULE(bdtDdMod) {
> | > | > Rcpp::class_<bdtDd>("bdtDd")
> | > | > // ... stuff omitted
> | > | >
> | > | > and nothing else for the four or so different modules there. That
> | > worked.
> | > | > I
> | > | > think you may rely on a helper macro which may have gotten stale
> as the
> | > | > _underlying R behaviour_ may have changed. I think I use one or
> two
> | > of the
> | > | > exporter macros in other places (ie RcppAnnoy) so you could try
> that.
> | > | >
> | > | > In general, I always recommend to simplify as much as you can.
> | > | >
> | > | > Hope this helps, Dirk
> | > | >
> | > | > | Thanks for your help, Chris
> | > | > |
> | > | > | > sessionInfo()
> | > | > | R version 3.3.2 (2016-10-31)
> | > | > | Platform: x86_64-apple-darwin13.4.0 (64-bit)
> | > | > | Running under: macOS Sierra 10.12.6
> | > | > |
> | > | > | locale:
> | > | > | [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-
> 8
> | > | > |
> | > | > | attached base packages:
> | > | > | [1] grid stats graphics grDevices utils datasets
> | > methods
> | > | > | base
> | > | > |
> | > | > | other attached packages:
> | > | > | [1] falconnr_0.0.0.9001 MASS_7.3-45 lattice_0.20-34
> | > | > |
> | > | > | loaded via a namespace (and not attached):
> | > | > | [1] Rcpp_0.12.13 codetools_0.2-15
> | > | > | _______________________________________________
> | > | > | Rcpp-devel mailing list
> | > | > | Rcpp-devel at lists.r-forge.r-project.org
> | > | > | https://lists.r-forge.r-project.org/cgi-bin/mailman/
> | > listinfo/rcpp-devel
> | > | >
> | > | > --
> | > | > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
> | > | >
> | >
> | > --
> | > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
> | >
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20171109/d993e508/attachment-0001.html>
More information about the Rcpp-devel
mailing list