<div dir="ltr"><div><div>Thanks again, Dirk. Great suggestions; I will certainly try that</div><div>approach.<br></div><div><br></div>I definitely agree that modules are very useful, and I very much</div><div>appreciate all your (and your team's) efforts in bringing this to the <br></div><div>community.</div><div><br></div><div>Regards, Chris</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 9, 2017 at 12:30 PM, Dirk Eddelbuettel <span dir="ltr"><<a href="mailto:edd@debian.org" target="_blank">edd@debian.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
On 9 November 2017 at 11:34, Christopher Genovese wrote:<br>
| Sorry, let me be clear. I've done it both ways with the same basic<br>
| result (for #2). Specifically, doing the R CMD CHECK, R CMD BUILD,<br>
| R CMD INSTALL sequence gives an installed package with the<br>
| the two classes invisible. So I don't think this is a devtools issue.<br>
|<br>
| This is rather frustrating, I agree, especially as it worked previously.<br>
<br>
</span>Agreed. But some things also changed in R ... and I am not entirely what the<br>
actual cause was for the change you see.<br>
<span class=""><br>
| Is there any documentation (preferably short of going through the Rcpp<br>
| modules<br>
| code) on the process by which these symbols get exported into R. The<br>
| symbols are in the .so file, but for instance, where/when would the<br>
| function R_init_falconnr function, which is created by<br>
| Rcpp::compileAttributes<br>
| and seems to load the modules, be called?<br>
<br>
</span>Well, I also find that combining Modules and compileAttributes() can be<br>
finicky. If output for this was was generated, you could place it e.g. in<br>
src/init.c -- R calls the resulting function on package load. But, and that<br>
is a big 'but' you don't have to unless you say .registration=TRUE in<br>
useDynLib() in NAMESPACE. And I seem to recall I actually wrote the<br>
src/init.c for RcppAnnoy "by hand" as the generator in R itself does not know<br>
about Modules. We may need to add a tool here.<br>
<br>
And again, note that you do not need src/init.c -- you only need it if you<br>
set .registration=TRUE. So as first step, I'd say 'do not do that'. I know<br>
it sounds silly -- but if I were you I'd start from the working Modules<br>
example eg in the Rcpp unittests and would then re-add my code. Slowly, and<br>
step by step. That way you will know which change breaks it.<br>
<br>
And by all I means I don't claim this is ideal -- if we learn something along<br>
the way we can try to make Modules better. It remains a useful yet somewhat<br>
obscure and underdocumented tool.<br>
<span class="HOEnZb"><font color="#888888"><br>
Dirk<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
|<br>
| Thanks, Chris<br>
|<br>
|<br>
| On Thu, Nov 9, 2017 at 9:59 AM, Dirk Eddelbuettel <<a href="mailto:edd@debian.org">edd@debian.org</a>> wrote:<br>
|<br>
| ><br>
| > On 9 November 2017 at 09:32, Christopher Genovese wrote:<br>
| > | Thanks, Dirk.<br>
| > |<br>
| > | What exactly do you mean by 1. and 2. ? Per 'Writing R Extensions' we<br>
| > are<br>
| > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then<br>
| > install<br>
| > | > it<br>
| > | > via 'R CMD INSTALL'.<br>
| > | ><br>
| > | > Are you doing that?<br>
| > | ><br>
| > |<br>
| > | Yes. I usually use devtools, but I've done it directly as well.<br>
| ><br>
| > Take this up with the devtools developers then, please.<br>
| ><br>
| > I operate with R command, and possibly very thin littler wrappers such as<br>
| > build.r and rcc.r to, respectively, build and check.<br>
| ><br>
| > | In #1, for instance: I go to the package source directory, start R,<br>
| > | do devtools::build and devtools::check. This builds and compiles<br>
| > | the package. After library() at this point, the module components<br>
| > | are visible, e.g., LshParameterSetter.<br>
| > |<br>
| > | In #2, for instance, I do devtools::install() or<br>
| > devtools::install_github()<br>
| > | or similar.<br>
| > | Quit R, go to some other directory, restart R, and call library(). The R<br>
| > | entry<br>
| > | points are visible, but none of the module classes.<br>
| > |<br>
| > | I'm unclear as to what in the installation/export process can explain the<br>
| > | difference<br>
| > | between #1 and #2.<br>
| ><br>
| > Again, I have no idea what devtools does here. Other people may.<br>
| ><br>
| > The Rcpp documentation does not cover devtools so you are on your own.<br>
| ><br>
| > | Before emailing yesterday, I had tried removing the<br>
| > RCPP_EXPOSED_CLASS()'s,<br>
| > | but then the code does not compile. There is a static_assert error in<br>
| > | wrap_dispatch_unknown_iterable in Rcpp's wrap.h (line 520). I did not<br>
| > see an<br>
| > | easy way to fix that. (I vaguely recall that I only included them<br>
| > | originally because<br>
| > | I had to.)<br>
| ><br>
| > I think you need those. I just glance at the different variants on the<br>
| > train<br>
| > in, they are mostly equivalent and just layers of each other.<br>
| ><br>
| > | I will look RcppAnnoy and see if I can find a way around using these<br>
| > | macros.<br>
| ><br>
| > Sounds good.<br>
| ><br>
| > Modules can be frustrating. I just tried with some larger code at work,<br>
| > and<br>
| > it doesn't load :-/ Something else may get in the way but hard to tell.<br>
| ><br>
| > Rcpp Attributes is just fine.<br>
| ><br>
| > Dirk<br>
| ><br>
| > | Thanks again, Chris<br>
| > |<br>
| > |<br>
| > |<br>
| > |<br>
| > | On Thu, Nov 9, 2017 at 7:55 AM, Dirk Eddelbuettel <<a href="mailto:edd@debian.org">edd@debian.org</a>><br>
| > wrote:<br>
| > |<br>
| > | ><br>
| > | > On 9 November 2017 at 01:54, Christopher Genovese wrote:<br>
| > | > | Hello,<br>
| > | > |<br>
| > | > | I have a package under development that uses Rcpp modules to expose<br>
| > | > | some C++ classes. I hadn't touched the package for some months (close<br>
| > | > | to a year). At that time the package compiled and installed without<br>
| > | > problem<br>
| > | > | and worked well. I could access the classes as expected. Recently,<br>
| > having<br>
| > | > | updated Rcpp and R in the meantime, I reinstalled the package with<br>
| > | > | the following result:<br>
| > | > |<br>
| > | > | 1. If building and loading from within the package directory, all<br>
| > | > works<br>
| > | > | fine.<br>
| > | > | 2. When installing the package, either from github, locally, or<br>
| > | > | otherwise,<br>
| > | > | the modules are *not* visible.<br>
| > | ><br>
| > | > What exactly do you mean by 1. and 2. ? Per 'Writing R Extensions' we<br>
| > are<br>
| > | > meant to create a tar.gz in either way via 'R CMD BUILD' and then<br>
| > install<br>
| > | > it<br>
| > | > via 'R CMD INSTALL'.<br>
| > | ><br>
| > | > Are you doing that?<br>
| > | ><br>
| > | > | The package passes devtools:check() and compiles fine.<br>
| > | ><br>
| > | > Well if it passes check you should be fine.<br>
| > | ><br>
| > | > | For instance, for one of those classes, LshParameterSetter, I do<br>
| > | > |<br>
| > | > |<br>
| > | > | RCPP_EXPOSED_CLASS(<wbr>LshParameterSetter)<br>
| > | > |<br>
| > | > | RCPP_MODULE(mod_params) {<br>
| > | > | class_<LshParameterSetter>("<wbr>LshParameterSetter")<br>
| > | > |<br>
| > | > | .constructor<int,int>()<br>
| > | > |<br>
| > | > | .method("withDefaults", &LshParameterSetter::<wbr>withDefaults,<br>
| > | > | "Fill with defaults")<br>
| > | > | .method("distance", &LshParameterSetter::distance,<br>
| > | > | "Set distance metric")<br>
| > | > | .method("numHashFunctions", &LshParameterSetter::<br>
| > numHashFunctions,<br>
| > | > | "Set numbers of hash functions and tables")<br>
| > | > | .method("numHashTables", &LshParameterSetter::<wbr>numHashTables,<br>
| > | > | "Set numbers of hash functions and tables")<br>
| > | > | .method("storage", &LshParameterSetter::storage,<br>
| > | > | "Set LSH Hash Table Storage type")<br>
| > | > | .method("family", &LshParameterSetter::family,<br>
| > | > | "Set LSH Family")<br>
| > | > | .method("rotations", &LshParameterSetter::<wbr>rotations,<br>
| > | > | "Number of rotations")<br>
| > | > | .method("asList", &LshParameterSetter::asList,<br>
| > | > | "List of parameter values by name")<br>
| > | > | ;<br>
| > | > | }<br>
| > | > |<br>
| > | > | and similarly for the other classes. This module is loaded in the R<br>
| > code<br>
| > | > | with<br>
| > | > | Rcpp::loadModule("mod_params", TRUE)<br>
| > | > | and similarly for the other modules.<br>
| > | > |<br>
| > | > | The installed package has the .rdb, .rdx, and .so file, so it's hard<br>
| > to<br>
| > | > see<br>
| > | > | if something has been lost in the installation. There's no<br>
| > indication<br>
| > | > what<br>
| > | > | is wrong, but after loading the installed package, a class like<br>
| > | > | LshParameterSetter<br>
| > | > | is not defined. (Loading from within the package source directory it<br>
| > is,<br>
| > | ><br>
| > | > What do you mean by 'loading from within the package source directory'<br>
| > ?<br>
| > | ><br>
| > | > | showing<br>
| > | > |<br>
| > | > | > > LshParameterSetter<br>
| > | > | > C++ class 'LshParameterSetter' <0x7fde72fb1940><br>
| > | > | > Constructors:<br>
| > | > | > LshParameterSetter(int, int)<br>
| > | > | ><br>
| > | > | > Fields: ...<br>
| > | > | ><br>
| > | > | as it should be.) I've done this with simple skeletons and not had a<br>
| > | > | similar problem. But I don't see a qualitative difference between<br>
| > these<br>
| > | > | cases.<br>
| > | > |<br>
| > | > | The full package code is at <a href="https://github.com/genovese/falconnr" rel="noreferrer" target="_blank">https://github.com/genovese/<wbr>falconnr</a><br>
| > | > | and the session info is below.<br>
| > | ><br>
| > | > Thanks for posting the link. Not a small package ...<br>
| > | ><br>
| > | > | I'd appreciate any pointers on this, as the problem is unclear to me,<br>
| > | > | especially since it worked without problem last year. I'm happy to<br>
| > | > | provide any useful information that I might have omitted here.<br>
| > | ><br>
| > | > Not sure what is going on there. Package clearly builds. Testing<br>
| > complains<br>
| > | > some but hey, it is not a CRAN package yet...<br>
| > | ><br>
| > | > You have<br>
| > | ><br>
| > | > falconnr::LshTable<br>
| > | ><br>
| > | > visible, but (what you probably want) only un-exported:<br>
| > | ><br>
| > | > falconnr:::LshNnTable<br>
| > | > falconnr:::LshParameterSetter<br>
| > | ><br>
| > | > I have been using Modules for about as long as we've had them, but I<br>
| > have<br>
| > | > always been a little mystified by the various UPPERCASE_MACROS<br>
| > exporting<br>
| > | > things. Looking at what was probably my first Modules package,<br>
| > RcppBDT, I<br>
| > | > seem to "simply" do<br>
| > | ><br>
| > | > RCPP_MODULE(bdtDdMod) {<br>
| > | > Rcpp::class_<bdtDd>("bdtDd")<br>
| > | > // ... stuff omitted<br>
| > | ><br>
| > | > and nothing else for the four or so different modules there. That<br>
| > worked.<br>
| > | > I<br>
| > | > think you may rely on a helper macro which may have gotten stale as the<br>
| > | > _underlying R behaviour_ may have changed. I think I use one or two<br>
| > of the<br>
| > | > exporter macros in other places (ie RcppAnnoy) so you could try that.<br>
| > | ><br>
| > | > In general, I always recommend to simplify as much as you can.<br>
| > | ><br>
| > | > Hope this helps, Dirk<br>
| > | ><br>
| > | > | Thanks for your help, Chris<br>
| > | > |<br>
| > | > | > sessionInfo()<br>
| > | > | R version 3.3.2 (2016-10-31)<br>
| > | > | Platform: x86_64-apple-darwin13.4.0 (64-bit)<br>
| > | > | Running under: macOS Sierra 10.12.6<br>
| > | > |<br>
| > | > | locale:<br>
| > | > | [1] en_US.UTF-8/en_US.UTF-8/en_US.<wbr>UTF-8/C/en_US.UTF-8/en_US.UTF-<wbr>8<br>
| > | > |<br>
| > | > | attached base packages:<br>
| > | > | [1] grid stats graphics grDevices utils datasets<br>
| > methods<br>
| > | > | base<br>
| > | > |<br>
| > | > | other attached packages:<br>
| > | > | [1] falconnr_0.0.0.9001 MASS_7.3-45 lattice_0.20-34<br>
| > | > |<br>
| > | > | loaded via a namespace (and not attached):<br>
| > | > | [1] Rcpp_0.12.13 codetools_0.2-15<br>
| > | > | ______________________________<wbr>_________________<br>
| > | > | Rcpp-devel mailing list<br>
| > | > | <a href="mailto:Rcpp-devel@lists.r-forge.r-project.org">Rcpp-devel@lists.r-forge.r-<wbr>project.org</a><br>
| > | > | <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/" rel="noreferrer" target="_blank">https://lists.r-forge.r-<wbr>project.org/cgi-bin/mailman/</a><br>
| > listinfo/rcpp-devel<br>
| > | ><br>
| > | > --<br>
| > | > <a href="http://dirk.eddelbuettel.com" rel="noreferrer" target="_blank">http://dirk.eddelbuettel.com</a> | @eddelbuettel | <a href="mailto:edd@debian.org">edd@debian.org</a><br>
| > | ><br>
| ><br>
| > --<br>
| > <a href="http://dirk.eddelbuettel.com" rel="noreferrer" target="_blank">http://dirk.eddelbuettel.com</a> | @eddelbuettel | <a href="mailto:edd@debian.org">edd@debian.org</a><br>
| ><br>
<br>
--<br>
<a href="http://dirk.eddelbuettel.com" rel="noreferrer" target="_blank">http://dirk.eddelbuettel.com</a> | @eddelbuettel | <a href="mailto:edd@debian.org">edd@debian.org</a><br>
</div></div></blockquote></div><br></div>