[Rcpp-devel] Shared code in C++ files

Thu Apr 14 23:28:36 CEST 2016

Hi All,

I am very grateful for your input/recommendations.  Let me try to wrap my
head around how to do this with a single package and get your approval on
an MWE.

Best regards,

Martin Lysy
Assistant Professor of Statistics
Department of Statistics and Actuarial Science
University of Waterloo

On Thu, Apr 14, 2016 at 12:33 PM, Kyle Baron <kyleb at metrumrg.com> wrote:

> Hi Martin -
>
> I, too, am unsure about the mechanics of what you are wanting to do.  But
> this piece:
>
>  I can think of many applications in which the useR could supply a minimal
>> amount of C++ code (e.g., a log-likelihood function) to hook in with a
>> large amount of code provided by the developer in order to speed things up
>> considerably.
>
>
> looks similar to what I am doing in a package that I maintain:  The user
> writes some C++ code, compiles it via `R CMD SHLIB` and loads the shared
> object into the current R session with `dyn.load()` (actually, I provide an
> R function or ten to help manage that).  This is that "minimal amount of
> C++ code" you were referring to.  The user then hands off  the address of
> those symbols (`?getNativeSymbolInfo`) to the developer-provided code base
> (in an R package), turn that into pointer to the function(s) that can be
> called by the code-base as needed.  My package is written with Rcpp and
> regular old C++;  all of the data that gets handed off to the user-defined
> functions are C++ data types so I don't need to link back to Rcpp.  But
> it's easy enough to link up with Rcpp  (or whatever else) when compiling
> the shared object.
>
> I'm pretty sure Dirk has similar(?) setup in his RcppDE package where
> users can provide a compiled function for DEoptim to call when needed.
> deSolve is another package that does something like this, but no Rcpp
> involvement.
>
> Of course, this requires some coordination between what is going on in the
> shared object and the main code base, but it's not unreasonably complicated
> to work out.  The flexibility you get is well worth it.   And absolutely no
> problem juggling multiple user-defined functions in the same R session.
>
> Best Regards,
>
> Kyle Baron
> Metrum Research Group
> https://github.com/metrumresearchgroup/mrgsolve
>
>
> On Thu, Apr 14, 2016 at 10:55 AM, JJ Allaire <jj.allaire at gmail.com> wrote:
>
>> I don't know enough about the mechanics of your scenario to comment
>> intelligently, but I would in general agree with Dirk that if there is a
>> way to put something like this in a package that's the way it ought to be
>> done. You could certainly dispatch to different C++ functions within a
>> package using a variety of mechanisms including C++ templates, C-style
>> function pointers, etc. (the mechanics of doing so for your situation I'm
>> unsure of, but it should be possible).
>>
>> On Thu, Apr 14, 2016 at 12:25 AM, Martin Lysy <mlysy at uwaterloo.ca> wrote:
>>
>>> Hello Dirk,
>>>
>>> For my specific purpose I have some reservations about package
>>> creation.  Here is a description of the project I have in mind.
>>>
>>> I would like to use Rcpp to create a set of generic MCMC algorithms,
>>> into which a useR could easily hook their own target distributions written
>>> in C++ with minimal effort.  For example, I would provide the following
>>> files:
>>>
>>> --------------------------------------
>>> GibbsSampler.cpp
>>>
>>> //[[Rcpp::export]]
>>> NumericMatrix GibbsSampler(int nIter, NumericVector xInit, List
>>> tuningParams) {
>>>   // run some flavor of the Gibbs sampler
>>> }
>>> ---------------------------------------
>>> GibbsSampler.h
>>>
>>> double logDensity(NumericVector x);
>>> ---------------------------------------
>>>
>>> Then the useR would write MyDensityA.cpp, which contains the definition
>>> of logDensity, compile the three files, and have a Gibbs sampler for their
>>> specific density function written in C++ and ready to go in R.  However, a
>>> useR might wish to use the GibbsSampler for a different density tomorrow.
>>> They would have a different definition of logDensity in MyDensityB.cpp.
>>> Ideally, the useR would have access to Gibbs samplers for both densities in
>>> the same R session.
>>>
>>> I can think of two ways of doing this with Rcpp:
>>>
>>> 1.  Compile with sourceCpp (this is what I'm currently doing).  There's
>>> the minor issue of giving separate R names to each Gibbs sampler, but there
>>> are many ways around that.  The major issue is that sourceCpp only accepts
>>> a single .cpp file (or at least as far as my attempts were unsuccessful in
>>> the original post).  So I'm stuck text-processing a bunch of .cpp and .h
>>> files together (the actual project I'm working on has about a dozen of
>>> each).
>>>
>>> 2.  Compile an entire R package for each of MyDensityA and MyDensityB.
>>> However, it seems somewhat cumbersome to have a package loaded for every
>>> density function in the workspace.  Moreover, naming conflicts are a bit
>>> more tricky.  Right now (with sourceCpp), I'm using an interface of the form
>>>
>>> smpA <- gibbs.sampler(density = MyDensityA, niter = 1e4)
>>> smpB <- gibbs.sampler(density = MyDensityB, niter = 1e4)
>>>
>>> This is to align with things like lm(formula = MyModel).  However, I
>>> can't quite see how to do this with separate packages loaded.  Rather it
>>> seems I'd need something like
>>>
>>> smpA <- gibbs.sampler.MyDensityA(niter = 1e4)
>>> smpB <- gibbs.sampler.MyDensityB(niter = 1e4)
>>>
>>> However, to do this with packages I feel like I would still have to do
>>> some text replacement, which I'm already doing with the sourceCpp approach.
>>>
>>> In summary, I am not opposed to package creation, but I hope you can see
>>> my reservations at taking this route.  Perhaps you could please suggest a
>>> way to achieve what I'm after with separate Rcpp packages for each density
>>> function.  I take it from your reluctance to answer my original post that
>>> Rcpp only supports compilation of multiple files through the package
>>> creation protocol.  I can think of many applications in which the useR
>>> could supply a minimal amount of C++ code (e.g., a log-likelihood function)
>>> to hook in with a large amount of code provided by the developer in order
>>> to speed things up considerably.  So in my opinion it would be worthwhile
>>> to devise a mechanism to do this correctly.
>>>
>>> Best regards,
>>>
>>>
>>> Martin Lysy
>>> Assistant Professor of Statistics
>>> Department of Statistics and Actuarial Science
>>> University of Waterloo
>>>
>>> On Wed, Apr 13, 2016 at 5:34 PM, Dirk Eddelbuettel <edd at debian.org>
>>> wrote:
>>>
>>>>
>>>> Martin,
>>>>
>>>> Please please please look into creating a package.
>>>>
>>>> If you use RStudio:  File -> New Project -> (New or Existing) Directory
>>>> ->
>>>> Package and then select Rcpp.
>>>>
>>>> If not, consider install the (very tiny) pkgKitten package and call
>>>> Rcpp.package.skeleton() from Rcpp itself (but enhanced by pkgKitten if
>>>> present) for a saner package.
>>>>
>>>> Cheers, Dirk
>>>>
>>>> --
>>>> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>>>>
>>>
>>>
>>> _______________________________________________
>>> Rcpp-devel mailing list
>>> Rcpp-devel at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>>>
>>
>>
>> _______________________________________________
>> Rcpp-devel mailing list
>> Rcpp-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>>
>
>
>
> --
> Kyle Baron
> Metrum Research Group
> 860-735-7043, Ext. 202
> kyleb at metrumrg.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20160414/68594805/attachment-0001.html>