[Rcpp-devel] RcppArmadillo BLAS/LAPACK force number of threads to 1

[Scott Ritchie]

Thanks Dirk and Yixuan,

I think I'm misstating the question.

I'm developing a package which makes use of LAPACK routines through
RcppArmadillo, and I'm handling the parallelism from within R.

At the moment it looks like I have to write a vignette, telling the user to
monitor their CPU usage with a tool like `top`, and if each thread is using
more than 100%:

   1. try setting the appropriate environment variables
   (OPENBLAS_NUM_THREADS=1, GOTO_NUM_THREADS=1,  OMP_NUM_THREADS=1) depending
   on their BLAS library
   2. Switch to a single-threaded BLAS library.

Ideally, I'd like to be able to abstract away these details from the user,
so they don't need to know whether they have a multithreaded or
single-threaded version of BLAS installed.

My first thought was to look up the multithreaded BLAS libraries, and set
the environment variables accordingly from my main R routine. However, it
seems like the changes made in Sys.setenv are not reflected in the
RcppArmadillo code.

I hope this makes things clearer,

Scott



On 12 August 2014 09:23, Yixuan Qiu <yixuan.qiu at cos.name> wrote:

> Good point by Dirk.
>
> And why not just adding a line "export OPENBLAS_NUM_THREADS=1" to your
> .bashrc file, or compiling OpenBlas again by setting NUM_THREADS = 1 in the
> Makefile.rule?
>
>
> Best,
> Yixuan
>
>
> 2014-08-11 18:55 GMT-04:00 Dirk Eddelbuettel <edd at debian.org>:
>
>
>> Hi Scott,
>>
>>
>> On 12 August 2014 at 08:43, Scott Ritchie wrote:
>> | Hi Everyone,
>> |
>> | I'm having trouble with restricting the Armadillo function "svd_econ"
>> to run on
>> | a single thread.
>> |
>> | I am able to restrict it if I `export OPENBLAS_NUM_THREADS=1` before
>> opening an
>> | R session, but not from within R (`Sys.setenv(OPENBLAS_NUM_THREADS=1)`
>> has no
>> | effect).
>> |
>> | I'm wondering if theirs an option native to armadillo/RcppArmadillo
>> that I'm
>> | missing to enable me to:
>> |
>> |   • Restrict the number of threads used to 1
>> |   • In a library agnostic manner (i.e. if the user has a different
>> LAPACK or
>> |     BLAS library installed).
>> |
>> | or if I can "trick" Rcpp into thinking the machine only has 1 core?
>>
>> I fear you may be misstating / misdiagnosing the problem.
>>
>> I am fairly certain that there is not a single line in Rcpp or
>> (Rcpp)Armadillo which sets thread counts for your LAPACK / BLAS.
>>
>> Rather, and this is a point that I tried to make in the 'Benchmarking GPUs
>> and CPUs on Debian-based systems' package (gcbd on CRAN etc) and its
>> vignette
>> is that __BLAS and LAPACK sit behind a standardized interface__ and can be
>> installed / swapped via plug and play. (Which in turn enables the
>> benchmarking ...)
>>
>> If you're on a Debian-based system, just remove openblas and use atlas.
>> Case
>> closed.  Atlas will not multithread (unless something changed recently).
>>
>> There is also a helper function or package somewhere, and a post by
>> Claudia
>> Beleites on StackOverflow, which show the explicit function call to reset
>> OpenBLAS in its count.  But if everything else fails -- just use a
>> different
>> BLAS.  I sometimes do that too when I use multicore / parallel to launch
>> multiple R jobs and each one of those should not spawn additional linear
>> algebra threads.
>>
>> Hope this helps,  Dirk
>>
>>
>>
>> | Regards,
>> |
>> | Scott Ritchie
>> | _______________________________________________
>> | Rcpp-devel mailing list
>> | Rcpp-devel at lists.r-forge.r-project.org
>> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>> _______________________________________________
>> Rcpp-devel mailing list
>> Rcpp-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
>
>
>
> --
> Yixuan Qiu <yixuan.qiu at cos.name>
> Department of Statistics,
> Purdue University
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20140812/88b15b65/attachment-0001.html>