<div dir="ltr"><div><div><div>Hi<br><br></div>If you have the time to test, you will find that some things that you think will be faster are actually slower, while some seemingly unimportant things will give huge acceleration. <br>
<br></div><div>I predict that you can learn more about R efficiencies, esp. crossprod & tcrossprod, and after you do that, my guess it can go faster using Rcpp & a BLAS like openBLAS.<br></div><div><br></div><div>
I wish you'd try these and see how they affect performance. Then let us know what you find out. If you find anything clear, I'll add to my collection of speedup advices. <a href="http://pj.freefaculty.org/blog/?p=122">http://pj.freefaculty.org/blog/?p=122</a>.</div>
<br></div><div><div>1. Avoid repeated costly calculations. Please examine the use of exp(A). and 1/A in this part<br><br>E <- (exp(A) * (1 - 1 / A) + 1 / A) / (exp(A) - 1)<br></div><div><br></div>If A is a matrix by then, isn't exp a very slow (and imprecise: <a href="http://blogs.mathworks.com/cleve/2012/07/23/a-balancing-act-for-the-matrix-exponential/">http://blogs.mathworks.com/cleve/2012/07/23/a-balancing-act-for-the-matrix-exponential/</a>) operation, isn't it?<br>
You do it twice on the same matrix. <br><br></div>Did you know that DIVISION is much slower than multiplication in a modern CPU? Surprised me to learn that last year. Can't you re-arrange this so that 1/A is not calculated repeatedly?<br>
<div><div><div><div><div><div><div class="gmail_extra"><br></div><div class="gmail_extra">2. please examine this usage:<br> <br></div><div class="gmail_extra"> delta <- (t(X) %*% E - t(X2) %*% E2)<br>
W <- W + delta<br>
<br></div><div class="gmail_extra">This allocates a big bloc "delta" that you don't need to do. <br></div><div class="gmail_extra"><br>W <- W + (t(X) %*% E - t(X2) %*% E2)<br><br></div><div class="gmail_extra">
Please write back what you find out. I'm always eager to have clear "do this, don't do that" examples in the classroom.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">In my blog, look at item 3. That was a big shocker to me.<br>
</div><div class="gmail_extra"><br></div><div class="gmail_extra">pj<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 6, 2013 at 12:04 PM, Xavier Robin <span dir="ltr"><<a href="mailto:xavier@cbs.dtu.dk" target="_blank">xavier@cbs.dtu.dk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="im">On 11/6/13 6:38 PM, Romain Francois wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
This very much depends on the code but there is a good chance that RcppArmadillo will generate code making less data copies, etc ...<br>
<br>
Hard to say without seeing the code.<br>
<br>
Romain<br>
<br>
</blockquote></div>
Most of the code (or at least the slow, highly repeated parts) look like:<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
A <- t(c + t(W) %*% X)<br>
E <- (exp(A) * (1 - 1 / A) + 1 / A) / (exp(A) - 1)<br>
E[abs(A) < sqrt(.Machine$double.eps) * 2 ] <- 0.5<br>
<br>
B <- t(b + W %*% t(E))<br>
X2 <- 1 / (1 + exp(-B))<br>
<br>
A2 <- t(c + t(W) %*% X2)<br>
E2 <- (exp(A2) * (1 - 1 / A2) + 1 / A2) / (exp(A2) - 1)<br>
E2[abs(A2) < sqrt(.Machine$double.eps) * 2 ] <- 0.5<br>
<br>
delta <- (t(X) %*% E - t(X2) %*% E2)<br>
W <- W + delta<br>
</blockquote>
<br>
Where b and c are vectors, W and X matrices. All this is encapsulated in a function, that is called a few thousand times in a for loop, with some sanity checks. (But it didn't appear to have much impact on the speed... if I remove the matrix operations so it does nothing, it executes nearly instantly). I understand from Dirk and Douglas that it probably isn't going to make a huge difference, though (not by orders).<div class="im">
<br>
<br>
Thanks,<br>
Xavier<br>
<br>
-- <br>
Xavier Robin, PhD<br>
Cellular Signal Integration Group (C-SIG) - <a href="http://www.lindinglab.org" target="_blank">http://www.lindinglab.org</a><br></div></blockquote></div><br>-- <br>Paul E. Johnson<br>Professor, Political Science Assoc. Director<br>
1541 Lilac Lane, Room 504 Center for Research Methods<br>University of Kansas University of Kansas<br><a href="http://pj.freefaculty.org" target="_blank">http://pj.freefaculty.org</a> <a href="http://quant.ku.edu" target="_blank">http://quant.ku.edu</a>
</div></div></div></div></div></div></div></div>