<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>Hello,</div>
<div> </div>
<div>I'm writing my first package for a machine learning algorithm called self-organizing map where I use compiled code (with Rcpp) and parallelization (RcppParallel).<br/>
My computer uses Windows 10 (64 bit, 8 GB RAM) and I currently have a problem with the memory usage (shown in the Windows task manager) which keeps going up the longer the algorithm runs. The usage doesn't increase immediately, but after a couple of seconds and I only noticed it when I tried larger data sets. The memory is only freed by terminating/restarting the R session.</div>
<div> </div>
<div>What is somewhat strange is that the memory usage is not attributed to Rstudio or the R session (i.e. the memory usage in the task manager does not go up for the respective processes). According to RAMMap (which gives more information about memory usage on Windows) the used memory belongs to the "nonpaged pool". The RStudio profiler and lineprof did not seem to detect the memory leak (if I read the output correctly). So far I have rewritten parts of the C++ code to use references and pre-allocated memory, but it did not help.</div>
<div> </div>
<div>The main function in the package calls several smaller functions written in C++ and it seems that all of those functions play a role here, but I have found a function where this problem occurs consistently. It calculates the (squared) euclidean norm for each row of a given matrix (in parallel) with a boolean vector (oldColumns) specifying which columns should be used/ignored during this calculation:</div>
<div> </div>
<div><a href="https://pastebin.com/qgyzx0M7">https://pastebin.com/qgyzx0M7</a></div>
<div> </div>
<div>When I pasted this code into a new project, I have noticed that the problem only happens when I build (with devtools::build()) and install a package containing this function, regardless of whether I build a source package or a binary package. When I just sourceCpp a file with this function, no memory problems occur. So could this have anything to do with how I build packages? Until now I have followed the "R packages" book written by Hadley Wickham for this.</div>
<div> </div>
<div>Here is some R code which generates some test data and calls the function.</div>
<div> </div>
<div><a href="https://pastebin.com/c0RaeW9K">https://pastebin.com/c0RaeW9K</a></div>
<div> </div>
<div>Everytime I run this code (which takes a couple of minutes), the memory usage goes up by 4% - 6% which makes my package unusable for larger sets of data.<br/>
I have been stuck on this problem for a week now and any help would be appreciated.</div>
<div> </div>
<div>Thank you,<br/>
Thanh</div>
</div></div></body></html>