<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><p id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; margin: 0px; line-height: auto;">Thank you very much. The documentation on := and set are really clear on this, thanks for pointing that out.</p> <br><div id="bloop_sign_1369765509787406848"><span style="font-family:helvetica,arial;font-size:13px"></span><div style="font-family: Helvetica; line-height: normal; ">-- </div><div style="font-family: Helvetica; line-height: normal; "><span style="font-family: arial; font-size: small; ">Alexandre Sieira</span><br style="font-family: arial; font-size: small; "><span style="font-family: arial; font-size: small; ">CISA, CISSP, ISO 27001 Lead Auditor</span><br style="font-family: arial; font-size: small; "><br style="font-family: arial; font-size: small; "><span style="font-family: arial; font-size: small; ">"The truth is rarely pure and never simple."</span><br style="font-family: arial; font-size: small; "><span style="font-family: arial; font-size: small; ">Oscar Wilde, The Importance of Being Earnest, 1895, Act I</span></div></div><p style="color:#A0A0A8;">On 28 de maio de 2013 at 15:11:04, Matthew Dowle (mdowle@mdowle.plus.com) wrote:</p> <blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;"><span><div>
<p> </p>
<p>Hi,</p>
<p>Yes this is expected because `[.data.table` is a function call with associated overhead. You don't want to loop calls to it. Consider all the arguments to `[.data.table` and all the checks that must be done for existence and type of arguments on each call. The idea is to give [.data.table meaty calls which it can chew on. It doesn't like tiny tasks one at a time.</p>
<p>`[[` on the other hand is an R primitive. It's part of the language. You can do very limited things with `[[` but in this case (looking up a single column by name or position) in a loop, that's best for the job. I use `[[` on data.table quite a lot.</p>
<p>This is also the very reason for set()'s existence: ?set says it's a 'loopable :=' because of the `[.data.table` overhead.</p>
<p>There's a feature request to detect when [.data.table is being looped, though :</p>
<p>https://r-forge.r-project.org/tracker/index.php?func=detail&aid=2028&group_id=240&atid=978</p>
<p>which would be more helpful of data.table, so at least it told you, rather than having to stumble across it.</p>
<p>Hope that helps,</p>
<p>Matthew</p>
<p> </p>
<p>On 28.05.2013 18:37, Alexandre Sieira wrote:</p>
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<p id="bloop_customfont" style="font-family: Helvetica,Arial; font-size: 13px; margin: 0px; line-height: auto;">I was working on some code today and encountered this scenario here where the performance behavior of data.table surprised me a little. Is this expected?</p>
<p id="bloop_customfont" style="font-family: Helvetica,Arial; font-size: 13px; margin: 0px; line-height: auto;"> </p>
<p id="bloop_customfont" style="font-family: Helvetica,Arial; font-size: 13px; margin: 0px; line-height: auto;"> </p>
<p id="bloop_customfont" style="margin: 0px;">> dt = data.table(a=rnorm(1000000))</p>
<p id="bloop_customfont" style="margin: 0px;"> </p>
<p id="bloop_customfont" style="margin: 0px;"> </p>
<p id="bloop_customfont" style="margin: 0px;">> system.time( for(i in 1:100000) j = dt[i, a] )</p>
<p id="bloop_customfont" style="margin: 0px;"> usuário sistema decorrido </p>
<p id="bloop_customfont" style="margin: 0px;"> 78.064 0.426 78.034 </p>
<p id="bloop_customfont" style="margin: 0px;"> </p>
<p id="bloop_customfont" style="margin: 0px;"> </p>
<p id="bloop_customfont" style="margin: 0px;">> system.time( for(i in 1:100000) j = dt[i, "a", with=F] )</p>
<p id="bloop_customfont" style="margin: 0px;"> usuário sistema decorrido </p>
<p id="bloop_customfont" style="margin: 0px;"> 27.814 0.154 27.810</p>
<p id="bloop_customfont" style="margin: 0px;"> </p>
<p id="bloop_customfont" style="margin: 0px;">> system.time( for(i in 1:100000) j = dt[["a"]][i] )</p>
<p id="bloop_customfont" style="margin: 0px;"> usuário sistema decorrido </p>
<p id="bloop_customfont" style="margin: 0px;"> 1.227 0.006 1.225 </p>
<div>(sorry about the output in portuguese)</div>
<div>Not knowing anything about how data.table is implemented internally, I would have assumed the three syntaxes for accessing the data.table should have similar or at the most a small difference in performance.</div>
<div id="bloop_sign_1369761945473322752"><span style="font-family: helvetica,arial; font-size: 13px;"></span>
<div style="font-family: Helvetica; line-height: normal;">-- </div>
<div style="font-family: Helvetica; line-height: normal;"><span style="font-family: arial; font-size: small;">Alexandre Sieira</span><br style="font-family: arial; font-size: small;"><span style="font-family: arial; font-size: small;">CISA, CISSP, ISO 27001 Lead Auditor</span><br style="font-family: arial; font-size: small;"><br style="font-family: arial; font-size: small;"><span style="font-family: arial; font-size: small;">"The truth is rarely pure and never simple."</span><br style="font-family: arial; font-size: small;"><span style="font-family: arial; font-size: small;">Oscar Wilde, The Importance of Being Earnest, 1895, Act I</span></div>
</div>
</blockquote>
<p> </p>
<div> </div>
</div></span></blockquote></body></html>