<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><p>Jeremiah,</p>
<p>Thanks. Just a few hours ago, I answered a similar question to a post from Ron (pasted below): </p>
<pre><code>`data.table` is designed for working with *really large* data sets in mind (> 100 or 200 GB in memory even). And therefore, as a design feature, it trades in "referential transparency" for manipulating data objects *as efficient as possible* in terms of both *speed* and *memory usage* (most of the times they go hand-in-hand).
This is perhaps the biggest design choice one needs to be aware of when working/choosing data.tables. It is possible to modify objects by reference using data.table - All the functions that begin with "set*" modify objects by reference. The only other non "set*" function is `:=` operator.
</code></pre>
<p>There’s a pending feature request on adding this point (on explicit copy) to the FAQs, which we’ve not gotten to, yet.</p>
<p>To our knowledge, people do overcome this difference quite quickly.</p>
<p>It’s not necessary to know about pointers to understand that the object gets modified in-place. I’m not a python user at all, but recently came to know that this is also a feature there: https://docs.python.org/2/library/copy.html</p>
<p>But point taken. That <em>explicit copy</em> will be required will be added to the FAQs.</p>
<p><style>body{font-family:Helvetica,Arial;font-size:13px}</style><style>body {
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
padding:1em;
margin:auto;
background:#fefefe;
}
h1, h2, h3, h4, h5, h6 {
font-weight: bold;
}
h1 {
color: #000000;
font-size: 28pt;
}
h2 {
border-bottom: 1px solid #CCCCCC;
color: #000000;
font-size: 24px;
}
h3 {
font-size: 18px;
}
h4 {
font-size: 16px;
}
h5 {
font-size: 14px;
}
h6 {
color: #777777;
background-color: inherit;
font-size: 14px;
}
hr {
height: 0.2em;
border: 0;
color: #CCCCCC;
background-color: #CCCCCC;
}
p, blockquote, ul, ol, dl, li, table, pre {
margin: 15px 0;
}
a, a:visited {
color: #4183C4;
background-color: inherit;
text-decoration: none;
}
#message {
border-radius: 6px;
border: 1px solid #ccc;
display:block;
width:100%;
height:60px;
margin:6px 0px;
}
button, #ws {
font-size: 12 pt;
padding: 4px 6px;
border-radius: 5px;
border: 1px solid #bbb;
background-color: #eee;
}
code, pre, #ws, #message {
font-family: Monaco;
font-size: 10pt;
border-radius: 3px;
background-color: #F8F8F8;
color: inherit;
}
code {
border: 1px solid #EAEAEA;
margin: 0 2px;
padding: 0 5px;
}
pre {
border: 1px solid #CCCCCC;
overflow: auto;
padding: 4px 8px;
}
pre > code {
border: 0;
margin: 0;
padding: 0;
}
#ws { background-color: #f8f8f8; }
table {
border-collapse: collapse;
font-family: Helvetica, arial, freesans, clean, sans-serif;
color: rgb(51, 51, 51);
font-size: 15px; line-height: 25px;
padding: 0; }
table tr {
border-top: 1px solid #cccccc;
background-color: white;
margin: 0;
padding: 0; }
table tr:nth-child(2n) {
background-color: #f8f8f8; }
table tr th {
font-weight: bold;
border: 1px solid #cccccc;
margin: 0;
padding: 6px 13px; }
table tr td {
border: 1px solid #cccccc;
margin: 0;
padding: 6px 13px; }
table tr th :first-child, table tr td :first-child {
margin-top: 0; }
table tr th :last-child, table tr td :last-child {
margin-bottom: 0; }
.send { color:#77bb77; }
.server { color:#7799bb; }
.error { color:#AA0000; }</style></p><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div> <div id="bloop_sign_1402723614314099968" class="bloop_sign"><div style="font-family:helvetica,arial;font-size:13px">Arun</div></div> <div style="color:black"><br>From: <span style="color:black">jeremiah rounds</span> <a href="mailto:roundsjeremiah@gmail.com">roundsjeremiah@gmail.com</a><br>Reply: <span style="color:black">jeremiah rounds</span> <a href="mailto:roundsjeremiah@gmail.com">roundsjeremiah@gmail.com</a><br>Date: <span style="color:black">June 14, 2014 at 7:23:22 AM</span><br>To: <span style="color:black">datatable-help@lists.r-forge.r-project.org</span> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>Subject: <span style="color:black"> [datatable-help] Are you aware of this? <br></span></div><br> <blockquote type="cite" class="clean_bq"><span><div><div></div><div>
<title></title>
<div dir="ltr">As a fan of your work I have always been curious if
you are aware of this? I find it causes new users to make
mistakes.
<div><br></div>
<div><br></div>
<div>
<div>> dt = list()</div>
<div>> dt$x = 1:10</div>
<div>> dt$y = letters[10:1]</div>
<div>> dt = as.data.table(as.data.frame(dt))</div>
<div>> dt</div>
<div> x y</div>
<div> 1: 1 j</div>
<div> 2: 2 i</div>
<div> 3: 3 h</div>
<div> 4: 4 g</div>
<div> 5: 5 f</div>
<div> 6: 6 e</div>
<div> 7: 7 d</div>
<div> 8: 8 c</div>
<div> 9: 9 b</div>
<div>10: 10 a</div>
<div>> x0 = dt$x</div>
<div>> x1 = dt$x</div>
<div>> x0[1] = 11</div>
<div>> setkeyv(dt,"y")</div>
<div>> x0</div>
<div> [1] 11 2 3 4 5 6 7
8 9 10</div>
<div>> x1</div>
<div> [1] 10 9 8 7 6 5 4
3 2 1</div>
<div>> x1 == x0</div>
<div> [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
FALSE FALSE</div>
</div>
<div><br></div>
<div><br></div>
<div>x0 and x1 have assignments at the same exact time, and since R
data.frame's will not do this, it lures people into thinking they
are then identical and distinct as they are with data.frame's.
My theory is they are not actually copied: they are promised.
When x0 has its index 1 changed it induces a copy distinct
from dt$x, but x1 has had no operation on it so it refers to dt$x
with its promise. Setting the key on dt reorders it and since x1
still hasn't been evaluated it now matches the order of dt.</div>
<div><br></div>
<div>I found new users getting unpredictable results because they
would try to use a data.table as a data.frame and induce this with
sorts. If you thought you copied something in a particular
order in dt by doing the assigning ahead of the setkeyv you make a
mistake. You don't really expect x1 assigned maybe a page of
code above to have its order changed by a setkeyv. You do if
you think about C pointers and references, but in R you really
don't think that way. Many R users don't even know what a
pointer is.</div>
<div><br></div>
<div><br></div>
<div>Thanks,</div>
<div>Jeremiah</div>
<div><br></div>
<div>
<div>> sessionInfo()</div>
<div>R version 3.0.1 (2013-05-16)</div>
<div>Platform: x86_64-unknown-linux-gnu (64-bit)</div>
<div><br></div>
<div>locale:</div>
<div> [1] LC_CTYPE=en_US.UTF-8
LC_NUMERIC=C </div>
<div> [3] LC_TIME=en_US.UTF-8
LC_COLLATE=en_US.UTF-8 </div>
<div> [5] LC_MONETARY=en_US.UTF-8
LC_MESSAGES=en_US.UTF-8 </div>
<div> [7] LC_PAPER=C
LC_NAME=C
</div>
<div> [9] LC_ADDRESS=C
LC_TELEPHONE=C
</div>
<div>[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
</div>
<div><br></div>
<div>attached base packages:</div>
<div>[1] splines parallel stats graphics
grDevices utils datasets </div>
<div>[8] methods base </div>
<div><br></div>
<div>other attached packages:</div>
<div>[1] locfit_1.5-9.1 edgeR_3.4.2
limma_3.18.13
</div>
<div>[4] data.table_1.9.2 GenomicRanges_1.14.4
XVector_0.2.0 </div>
<div>[7] IRanges_1.20.7 BiocGenerics_0.8.0
</div>
<div><br></div>
<div>loaded via a namespace (and not attached):</div>
<div>[1] grid_3.0.1 lattice_0.20-15 plyr_1.8.1
Rcpp_0.11.1 </div>
<div>[5] reshape2_1.4 stats4_3.0.1
stringr_0.6.2 tools_3.0.1 </div>
</div>
<div><br></div>
<div><br></div>
<div><br></div>
</div>
_______________________________________________
<br>datatable-help mailing list
<br>datatable-help@lists.r-forge.r-project.org
<br>https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</div></div></span></blockquote><p></p></body></html>