<div dir="ltr">+1 to using ~ for the not-join/join on complement/complement then join. Having some logical-looking i's lead to subsetting and others to not-joins can (for me) lead to mistakes that I'm not likely to catch until much later, if at all.<div>
<br></div><div style>I'm not sure I follow Arun's second example. If the syntax is changed so that ~ works as ! does now, then presumably !x will be reverted to having only a logical interpretation -- coercing x to logical and taking the subset where x == 0 -- which is the behavior you want. So why is it a separate issue? The remaining difference from data.frames would be that DF[!x] would show NA rows, if any, while DT[!x] would not.</div>
<div><br><div>--Frank</div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jun 10, 2013 at 4:21 AM, Arunkumar Srinivasan <span dir="ltr"><<a href="mailto:aragorn168b@gmail.com" target="_blank">aragorn168b@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
Matthew,
</div><div><br></div>
<div><div class="im"><div><blockquote type="cite"><p>How about ~ instead of ! ? I ruled out - previously to leave + and - available for future use. NJ() may be possible too.</p></blockquote></div></div>
<div><p>Both "NJ()" and "~" are okay for me.</p><div class="im"><p></p><blockquote type="cite"><div>That result makes perfect sense to me. I don't think of !(x==.) being the same as x!=. ! is simply a prefix. It's all the rows that aren't returned if the ! prefix wasn't there.</div>
<blockquote type="cite"><div></div></blockquote></blockquote><p></p></div><p>I understand that `DT[!(x)]` does what `data.table` is designed to do currently. What I failed to mention was that if one were to consider implementing `!(x==.)` as the same as `x != .` then this behaviour has to be changed. Let's forget this point for a moment.</p>
<div class="im"><p></p><blockquote type="cite"><div>That needs to be fixed. But we're getting quite theoretical here and far away from common use cases. Why would we ever have row numbers of the table, as a column of the table itself and want to select the rows by number not mentioned in that column?</div>
</blockquote></div><div>Probably I did not choose a good example. Suppose that I've a data.table and I want to get all rows where "x == 0". Let's say:</div><div><br></div><div>set.seed(45)</div><div>DT <- data.table( x = sample(c(0,5,10,15), 10, replace=TRUE), y = sample(15)) </div>
<div><div><p></p><div>DF <- as.data.frame(DT)</div><p></p></div></div><div>To get all rows where x == 0, it could be done with DT[x == 0]. But it makes sense, at least in the context of data.frames, to do equivalently,</div>
<div><br></div><div>DF[!(DF$x), ] (or) DF[DF$x == 0, ]</div><div><br></div><div>All I want to say is, I expect `DT[!(x)]` should give the same result as `DT[x == 0]` (even though I fully understand it's not the intended behaviour of data.table), as it's more intuitive and less confusing. </div>
<div><br></div><div>So, changing `!` to `~` or `NJ` is one half of the issue for me. The other is to replace the actual function of `!` in all contexts. I hope I came across with what I wanted to say, better this time.</div>
<div><br></div><div>Best,</div><div><br></div><div>Arun</div><p></p></div><div><br></div></div><div class="HOEnZb"><div class="h5">
<p style="color:#a0a0a8">On Monday, June 10, 2013 at 10:52 AM, Matthew Dowle wrote:</p>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px">
<span><div><div>
<p> </p>
<p>Hi,</p>
<p>How about ~ instead of ! ? I ruled out - previously to leave + and - available for future use. NJ() may be possible too.</p>
<p>Matthew</p>
<p> </p>
<p>On 10.06.2013 09:35, Arunkumar Srinivasan wrote:</p><blockquote type="cite"><div>
<div>Hi Matthew,</div>
<div>My view (from the last reply) more or less reflects mnel's comments here: <a href="http://stackoverflow.com/questions/16239153/dtx-and-dtx-treat-na-in-x-inconsistently#comment23317096_16240143" target="_blank">http://stackoverflow.com/questions/16239153/dtx-and-dtx-treat-na-in-x-inconsistently#comment23317096_16240143</a> </div>
<div>
<div>Pasted here for convenience:</div>
<div><span><code>data.table</code> is mimicing <code>subset</code> in its handling of <code>NA</code> values in logical <code>i</code> arguments. -- the only issue is the <code>!</code> prefix signifying a not-join, not the way one might expect. Perhaps the not join prefix could have been <code>NJ</code> not <code>!</code> to avoid this confusion -- this might be another discussion to have on the mailing list -- (I think it is a discussion worth having)</span><span> </span></div>
<div><span><br></span></div>
<div>Arun</div>
</div>
<p style="color:#a0a0a8">On Monday, June 10, 2013 at 10:28 AM, Arunkumar Srinivasan wrote:</p><blockquote style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px">
<div>
<div><blockquote type="cite"><div>
<div>
<p>Hm, good point. Is data.table consistent with SQL already, for both == and !=, and so no change needed? </p>
</div>
</div></blockquote><div>Yes, I believe it's already consistent with SQL. However, the current interpretation of NA (documentation) being treated as FALSE is not needed / untrue, imho (Please see below).</div>
<div> </div><blockquote type="cite"><div>
<div>
<p>And it was correct for Frank to be mistaken. </p>
</div>
</div></blockquote><div>Yes, it seems like he was mistaken.</div><blockquote type="cite"><div>
<div>
<p>Maybe just some more documentation and examples needed then.</p>
</div>
</div></blockquote><div>It'd be much more appropriate if the documentation reflects the role of subsetting in data.table mimicking "subset" function (in order to be in line with SQL) by dropping NA evaluated logicals. From a couple of posts before, where I pasted the code where NAs are replaced to FALSE were not necessary as `irows <- which(i)` makes clear that `which` is being used to get indices and then subset, this fits perfectly well with the interpretation of NA in data.table. </div>
<blockquote type="cite"><div>
<div>
<p>Are you happy that DT[!(x==.)] and DT[x!=.] do treat NA inconsistently? :</p>
<p><a href="http://stackoverflow.com/questions/16239153/dtx-and-dtx-treat-na-in-x-inconsistently" target="_blank">http://stackoverflow.com/questions/16239153/dtx-and-dtx-treat-na-in-x-inconsistently</a></p>
</div>
</div></blockquote><div> Ha, I like the idea behind the use of () in evaluating expressions. It's another nice layer towards simplicity in data.table. But I still think there should not be an inconsistency in equivalent logical operations to provide different results. If !(x== .) and x != . are indeed different, then I'd suppose replacing `!` with a more appropriate name as it's much easier to get confused otherwise. </div>
<div>In essence, either !(x == .) must evaluate to (x != .) if the underlying meaning of these are the same, or the `!` in `!(x==.)` must be replaced to something that's more appropriate for what it's supposed to be. Personally, I prefer the former. It would greatly tighten the structure and consistency.</div>
<blockquote type="cite"><div>
<div>
<p>"na.rm = TRUE/FALSE" sounds good to me. I'd only considered nomatch before in the context of joins, not logical subsets.</p>
</div>
</div></blockquote><div>Yes, I find this option would give more control in evaluating expressions with ease in `i`, by providing both "subset" (default) and the typical data.frame subsetting (na.rm = FALSE).</div>
<div>Best regards,</div>
<div> </div>
<div>
<div>Arun</div>
</div>
</div>
</div>
</blockquote></div></blockquote><p> </p>
<div> </div>
</div></div></span>
</blockquote>
<div>
<br>
</div>
</div></div><br>_______________________________________________<br>
datatable-help mailing list<br>
<a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
<a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br></blockquote></div><br></div>