<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Does this sentence from the warning
      help?<br>
      <br>
      " Also, in R<v3.1.0, list(DT1,DT2) copied the entire DT1 and
      DT2 (R's list() used to copy named objects); please upgrade to
      R>=v3.1.0 if that is biting. "<br>
      <br>
      Matthew<br>
      <br>
      On 20/09/13 19:01, Ricardo Saporta wrote:<br>
    </div>
    <blockquote
cite="mid:CAE7Aa4Q4jvZVmhAUJOLWsdTJ656EQSPM4rUW9QE-brsB46KBSQ@mail.gmail.com"
      type="cite">
      <div dir="ltr">One warning per DT in the list
        <div>  (I added the line breaks) 
          <div>-Rick</div>
          <div>=============================================</div>
          <div>
            <div>Warning messages:</div>
            <div><br>
            </div>
            <div>1: In `[.data.table`(DT, , `:=`(c("Col3", "Col4"),
              list(C3, C4))) :</div>
            <div><br>
            </div>
            <div>  Invalid .internal.selfref detected and fixed by
              taking a copy of the whole table so that := can add this
              new column by reference. At an earlier point, this
              data.table has been copied by R (or been created manually
              using structure() or similar). Avoid key<-, names<-
              and attr<- which in R currently (and oddly) may copy
              the whole data.table. Use set* syntax instead to avoid
              copying: ?set, ?setnames and ?setattr. Also, in
              R<v3.1.0, list(DT1,DT2) copied the entire DT1 and DT2
              (R's list() used to copy named objects); please upgrade to
              R>=v3.1.0 if that is biting. If this message doesn't
              help, please report to datatable-help so the root cause
              can be fixed.</div>
            <div><br>
            </div>
            <div>2: In `[.data.table`(DT, , `:=`(c("Col3", "Col4"),
              list(C3, C4))) :</div>
            <div><br>
            </div>
            <div>  Invalid .internal.selfref detected and fixed by
              taking a copy of the whole table so that := can add this
              new column by reference. At an earlier point, this
              data.table has been copied by R (or been created manually
              using structure() or similar). Avoid key<-, names<-
              and attr<- which in R currently (and oddly) may copy
              the whole data.table. Use set* syntax instead to avoid
              copying: ?set, ?setnames and ?setattr. Also, in
              R<v3.1.0, list(DT1,DT2) copied the entire DT1 and DT2
              (R's list() used to copy named objects); please upgrade to
              R>=v3.1.0 if that is biting. If this message doesn't
              help, please report to datatable-help so the root cause
              can be fixed.</div>
          </div>
          <div>=============================================<br>
          </div>
          <div><br>
          </div>
        </div>
        <div class="gmail_extra"><br>
          <br>
          <br>
          <div class="gmail_quote">On Fri, Sep 20, 2013 at 12:49 PM,
            Matthew Dowle <span dir="ltr"><<a moz-do-not-send="true"
                href="mailto:mdowle@mdowle.plus.com" target="_blank">mdowle@mdowle.plus.com</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <div><br>
                  Hi,<br>
                  <br>
                  What's the warning?<br>
                  <br>
                  Matthew
                  <div>
                    <div class="h5"><br>
                      <br>
                      <br>
                      On 20/09/13 14:48, Ricardo Saporta wrote:<br>
                    </div>
                  </div>
                </div>
                <blockquote type="cite">
                  <div>
                    <div class="h5">
                      <div dir="ltr">
                        <div>
                          <div>I've encountered the following issue
                            iterating over a list of data.tables. </div>
                          <div>The issue is only with mapply, not with
                            lapply .</div>
                          <div><br>
                          </div>
                          <div> </div>
                          <div>Given a list of data.table's, mapply'ing
                            over the list directly </div>
                          <div>cannot modify in place. </div>
                          <div><br>
                          </div>
                          <div>Also if attempting to add a new column,
                            we get an "Invalid .internal.selfref"
                            warning. </div>
                          <div>Modifying an existing column does not
                            issue a warning, but still fails to
                            modify-in-place</div>
                          <div><br>
                          </div>
                          <div>WORKAROUND: </div>
                          <div>----------</div>
                          <div>The workaround is to iterate over an
                            index to the list, then to </div>
                          <div>  modify each data.table via
                            list.of.DTs[[i]][ .. ]</div>
                          <div><br>
                          </div>
                          <div>**Interestingly, this issue occurs with
                            `mapply`, but not `lapply`.**</div>
                          <div><br>
                          </div>
                          <div> </div>
                          <div>EXAMPLE:</div>
                          <div>-------- </div>
                          <div>  # Given a list of DT's and two lists of
                            vectors, </div>
                          <div>  #   we want to add the corresponding
                            vectors as columns to the DT. </div>
                          <div><br>
                          </div>
                          <div>## ---------------- ##</div>
                          <div>##   SAMPLE DATA:   ##</div>
                          <div>## ---------------- ##</div>
                          <div>  # list of data.tables</div>
                          <div>  list.DT <- list(</div>
                          <div>    DT1=data.table(Col1=111:115,
                            Col2=121:125),</div>
                          <div>    DT2=data.table(Col1=211:215,
                            Col2=221:225)</div>
                          <div>    )</div>
                          <div><br>
                          </div>
                          <div>  # lists of columns to add</div>
                          <div>  list.Col3 <- list(131:135, 231:235)</div>
                          <div>  list.Col4 <- list(141:145, 241:245)</div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##   Iterating over the list elements  
                            ##</div>
                          <div>##     adding a new column            
                             ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##   Will issue warning and            
                            ##</div>
                          <div>##     will fail to modify in place    
                            ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>  mapply (</div>
                          <div>      function(DT, C3, C4)</div>
                          <div>          DT[, c("Col3", "Col4") :=
                            list(C3, C4)],</div>
                          <div>      </div>
                          <div>      list.DT,  # iterating over the list</div>
                          <div>      list.Col3, list.Col4,</div>
                          <div>      SIMPLIFY=FALSE</div>
                          <div>    ) </div>
                          <div><br>
                          </div>
                          <div>  ## Note the lack of change</div>
                          <div>  list.DT</div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##   Iterating over an index          
                             ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>  mapply (</div>
                          <div>      function(i, C3, C4)</div>
                          <div>         list.DT[[i]] [, c("Col3",
                            "Col4") := list(C3, C4)],</div>
                          <div>     </div>
                          <div>      seq(list.DT),   # iterating over an
                            index to the list</div>
                          <div>      list.Col3, list.Col4,</div>
                          <div>      SIMPLIFY=FALSE</div>
                          <div>    )</div>
                          <div><br>
                          </div>
                          <div>  ## Note each DT _has_ been modified</div>
                          <div>  list.DT</div>
                          <div><br>
                          </div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##   Iterating over the list elements  
                            ##</div>
                          <div>##     modifying existing column      
                             ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##   No warning issued, but            
                            ##</div>
                          <div>##     Will fail to modify in place    
                            ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>  mapply (</div>
                          <div>      function(DT, C3, C4)</div>
                          <div>         DT[, c("Col3", "Col4") :=
                            list(Col3*1e3, Col4*1e4)],</div>
                          <div><br>
                          </div>
                          <div>       list.DT,  # iterating over the
                            list</div>
                          <div>      list.Col3, list.Col4,</div>
                          <div>      SIMPLIFY=FALSE</div>
                          <div>    ) </div>
                          <div><br>
                          </div>
                          <div>  ## Note the lack of change (compare
                            with output from `mapply`)</div>
                          <div>  list.DT</div>
                          <div><br>
                          </div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>##                                    
                             ##</div>
                          <div>##   `lapply` works as expected.      
                             ##</div>
                          <div>##                                    
                             ##</div>
                          <div>## ------------------------------------
                            ##</div>
                          <div>  </div>
                          <div>  ## NOW WITH lapply</div>
                          <div>  lapply(list.DT, </div>
                          <div>    function(DT)</div>
                          <div>      DT[, newCol := LETTERS[1:5]]</div>
                          <div>  )</div>
                          <div><br>
                          </div>
                          <div>  ## Note the new column: </div>
                          <div>  list.DT</div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div># ========================== # </div>
                          <div><br>
                          </div>
                          <div>##   NON-WORKAROUNDS   ## </div>
                          <div>##</div>
                          <div>## I also tried all of the following
                            alternatives</div>
                          <div>##   in hopes of being able to iterate
                            over the list </div>
                          <div>##   directly, using `mapply`.  </div>
                          <div>## None of these worked. </div>
                          <div><br>
                          </div>
                          <div># (1) Creating the DTs First, then
                            creating the list from them</div>
                          <div>    DT1 <- data.table(Col1=111:115,
                            Col2=121:125)</div>
                          <div>    DT2 <- data.table(Col1=211:215,
                            Col2=221:225)</div>
                          <div><br>
                          </div>
                          <div>    list.DT <- list(DT1=DT1,DT2=DT2 )</div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div># (2) Same as 1, and using `copy()` in
                            the call to `list()`</div>
                          <div>    list.DT <- list(DT1=copy(DT1), </div>
                          <div>                    DT2=copy(DT2) )</div>
                          <div><br>
                          </div>
                          <div># (3) lapply'ing `copy` and then
                            iterating over that list</div>
                          <div>    list.DT <- lapply(list.DT, copy)</div>
                          <div><br>
                          </div>
                          <div># (4) Not naming the list elements</div>
                          <div>    list.DT <- list(DT1, DT2)</div>
                          <div>    # and tried</div>
                          <div>    list.DT <- list(copy(DT1),
                            copy(DT2))</div>
                          <div><br>
                          </div>
                          <div>## All of the above still failed to
                            modify in place</div>
                          <div>##   (and also issued the same warning if
                            trying to add a column)</div>
                          <div>##    when iterating using mapply</div>
                          <div><br>
                          </div>
                          <div>  mapply(function(DT, C3, C4)</div>
                          <div>    DT[, c("Col3", "Col4") := list(C3,
                            C4)],</div>
                          <div>    list.DT, list.Col3, list.Col4,</div>
                          <div>    SIMPLIFY=FALSE)</div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div># ========================== # </div>
                        </div>
                        <div><br>
                        </div>
                        <br clear="all">
                        <div>
                          <div
                            style="color:rgb(34,34,34);font-size:13px;font-family:arial,sans-serif">
                            <div style="font-size:13px">Ricardo Saporta</div>
                            <div style="font-size:13px"> Rutgers
                              University, New Jersey<br>
                            </div>
                            <div style="font-size:13px"><span
                                style="font-size:13px">e: </span><a
                                moz-do-not-send="true"
                                href="mailto:saporta@rutgers.edu"
                                style="color:rgb(17,85,204);font-size:13px"
                                target="_blank">saporta@rutgers.edu</a></div>
                            <div><br>
                            </div>
                          </div>
                        </div>
                      </div>
                      <br>
                      <fieldset></fieldset>
                      <br>
                    </div>
                  </div>
                  <pre>_______________________________________________
datatable-help mailing list
<a moz-do-not-send="true" href="mailto:datatable-help@lists.r-forge.r-project.org" target="_blank">datatable-help@lists.r-forge.r-project.org</a>
<a moz-do-not-send="true" href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a></pre>
                </blockquote>
                <br>
              </div>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>