<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I’m running into this “truelength is greater than 1000 items over-allocated” warning/error as I use := to add columns to a data.frame, e.g.:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>tl (1346) is greater than 1000 items over-allocated (ncol = 308). If you didn't set the datatable.alloccol option very large, please report this to datatable-help including the result of sessionInfo().<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>The long preamble to this is a stackoverflow thread (<a href="http://stackoverflow.com/questions/10015544">http://stackoverflow.com/questions/10015544</a>) in which I needed to update the contents of one data.table with the contents of another. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>The solution required the columns of both data.tables to match, hence my pre-processing loop to add columns to each data.table to satisfy the identical(names(dt1),names(dt2)) criteria. I may have to re-architect this depending on what is going on with this allocation business.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>If, for example, dt1 has 200 columns, and dt2 has 2000, and together they have 2100 unique columns, I’m going to add 1900 columns to dt1. If I set alloc.col to 2100 before my column-adding loop, I’ll get slapped because 2100 is more than 1000 greater than the 200 columns present in dt1.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>So do I need to spoon-feed alloc.col? Every iteration through the loop set it to length(dt1)+1 before adding a column? That seems rather brutal. Alternatively checking for the delta between truelength and length, and how close that is to the magic 1000 number, and then only adjusting the setting seems fragile.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I did try to make sense of the help for alloc.col. Regarding the bit about “if two or more variables are bound to the same data.table”; the column addition is within a function, and only one variable references the data.table, at least in the scope of the function. The function calling that function has a variable for the data.table too, so I don’t know if that counts. Then there is mention of using copy (not sure how that helps, and BTW the hyperlink for copy goes to the page for setkey, which does mention copy, but suggests “See ?copy” which just conjures up the setkey page again), setting alloc.col, or changing datatable.alloccol (doesn’t seem to help).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>The warning asked for sessionInfo; FWIW, here it is:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>R version 2.15.0 (2012-03-30)<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>Platform: x86_64-unknown-linux-gnu (64-bit)<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>locale:<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'> [7] LC_PAPER=C LC_NAME=C<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'> [9] LC_ADDRESS=C LC_TELEPHONE=C<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>attached base packages:<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>[1] stats graphics grDevices utils datasets methods<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>[7] base<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>other attached packages:<o:p></o:p></span></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:10.0pt;font-family:"Courier New"'>[1] data.table_1.8.2<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Thanks<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>George<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p></div></body></html>