[datatable-help] Random segfaults

[Matthew Dowle]

Great fingers and toes crossed. If you could unset alloccol option just to
be sure please, that would be great. You're our best hope of confirming
it's fixed since it was biting you several times an hour. If you use [<-
or $<- syntax then R will copy via *tmp* and at that point the *tmp*
data.table is similar to a data.table loaded from disk in that it isn't
over-allocated anymore, I realised. Also a copy() will lose
over-allocation until the next column addition.  That 'should' all be fine
now in both <=2.13.2 and >=2.14.0, although the bug was something simpler.

1.7.7 is on CRAN now and been built for windows so if CRAN check results
tick over from "ERROR" to "OK" later today (for both windows and mac
old-rel), and, you're ok too, then it's fixed.


> I've updated to the latest SVN version, and I'll be sure to let you
> know if it still crashes (however I do have the alloccol option set to
> 1000, so I shouldn't be bumping into reallocation very often). Thanks
> for finding the bug so fast!
>
> On 14 December 2011 19:56, Matthew Dowle <mdowle at mdowle.plus.com> wrote:
>>
>> Hm. Sounds like it could be a different problem then if it was in R
>> 2.14. There have been quite a few fixes since 1.7.4 so if you can
>> reproduce with 1.7.7 would be great.  Or, we've sometimes seen that just
>> after a package upgrade that a clean re-install can often fix things.
>> Perhaps if the .so was in use by another R process or a zombie, or
>> something. R seems to report data.table v1.7.4 (say) but it hasn't fully
>> installed it properly and is still (perhaps partially) at 1.7.3. So quit
>> all R (reboot to clear zombies too perhaps) and try reinstalling using R
>> CMD INSTALL. Next time it happens I mean. Can also run test.data.table()
>> to check the install.
>>
>> On Wed, 2011-12-14 at 17:40 +0000, Timothée Carayol wrote:
>>> Hi --
>>>
>>> I have been having many unreproducible bugs with R 2.14, data.table
>>> 1.7.4 and ubuntu 64 bits about 10 days ago. Data was getting
>>> corrupted, and then R crashed. I had to go back to data.frame for the
>>> bits of code affected. I was doing a lot of rather unsafe
>>> manipulations with row names, rbind and cbinds.
>>> I didn't file a report, nor signal it, as it was occurring seemingly
>>> at random, and I was doing operations which aren't really what
>>> data.table was made for (tons of little manipulations on small data);
>>> still I guess I should now signal that 2.14 didn't fix everything for
>>> me. I do not know whether bugs subsist on post-1.7.4 versions.
>>>
>>> t
>>>
>>> On Wed, Dec 14, 2011 at 5:31 PM, Matthew Dowle <mdowle at mdowle.plus.com>
>>> wrote:
>>> >
>>> > Maybe, worth a try. Are you loading any data.table objects from disk?
>>> >
>>> >> 64 bit 2.12.1 linux.
>>> >>
>>> >> Is there an option I can set in my session in order to work around
>>> the
>>> >> truelength issue? I don't care if I lose some of the over-allocation
>>> >> niceties if it stops things from crashing. Looking at the truelength
>>> >> help, would just doing:
>>> >>
>>> >> options(datatable.alloc=quote(1000))
>>> >>
>>> >> stop this? I never have more than about 50 columns at a time.
>>> >>
>>> >> On 14 December 2011 11:43, Matthew Dowle <mdowle at mdowle.plus.com>
>>> wrote:
>>> >>>
>>> >>> You're R < 2.14.0, right?  I'm really struggling in R < 2.14.0 to
>>> make
>>> >>> over-allocation work because R only started to initialize
>>> truelength to
>>> >>> 0
>>> >>> in R 2.14.0+. Before that it's unitialized (random). Trouble is my
>>> >>> attempts in R < 2.14.0 to work around that work fine for me in
>>> linux
>>> >>> 32bit
>>> >>> when I test in R 2.13.2, and I even test in 2.12.0 too. I test on
>>> 64bit
>>> >>> too but just 2.14.0.  CRAN is also showing errors on 2.13.2
>>> (old-rel)
>>> >>> for
>>> >>> both mac and windows.
>>> >>>
>>> >>> So, this is a pre-2.14.0 (only) problem that I'll continue to try
>>> and
>>> >>> fix.
>>> >>>
>>> >>> Are you 64bit pre-2.14.0? Which OS?  If you are 64bit linux then it
>>> adds
>>> >>> weight to me installing pre-2.14.0 on my 64bit instance in an
>>> effort to
>>> >>> reproduce.
>>> >>>
>>> >>>
>>> >>>> This will be a crappy help request because I can't seem to
>>> reproduce
>>> >>>> it, but the past few days I've been getting a lot of segfaults.
>>>  The
>>> >>>> only common thing between every crash is that it happens when I do
>>> >>>>
>>> >>>> DT[, z := x]
>>> >>>>
>>> >>>> where z was not a column that existed in DT before, and x is
>>> either an
>>> >>>> existing column of DT or a separate variable, doesn't matter.
>>>  Beyond
>>> >>>> that I can't reproduce a set of steps that gets R to crash.  This
>>> is
>>> >>>> with the latest SVN version.
>>> >>>>
>>> >>>> Is there more information I can provide to help track this down?
>>> >>>> _______________________________________________
>>> >>>> datatable-help mailing list
>>> >>>> datatable-help at lists.r-forge.r-project.org
>>> >>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>> >>>>
>>> >>>
>>> >>>
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > datatable-help mailing list
>>> > datatable-help at lists.r-forge.r-project.org
>>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>
>>
>> _______________________________________________
>> datatable-help mailing list
>> datatable-help at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>