[datatable-help] Random segfaults

Thu Dec 15 15:43:26 CET 2011

Result of test.data.table(), sessionInfo() and confirm it's a clean
install after a reboot to make sure no old .so is still knocking around
somehow please. Definitely installed to the right library? If it's
crashing a lot then it should be reproducible?
Still waiting for CRAN check results for 1.7.7 in old-rel. If it's not
fixed there either that'll help to know....

> Latest SVN version, no alloccol set, still crashing a lot.  I don't
> use [<- or $<-, the only times I modify a data.table are with :=  or
> by doing DT=merge(DT,blah).
>
> Any more info I can provide?
>
> On 15 December 2011 08:32, Matthew Dowle <mdowle at mdowle.plus.com> wrote:
>> Great fingers and toes crossed. If you could unset alloccol option just
>> to
>> be sure please, that would be great. You're our best hope of confirming
>> it's fixed since it was biting you several times an hour. If you use [<-
>> or $<- syntax then R will copy via *tmp* and at that point the *tmp*
>> data.table is similar to a data.table loaded from disk in that it isn't
>> over-allocated anymore, I realised. Also a copy() will lose
>> over-allocation until the next column addition.  That 'should' all be
>> fine
>> now in both <=2.13.2 and >=2.14.0, although the bug was something
>> simpler.
>>
>> 1.7.7 is on CRAN now and been built for windows so if CRAN check results
>> tick over from "ERROR" to "OK" later today (for both windows and mac
>> old-rel), and, you're ok too, then it's fixed.
>>
>>
>>> I've updated to the latest SVN version, and I'll be sure to let you
>>> know if it still crashes (however I do have the alloccol option set to
>>> 1000, so I shouldn't be bumping into reallocation very often). Thanks
>>> for finding the bug so fast!
>>>
>>> On 14 December 2011 19:56, Matthew Dowle <mdowle at mdowle.plus.com>
>>> wrote:
>>>>
>>>> Hm. Sounds like it could be a different problem then if it was in R
>>>> 2.14. There have been quite a few fixes since 1.7.4 so if you can
>>>> reproduce with 1.7.7 would be great.  Or, we've sometimes seen that
>>>> just
>>>> after a package upgrade that a clean re-install can often fix things.
>>>> Perhaps if the .so was in use by another R process or a zombie, or
>>>> something. R seems to report data.table v1.7.4 (say) but it hasn't
>>>> fully
>>>> installed it properly and is still (perhaps partially) at 1.7.3. So
>>>> quit
>>>> all R (reboot to clear zombies too perhaps) and try reinstalling using
>>>> R
>>>> CMD INSTALL. Next time it happens I mean. Can also run
>>>> test.data.table()
>>>> to check the install.
>>>>
>>>> On Wed, 2011-12-14 at 17:40 +0000, Timothée Carayol wrote:
>>>>> Hi --
>>>>>
>>>>> I have been having many unreproducible bugs with R 2.14, data.table
>>>>> 1.7.4 and ubuntu 64 bits about 10 days ago. Data was getting
>>>>> corrupted, and then R crashed. I had to go back to data.frame for the
>>>>> bits of code affected. I was doing a lot of rather unsafe
>>>>> manipulations with row names, rbind and cbinds.
>>>>> I didn't file a report, nor signal it, as it was occurring seemingly
>>>>> at random, and I was doing operations which aren't really what
>>>>> data.table was made for (tons of little manipulations on small data);
>>>>> still I guess I should now signal that 2.14 didn't fix everything for
>>>>> me. I do not know whether bugs subsist on post-1.7.4 versions.
>>>>>
>>>>> t
>>>>>
>>>>> On Wed, Dec 14, 2011 at 5:31 PM, Matthew Dowle
>>>>> <mdowle at mdowle.plus.com>
>>>>> wrote:
>>>>> >
>>>>> > Maybe, worth a try. Are you loading any data.table objects from
>>>>> disk?
>>>>> >
>>>>> >> 64 bit 2.12.1 linux.
>>>>> >>
>>>>> >> Is there an option I can set in my session in order to work around
>>>>> the
>>>>> >> truelength issue? I don't care if I lose some of the
>>>>> over-allocation
>>>>> >> niceties if it stops things from crashing. Looking at the
>>>>> truelength
>>>>> >> help, would just doing:
>>>>> >>
>>>>> >> options(datatable.alloc=quote(1000))
>>>>> >>
>>>>> >> stop this? I never have more than about 50 columns at a time.
>>>>> >>
>>>>> >> On 14 December 2011 11:43, Matthew Dowle <mdowle at mdowle.plus.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> You're R < 2.14.0, right?  I'm really struggling in R < 2.14.0 to
>>>>> make
>>>>> >>> over-allocation work because R only started to initialize
>>>>> truelength to
>>>>> >>> 0
>>>>> >>> in R 2.14.0+. Before that it's unitialized (random). Trouble is
>>>>> my
>>>>> >>> attempts in R < 2.14.0 to work around that work fine for me in
>>>>> linux
>>>>> >>> 32bit
>>>>> >>> when I test in R 2.13.2, and I even test in 2.12.0 too. I test on
>>>>> 64bit
>>>>> >>> too but just 2.14.0.  CRAN is also showing errors on 2.13.2
>>>>> (old-rel)
>>>>> >>> for
>>>>> >>> both mac and windows.
>>>>> >>>
>>>>> >>> So, this is a pre-2.14.0 (only) problem that I'll continue to try
>>>>> and
>>>>> >>> fix.
>>>>> >>>
>>>>> >>> Are you 64bit pre-2.14.0? Which OS?  If you are 64bit linux then
>>>>> it
>>>>> adds
>>>>> >>> weight to me installing pre-2.14.0 on my 64bit instance in an
>>>>> effort to
>>>>> >>> reproduce.
>>>>> >>>
>>>>> >>>
>>>>> >>>> This will be a crappy help request because I can't seem to
>>>>> reproduce
>>>>> >>>> it, but the past few days I've been getting a lot of segfaults.
>>>>>  The
>>>>> >>>> only common thing between every crash is that it happens when I
>>>>> do
>>>>> >>>>
>>>>> >>>> DT[, z := x]
>>>>> >>>>
>>>>> >>>> where z was not a column that existed in DT before, and x is
>>>>> either an
>>>>> >>>> existing column of DT or a separate variable, doesn't matter.
>>>>>  Beyond
>>>>> >>>> that I can't reproduce a set of steps that gets R to crash.
>>>>>  This
>>>>> is
>>>>> >>>> with the latest SVN version.
>>>>> >>>>
>>>>> >>>> Is there more information I can provide to help track this down?
>>>>> >>>> _______________________________________________
>>>>> >>>> datatable-help mailing list
>>>>> >>>> datatable-help at lists.r-forge.r-project.org
>>>>> >>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > datatable-help mailing list
>>>>> > datatable-help at lists.r-forge.r-project.org
>>>>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>>>
>>>>
>>>> _______________________________________________
>>>> datatable-help mailing list
>>>> datatable-help at lists.r-forge.r-project.org
>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>>
>>
>>
>