[GenABEL-dev] bug_in_OmicABEL_reshuffle_fixed

[Содбо Шарапов]

Thank you, Lennart and Yurii=)

>I've got a similar feature request/bug report for ProbABEL, do you know
>what the effect of going from unsigned int to int64 will be on memory
>usage?

int64_t use 8 bytes instead of 4 bytes for int.
In case of "reshuffle", now there are only three int64_t variables. As you
can see, there is no problem with size of memory.
But, during "reshuffling" tile_coordinates counting many times (about one
time per 5-10 doubles from data).
So, now reshuffle's runtime for data [1080traits;122756SNP;5 columns] is
about 21 sec (this runtime is for --chi=25 opertaion).
Before correction, runtime was about 16 sec...faster than now.

PS: I found some another bugs in reshuffle(with --heritabilities) and
,also, ways to optimized work with big data. As soon as possible, I'll do
it.


2013/7/1 Yurii Aulchenko <yurii.aulchenko at gmail.com>

> Thanks, Sodbo - does pass my test now! :)
>
> This is actually very good - I was so depressed not seeing any
> association, then happy to discover a bug, and now even more happy to see
> quite a few significant hits!
>
> YA
>
> On Mon, Jul 1, 2013 at 9:51 AM, Содбо Шарапов <sharapovsodbo at gmail.com>wrote:
>
>> Dear all!
>> I fixed bug in OmicABEL_reshuffle.
>> This bug was only for big data. The reason is, that for big output data
>> value of tile_coordinate is higher, than max(int).
>> For example: for data with 1080 ids and 122756 SNPs
>> max(tile_coordinate)=1080(ids) * 122756(SNPs) * 8 (sizeof(double)) * 5
>> (columns:beta_1,se_1,beta_SNP,se_SNP, etc) =  5 303 059 200
>> max(int) = 2 147 483 647
>> max(unsigned int) = 4 294 967 295
>> This values is lower than max(tile_coordinate). That's why
>> tile_coordinates for a half of data were incorrect and senseless.
>> So, the solution of this problem is change type of variabels for
>> tile_coordinates: I select int64_t instead of int.
>> max (int64_t)= 9,223,372,036,854,775,808. I think this is enough!=)
>> Now, "reshuffle" works with big data correctly. Compilation for Linux and
>> Windows was succesful.
>> --
>> *_________________________________*
>> *
>> *With best regards
>>
>> Sodbo Zh. Sharapov
>> Phone:  +79831347688
>> Email:    sharapovsodbo at gmail.com
>>              sharapov at bionet.nsc.ru
>> Skype:   sharapovsodbo
>>
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>
>
>
>
> --
> -----------------------------------------------------
> Yurii S. Aulchenko
>
> [ LinkedIn <http://nl.linkedin.com/in/yuriiaulchenko> ] [ Twitter<http://twitter.com/YuriiAulchenko>] [
> Blog <http://yurii-aulchenko.blogspot.nl/> ]
>



-- 
*_________________________________*
*
*With best regards

Sodbo Zh. Sharapov
Phone:  +79831347688
Email:    sharapovsodbo at gmail.com
             sharapov at bionet.nsc.ru
Skype:   sharapovsodbo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20130701/f44aba55/attachment-0001.html>