From yurii.aulchenko at gmail.com  Tue Jul  1 18:41:19 2014
From: yurii.aulchenko at gmail.com (Yury Aulchenko)
Date: Tue, 1 Jul 2014 18:41:19 +0200
Subject: [GenABEL-dev] Missing file for unit test in GenABEL
In-Reply-To: <20140627123455.GC17339@an3as.eu>
References: <20140620084443.GA17492@an3as.eu>
 <20140626134702.GA19423@an3as.eu> <1145061290386285028@unknownmsgid>
 <20140627123455.GC17339@an3as.eu>
Message-ID: <CA5E38E3-B8CD-48DD-AFA5-F2C32D2D7A79@gmail.com>

Dear Andreas,

On Jun 27, 2014, at 14:34, Andreas Tille <andreas at an3as.eu> wrote:

> Hi Yurii,
> 
> On Fri, Jun 27, 2014 at 02:29:11PM +0200, Yurii Aulchenko wrote:
>> Dear Andreas,
>> 
>> Sorry it delay with answer.
> 
> No problem. :-)
> 
>> We specifically remove unit tests before submitting to cran. Our dev
>> repo does contain a version with unit tests, also tags for the
>> versions.
> 
> Do you have any reasons to remove the tests from the tarball?  I could
> imagine that larger chunks of data might be used but I assumed that for
> this very purpose GenABEL.data exists ... and so I did package it for
> Debian as well.

This was requested by CRAN maintenance team some time (a year?) ago. My understanding this first takes long time to run these tests and secondly RUnit tests are somehow not very stable. Apparently this is not only for GenABEL, but is a general policy (not 100% sure though)


> 
>> So one option would be to use our code repo instead of what is on
>> cran. What do you think?
> 
> While this would possible in prinziple I personally really prefer a
> downloadable source tarball including testing features if any possible.
> However, if you might have strong reasons I would find ways to drain
> the stuff needed from the repository.

Not sure what is the best option, but we have a few

1. use CRAN release, no unit tests

2. use SVN tag for latest release - easiest for us (but I understand you'd prefer tarball)

3. we provide the tarball for latest release, including RUnit part on the code on our web-site http://www.genabel.org/

what do you think? I am somewhat afraid that it may be difficult to implement/keep coherent option 3. - to be fair, with me being overstretched, we struggle a bit with keeping sustainable maintenance of the GenABEL-package. 

Any other options, anyone?

best wishes,
Yurii

> 
> Kind regards
> 
>      Andreas.
> 
>> ----------------------
>> Yurii Aulchenko
>> (sent from mobile device)
>> 
>>> On Jun 26, 2014, at 15:47, Andreas Tille <andreas at an3as.eu> wrote:
>>> 
>>> Hi Yurii,
>>> 
>>>> On Fri, Jun 20, 2014 at 10:44:43AM +0200, Andreas Tille wrote:
>>>> Hi Yurii,
>>>> 
>>>> I'm trying to update the Debian package of GenABEL.  Since some time
>>>> there is an effort to automatically run unit tests of software if
>>>> available.  Since I noticed that GenABEL comes with unit tests I
>>>> tried
>>>> 
>>>> $ make test
>>>> export RCMDCHECK=FALSE;\
>>>>       cd ../../tests;\
>>>>       R --vanilla --slave < doRUnit.R
>>>> /bin/sh: 2: cd: can't cd to ../../tests
>>>> /bin/sh: 3: cannot open doRUnit.R: No such file
>>>> make: *** [test] Error 2
>>>> 
>>>> 
>>>> As you can see the file doRUnit.R is missing.  It would be great if you
>>>> could include this file into the source diustribution to make sure we
>>>> can reproduce your exact test procedure in the Debian package.
>>> 
>>> I cloned https://github.com/cran/GenABEL.git and had a look into the
>>> code.  I realised that the last tag containing the file doRUnit.R was
>>> 1.6-7.  I wonder how you are doing unit testing in the current version.
>>> 
>>> Thanks for any enlightenment
>>> 
>>>     Andreas.
>>> 
>>> --
>>> http://fam-tille.de
>> 
> 
> -- 
> http://fam-tille.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140701/de7b9659/attachment.html>

From andreas at an3as.eu  Tue Jul  1 21:33:54 2014
From: andreas at an3as.eu (Andreas Tille)
Date: Tue, 1 Jul 2014 21:33:54 +0200
Subject: [GenABEL-dev] Missing file for unit test in GenABEL
In-Reply-To: <CA5E38E3-B8CD-48DD-AFA5-F2C32D2D7A79@gmail.com>
References: <20140620084443.GA17492@an3as.eu> <20140626134702.GA19423@an3as.eu>
 <1145061290386285028@unknownmsgid>
 <20140627123455.GC17339@an3as.eu>
 <CA5E38E3-B8CD-48DD-AFA5-F2C32D2D7A79@gmail.com>
Message-ID: <20140701193354.GB21165@an3as.eu>

Hi Yury,

On Tue, Jul 01, 2014 at 06:41:19PM +0200, Yury Aulchenko wrote:
> > Do you have any reasons to remove the tests from the tarball?  I could
> > imagine that larger chunks of data might be used but I assumed that for
> > this very purpose GenABEL.data exists ... and so I did package it for
> > Debian as well.
> 
> This was requested by CRAN maintenance team some time (a year?) ago. My understanding this first takes long time to run these tests and secondly RUnit tests are somehow not very stable. Apparently this is not only for GenABEL, but is a general policy (not 100% sure though)

??? I admit it sounds pretty strange to me that there should be a policy
to avoid testing software.  At least in Debian we are doing the contrary
and try to add tests whereever possible that can run at package build
time and in periodical tests of the full archive.

> > While this would possible in prinziple I personally really prefer a
> > downloadable source tarball including testing features if any possible.
> > However, if you might have strong reasons I would find ways to drain
> > the stuff needed from the repository.
> 
> Not sure what is the best option, but we have a few
> 
> 1. use CRAN release, no unit tests
> 
> 2. use SVN tag for latest release - easiest for us (but I understand you'd prefer tarball)
> 
> 3. we provide the tarball for latest release, including RUnit part on the code on our web-site http://www.genabel.org/
> 
> what do you think? I am somewhat afraid that it may be difficult to implement/keep coherent option 3. - to be fair, with me being overstretched, we struggle a bit with keeping sustainable maintenance of the GenABEL-package. 
> 
> Any other options, anyone?

I admit that having a canonical way to obtain a tarball from CRAN is
quite attractive.  Is there any chance to somehow "hide" the tests -
perhaps by renaming the directory the test files are located in - before
uploading the tarball to CRAN so the test will not run there but we can
"uncover" the files again and run the tests.  For me this sounds like a
good compromise to fullfill all needs without changing the workflow
drastically.

If you might document this in some README file also other potential
distributors and users could profit from this. 

> best wishes,

Same to you and thanks for your quick response

    Andreas, keeps on shaking head why CRAN refuses testing ...

> Yurii
> 
> >>> On Jun 26, 2014, at 15:47, Andreas Tille <andreas at an3as.eu> wrote:
> >>> 
> >>> Hi Yurii,
> >>> 
> >>>> On Fri, Jun 20, 2014 at 10:44:43AM +0200, Andreas Tille wrote:
> >>>> Hi Yurii,
> >>>> 
> >>>> I'm trying to update the Debian package of GenABEL.  Since some time
> >>>> there is an effort to automatically run unit tests of software if
> >>>> available.  Since I noticed that GenABEL comes with unit tests I
> >>>> tried
> >>>> 
> >>>> $ make test
> >>>> export RCMDCHECK=FALSE;\
> >>>>       cd ../../tests;\
> >>>>       R --vanilla --slave < doRUnit.R
> >>>> /bin/sh: 2: cd: can't cd to ../../tests
> >>>> /bin/sh: 3: cannot open doRUnit.R: No such file
> >>>> make: *** [test] Error 2
> >>>> 
> >>>> 
> >>>> As you can see the file doRUnit.R is missing.  It would be great if you
> >>>> could include this file into the source diustribution to make sure we
> >>>> can reproduce your exact test procedure in the Debian package.
> >>> 
> >>> I cloned https://github.com/cran/GenABEL.git and had a look into the
> >>> code.  I realised that the last tag containing the file doRUnit.R was
> >>> 1.6-7.  I wonder how you are doing unit testing in the current version.
> >>> 
> >>> Thanks for any enlightenment
> >>> 
> >>>     Andreas.
> >>> 
> >>> --
> >>> http://fam-tille.de
> >> 
> > 
> > -- 
> > http://fam-tille.de
> 

-- 
http://fam-tille.de

From lennart at karssen.org  Wed Jul  2 08:30:09 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Wed, 02 Jul 2014 08:30:09 +0200
Subject: [GenABEL-dev] Missing file for unit test in GenABEL
In-Reply-To: <20140701193354.GB21165@an3as.eu>
References: <20140620084443.GA17492@an3as.eu>
 <20140626134702.GA19423@an3as.eu> <1145061290386285028@unknownmsgid>
 <20140627123455.GC17339@an3as.eu>
 <CA5E38E3-B8CD-48DD-AFA5-F2C32D2D7A79@gmail.com>
 <20140701193354.GB21165@an3as.eu>
Message-ID: <53B3A6F1.8040607@karssen.org>

Hi Andreas, Yurii,

On 01-07-14 21:33, Andreas Tille wrote:
> Hi Yury,
> 
> On Tue, Jul 01, 2014 at 06:41:19PM +0200, Yury Aulchenko wrote:
>>> Do you have any reasons to remove the tests from the tarball?  I could
>>> imagine that larger chunks of data might be used but I assumed that for
>>> this very purpose GenABEL.data exists ... and so I did package it for
>>> Debian as well.
>>
>> This was requested by CRAN maintenance team some time (a year?) ago. My understanding this first takes long time to run these tests and secondly RUnit tests are somehow not very stable. Apparently this is not only for GenABEL, but is a general policy (not 100% sure though)
> 
> ??? I admit it sounds pretty strange to me that there should be a policy
> to avoid testing software.  At least in Debian we are doing the contrary
> and try to add tests whereever possible that can run at package build
> time and in periodical tests of the full archive.

Maybe we should try to enable the tests again and see what the CRAN
people say?


> 
>>> While this would possible in prinziple I personally really prefer a
>>> downloadable source tarball including testing features if any possible.
>>> However, if you might have strong reasons I would find ways to drain
>>> the stuff needed from the repository.
>>
>> Not sure what is the best option, but we have a few
>>
>> 1. use CRAN release, no unit tests
>>
>> 2. use SVN tag for latest release - easiest for us (but I understand you'd prefer tarball)
>>
>> 3. we provide the tarball for latest release, including RUnit part on the code on our web-site http://www.genabel.org/
>>
>> what do you think? I am somewhat afraid that it may be difficult to implement/keep coherent option 3. - to be fair, with me being overstretched, we struggle a bit with keeping sustainable maintenance of the GenABEL-package. 
>>
>> Any other options, anyone?


Using a simple SVN checkout is not going to work. Currently we use a
shell script (also in SVN,
pkg/GenABEL-general/distrib_scripts/makedistrib_GenABEL.sh) to prepare
the package before releasing to CRAN. This script copies some stuff
around (e.g. some .cpp/.h files that are shared with DatABEL, but also
files that are in subdirectories in GenABEL/src/ in SVN (e.g. GAlib/,
ITERlib/), but get copied to GenABEL/src/. I guess that a proper
Makevars file would obsolete the need for this file juggling. It's been
on my list for quite some time (also for other packages), but as Yurii
mentioned, we're a bit short on man power.

> 
> I admit that having a canonical way to obtain a tarball from CRAN is
> quite attractive.  Is there any chance to somehow "hide" the tests -
> perhaps by renaming the directory the test files are located in - before
> uploading the tarball to CRAN so the test will not run there but we can
> "uncover" the files again and run the tests.  For me this sounds like a
> good compromise to fullfill all needs without changing the workflow
> drastically.

That should definitely be possible. I'll have a look.

> 
> If you might document this in some README file also other potential
> distributors and users could profit from this. 

Thanks for the suggestion. We should definitely do that.

> 
>> best wishes,
> 
> Same to you and thanks for your quick response
> 
>     Andreas, keeps on shaking head why CRAN refuses testing ...


Thanks for thinking along!

Lennart.

> 
>> Yurii
>>
>>>>> On Jun 26, 2014, at 15:47, Andreas Tille <andreas at an3as.eu> wrote:
>>>>>
>>>>> Hi Yurii,
>>>>>
>>>>>> On Fri, Jun 20, 2014 at 10:44:43AM +0200, Andreas Tille wrote:
>>>>>> Hi Yurii,
>>>>>>
>>>>>> I'm trying to update the Debian package of GenABEL.  Since some time
>>>>>> there is an effort to automatically run unit tests of software if
>>>>>> available.  Since I noticed that GenABEL comes with unit tests I
>>>>>> tried
>>>>>>
>>>>>> $ make test
>>>>>> export RCMDCHECK=FALSE;\
>>>>>>       cd ../../tests;\
>>>>>>       R --vanilla --slave < doRUnit.R
>>>>>> /bin/sh: 2: cd: can't cd to ../../tests
>>>>>> /bin/sh: 3: cannot open doRUnit.R: No such file
>>>>>> make: *** [test] Error 2
>>>>>>
>>>>>>
>>>>>> As you can see the file doRUnit.R is missing.  It would be great if you
>>>>>> could include this file into the source diustribution to make sure we
>>>>>> can reproduce your exact test procedure in the Debian package.
>>>>>
>>>>> I cloned https://github.com/cran/GenABEL.git and had a look into the
>>>>> code.  I realised that the last tag containing the file doRUnit.R was
>>>>> 1.6-7.  I wonder how you are doing unit testing in the current version.
>>>>>
>>>>> Thanks for any enlightenment
>>>>>
>>>>>     Andreas.
>>>>>
>>>>> --
>>>>> http://fam-tille.de
>>>>
>>>
>>> -- 
>>> http://fam-tille.de
>>
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140702/6c642873/attachment.sig>

From lennart at karssen.org  Wed Jul  2 09:27:15 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Wed, 02 Jul 2014 09:27:15 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1760 - pkg/filevector/fvlib
In-Reply-To: <20140702053119.918F5187617@r-forge.r-project.org>
References: <20140702053119.918F5187617@r-forge.r-project.org>
Message-ID: <53B3B453.909@karssen.org>

Hi Maksim,


On 02-07-14 07:31, noreply at r-forge.r-project.org wrote:
> Author: maksim
> Date: 2014-07-02 07:31:19 +0200 (Wed, 02 Jul 2014)
> New Revision: 1760
> 
> Modified:
>    pkg/filevector/fvlib/CastUtils.h
> Log:
> Constants was not declared here

Strange that we didn't catch that before. I hope this fixes the bug
(reported on the forum at
http://forum.genabel.org/viewtopic.php?f=6&t=884&p=1877#p1877 )

I created a bug report (#5973) so we can keep track of things:
https://r-forge.r-project.org/tracker/?func=detail&atid=2058&aid=5793&group_id=505


Best,

Lennart.

> 
> Modified: pkg/filevector/fvlib/CastUtils.h
> ===================================================================
> --- pkg/filevector/fvlib/CastUtils.h	2014-07-01 04:47:21 UTC (rev 1759)
> +++ pkg/filevector/fvlib/CastUtils.h	2014-07-02 05:31:19 UTC (rev 1760)
> @@ -1,6 +1,9 @@
>  #ifndef __CastUtils__
>  #define __CastUtils__
>  
> +#include "const.h"
> +
> +
>  // these copy-pastes are for template support
>  void setNan(unsigned short int &i);
>  void setNan(short int &i);
> 
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140702/7dad4c42/attachment-0001.sig>

From andreas at an3as.eu  Wed Jul  2 20:11:20 2014
From: andreas at an3as.eu (Andreas Tille)
Date: Wed, 2 Jul 2014 20:11:20 +0200
Subject: [GenABEL-dev] Missing file for unit test in GenABEL
In-Reply-To: <53B3A6F1.8040607@karssen.org>
References: <20140620084443.GA17492@an3as.eu> <20140626134702.GA19423@an3as.eu>
 <1145061290386285028@unknownmsgid>
 <20140627123455.GC17339@an3as.eu>
 <CA5E38E3-B8CD-48DD-AFA5-F2C32D2D7A79@gmail.com>
 <20140701193354.GB21165@an3as.eu> <53B3A6F1.8040607@karssen.org>
Message-ID: <20140702181120.GA19239@an3as.eu>

Hi Lennart,

On Wed, Jul 02, 2014 at 08:30:09AM +0200, L.C. Karssen wrote:
> > ??? I admit it sounds pretty strange to me that there should be a policy
> > to avoid testing software.  At least in Debian we are doing the contrary
> > and try to add tests whereever possible that can run at package build
> > time and in periodical tests of the full archive.
> 
> Maybe we should try to enable the tests again and see what the CRAN
> people say?

+1
 
> >> 1. use CRAN release, no unit tests
> >>
> >> 2. use SVN tag for latest release - easiest for us (but I understand you'd prefer tarball)
> >>
> >> 3. we provide the tarball for latest release, including RUnit part on the code on our web-site http://www.genabel.org/
> >>
> >> what do you think? I am somewhat afraid that it may be difficult to implement/keep coherent option 3. - to be fair, with me being overstretched, we struggle a bit with keeping sustainable maintenance of the GenABEL-package. 
> >>
> >> Any other options, anyone?
> 
> 
> Using a simple SVN checkout is not going to work. Currently we use a
> shell script (also in SVN,
> pkg/GenABEL-general/distrib_scripts/makedistrib_GenABEL.sh) to prepare
> the package before releasing to CRAN. This script copies some stuff
> around (e.g. some .cpp/.h files that are shared with DatABEL, but also
> files that are in subdirectories in GenABEL/src/ in SVN (e.g. GAlib/,
> ITERlib/), but get copied to GenABEL/src/. I guess that a proper
> Makevars file would obsolete the need for this file juggling. It's been
> on my list for quite some time (also for other packages), but as Yurii
> mentioned, we're a bit short on man power.

... as always.

> > I admit that having a canonical way to obtain a tarball from CRAN is
> > quite attractive.  Is there any chance to somehow "hide" the tests -
> > perhaps by renaming the directory the test files are located in - before
> > uploading the tarball to CRAN so the test will not run there but we can
> > "uncover" the files again and run the tests.  For me this sounds like a
> > good compromise to fullfill all needs without changing the workflow
> > drastically.
> 
> That should definitely be possible. I'll have a look.

So my prefered solution would be in this sequence

  1. Follow your suggestion to re-enable the tests on CRAN
  2. Follow my suggestion to provide "hidden" tests
  3. anything else

> > If you might document this in some README file also other potential
> > distributors and users could profit from this. 
> 
> Thanks for the suggestion. We should definitely do that.
> 
> Thanks for thinking along!

Thanks also to you for the constructive cooperation

      Andreas.

-- 
http://fam-tille.de

From yurii.aulchenko at gmail.com  Sat Jul  5 15:20:37 2014
From: yurii.aulchenko at gmail.com (Yury Aulchenko)
Date: Sat, 5 Jul 2014 15:20:37 +0200
Subject: [GenABEL-dev] Doi for tutorial?
Message-ID: <8B813E39-FDB1-4545-B1C5-2FC40FEE886B@gmail.com>

Dear all,

Just thought that much effort went into the GenABEL tutorial, but it does not count as a publication or anything. Also, not quite 'citable'. 

What do you think about idea of getting doi for GenABEL tutorial? Anybody has experience with this? We can probably try to work this through f1000research. 

Other - related - idea would be to try to publish the tutorial as a regular paper book (of course we will need a agreement with publisher that there is free online version). 

Yurii

----------------
Sent from mobile device, please excuse possible typos

From lennart at karssen.org  Mon Jul  7 10:58:01 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Mon, 07 Jul 2014 10:58:01 +0200
Subject: [GenABEL-dev] Doi for tutorial?
In-Reply-To: <8B813E39-FDB1-4545-B1C5-2FC40FEE886B@gmail.com>
References: <8B813E39-FDB1-4545-B1C5-2FC40FEE886B@gmail.com>
Message-ID: <53BA6119.7050302@karssen.org>

Hi all,

On 05-07-14 15:20, Yury Aulchenko wrote:
> Dear all,
> 
> Just thought that much effort went into the GenABEL tutorial, but it
> does not count as a publication or anything. Also, not quite
> 'citable'.
> 
> What do you think about idea of getting doi for GenABEL tutorial?
> Anybody has experience with this? 

I think this is a wonderful idea! As you pointed out, this would make
the tutorial much more easily citable and therefore visible.

I've never done this before. I think it is a good idea. However, I'm not
sure if it works for material (like the tutorial) that is 'fluid' in
nature (from the doi.org FAQ):
  7. If I have assigned a DOI name and I make a change to my material,
  should I assign a new DOI?
  The IDF does not have any rules on this. Individual RAs adopt
  appropriate rules for their community and application. As a general
  rule, if the change is substantial and/or it is necessary to identify
  both the original and the changed material, assign a new DOI name.

So maybe we should make releases/editions of the tutorial and ask for a
DIO for each major release.

> We can probably try to work this through f1000research.

From the doi.org FAQ:

  1. How do I get a DOI for my material?
  You must use a service offered by a DOI Registration Agency (RA).

A list of RAs can be found at [1]. The cost for registering apparently
differ per RA. I haven't yet checked what the amount would (roughly) be.

> 
> Other - related - idea would be to try to publish the tutorial as a
> regular paper book (of course we will need a agreement with publisher
> that there is free online version).

Or we could publish it ourselves through one of the various self-publish
services. Although I must admit I don't have any experience with that.


Best,

Lennart.

> 
> Yurii
> 
> ---------------- Sent from mobile device, please excuse possible
> typos _______________________________________________ genabel-devel
> mailing list genabel-devel at lists.r-forge.r-project.org 
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>
> 


Best,

Lennart.

[1] http://www.doi.org/registration_agencies.html

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140707/706ea8e9/attachment.sig>

From lennart at karssen.org  Tue Jul 22 18:42:09 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Tue, 22 Jul 2014 18:42:09 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1763 - in pkg/OmicABELnoMM: .
	src tests
In-Reply-To: <20140715081730.1C04B1874A0@r-forge.r-project.org>
References: <20140715081730.1C04B1874A0@r-forge.r-project.org>
Message-ID: <53CE9461.8000702@karssen.org>

Thanks Alvaro!

I ran only a quick check, and I noticed that after an 'autoreconf -i'
./configure, make and make check all work. Great!
However, running make distclean doesn't work. I'll commit a fix for that
later.

Now that the Makefile.am files are working fine, I'll delete the
tests/Makefile as well.


Thanks again,

Lennart.

On 15-07-14 10:17, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-15 10:17:29 +0200 (Tue, 15 Jul 2014)
> New Revision: 1763
> 
> Added:
>    pkg/OmicABELnoMM/tests/Makefile.am
> Modified:
>    pkg/OmicABELnoMM/Makefile.am
>    pkg/OmicABELnoMM/configure.ac
>    pkg/OmicABELnoMM/src/AIOwrapper.cpp
>    pkg/OmicABELnoMM/src/AIOwrapper.h
>    pkg/OmicABELnoMM/src/Algorithm.cpp
>    pkg/OmicABELnoMM/src/Algorithm.h
>    pkg/OmicABELnoMM/src/Definitions.h
>    pkg/OmicABELnoMM/src/Utility.cpp
>    pkg/OmicABELnoMM/src/Utility.h
>    pkg/OmicABELnoMM/src/main.cpp
>    pkg/OmicABELnoMM/tests/Makefile
>    pkg/OmicABELnoMM/tests/test.cpp
> Log:
> Added R2, T-stat, Pval functions, but inactive atm. Added exclusion list support to remove individuals. Fixed propagation of missing values. Performance Improvements.
> 
> Modified: pkg/OmicABELnoMM/Makefile.am
> ===================================================================
> --- pkg/OmicABELnoMM/Makefile.am	2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/Makefile.am	2014-07-15 08:17:29 UTC (rev 1763)
> @@ -2,7 +2,7 @@
>  
>  oanomm_headers = src/Definitions.h src/AIOwrapper.h src/Algorithm.h	\
>   src/Utility.h
> -oanomm_cpp = src/AIOwrapper.cpp src/Algorithm.cpp src/Utility.cpp	\
> +oanomm_cpp = src/AIOwrapper.cpp src/Algorithm.cpp src/Utility.cpp \
>   src/main.cpp
>  
>  omicabelnomm_SOURCES = $(oanomm_headers) $(oanomm_cpp)
> 
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac	2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/configure.ac	2014-07-15 08:17:29 UTC (rev 1763)
> @@ -17,11 +17,12 @@
>  # Set some default compile flags
>  if test -z "$CXXFLAGS"; then
>     # User did not set CXXFLAGS, so we can put in our own defaults
> -    CXXFLAGS="-g -O2"
> +    CXXFLAGS="-O3"
>  fi
>  if test -z "$CPPFLAGS"; then
>     # User did not set CPPFLAGS, so we can put in our own defaults
>      CPPFLAGS="-Wall -g -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> +    #CPPFLAGS="-Wall"
>  fi
>  # If CXXFLAGS/CPPFLAGS are already set AC_PROG_CXX will not overwrite them
>  # with its own defaults
> @@ -45,13 +46,18 @@
>     AC_MSG_ERROR([Unable to find a Lapack library])
>  ])
>  
> +#Boost
>  
> +
> +
>  # Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
>  AC_OPENMP
>  AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>  
>  
>  
> +
> +
>  # Checks for header files.
>  AC_CHECK_HEADERS([limits.h stdlib.h string.h sys/time.h unistd.h])
>  
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-15 08:17:29 UTC (rev 1763)
> @@ -28,16 +28,21 @@
>      pthread_barrier_init(&(FHandler.finalize_barrier),NULL,2);
>  
>  
> -    Fhandler->fakefiles = params.use_fake_files;
> +    Fhandler->fakefiles = params.use_fake_files;
> +
> +
> +
>  
>  
> +
>      if(!Fhandler->fakefiles)
>      {
>          Fhandler->fnameAL = params.fnameAL;
>          Fhandler->fnameAR = params.fnameAR;
>          Fhandler->fnameY = params.fnameY;
> -        Fhandler->fnameOutB = params.fnameOutB;
> +        Fhandler->fnameOutB = params.fnameOutB;
>  
> +
>          Yfvi  = load_databel_fvi( (Fhandler->fnameY+".fvi").c_str() );
>          ALfvi = load_databel_fvi( (Fhandler->fnameAL+".fvi").c_str() );
>          ARfvi = load_databel_fvi( (Fhandler->fnameAR+".fvi").c_str() );
> @@ -46,7 +51,7 @@
>          params.t = Yfvi->fvi_header.numVariables;
>          params.l = ALfvi->fvi_header.numVariables;
>  
> -	int opt_tb = 1000;
> +        int opt_tb = 1000;
>          int opt_mb = 1000;
>  
>          params.mb = min(params.m, opt_tb);
> @@ -56,7 +61,22 @@
>      else
>      {
>  
> -    }
> +    }
> +
> +    //params.fname_excludelist = "exclfile.txt";
> +    int excl_count = 0;
> +    Fhandler->excl_List = new list< pair<int,int> >();
> +
> +    if(params.fname_excludelist.size()==0)
> +    {
> +        (Fhandler->excl_List)->push_back( make_pair(0,params.n) );
> +    }
> +    else
> +    {
> +        read_excludeList( Fhandler->excl_List,excl_count,params.n,params.fname_excludelist);
> +    }
> +
> +    params.n -= excl_count;
>  
>      params.p = params.l + params.r;
>  
> @@ -106,7 +126,9 @@
>      pthread_cond_destroy(&(Fhandler->condition_more));
>  
>      pthread_mutex_destroy(&(Fhandler->m_read));
> -    pthread_cond_destroy(&(Fhandler->condition_read));
> +    pthread_cond_destroy(&(Fhandler->condition_read));
> +
> +    delete Fhandler->excl_List;
>  
>  
>  }
> @@ -186,12 +208,15 @@
>          }
>          //cout << "\nEnd preping files\n" << flush;
>  
> -    }
> +    }
> +
> +    //pthread_barrier_wait(&(Fhandler->finalize_barrier));//for testing only
>  
>  
>      Fhandler->not_done = true;
> -    Fhandler->reset_wait = false;
> +    Fhandler->reset_wait = false;
>  
> +
>      while(Fhandler->not_done)
>      {
>  
> @@ -226,9 +251,30 @@
>                  Fhandler->seed += 75;
>              }
>              else
> -            {
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> -                result++;
> +            {
> +                list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +
> +                int chunk_size_buff;
> +                int buff_pos=0;
> +                int file_pos;
> +
> +                for(int i = 0; i < tmp_y_blockSize; i++)
> +                {
> +                    for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +                    {
> +                        file_pos = Fhandler->n*i + it->first;
> +                        fseek ( fp_Y , file_pos*sizeof(type_precision) , SEEK_SET );
> +                        chunk_size_buff = it->second;
> +
> +                        size_t result = fread (&tobeFilled->buff[buff_pos],sizeof(type_precision),chunk_size_buff,fp_Y); result++;
> +                        buff_pos += chunk_size_buff;
> +
> +
> +                    }
> +                }
> +//                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> +//                result++;
>                  if(Fhandler->y_to_readSize <= 0)
>                  {
>                      fseek ( fp_Y , 0 , SEEK_SET );
> @@ -267,17 +313,38 @@
>              if(Fhandler->fakefiles)
>              {
>                  fseek ( fp_Ar , 0 , SEEK_SET );
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -                result++;
> +                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar); result++;
>  
>                  re_random_vec(tobeFilled->buff , Fhandler->n * tmp_ar_blockSize*Fhandler->r );
>                  re_random_vec_nan(tobeFilled->buff , Fhandler->n * tmp_ar_blockSize*Fhandler->r );
>  
>              }
>              else
> -            {
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -                result++;
> +            {
> +
> +                list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +                int chunk_size_buff;
> +                int buff_pos=0;
> +                int file_pos;
> +
> +                for(int i = 0; i < tmp_ar_blockSize*Fhandler->r; i++)
> +                {
> +                    for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +                    {
> +                        file_pos = Fhandler->n*i + it->first;
> +                        fseek ( fp_Ar , file_pos*sizeof(type_precision) , SEEK_SET );
> +
> +                        chunk_size_buff = it->second;
> +                        size_t result = fread (&tobeFilled->buff[buff_pos],sizeof(type_precision),chunk_size_buff,fp_Ar); result++;
> +                        buff_pos += chunk_size_buff;
> +
> +
> +                    }
> +                }
> +
> +//                size_t result = fread(tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> +//                result++;
>                  if (Fhandler->Ar_to_readSize <= 0)
>                  {
>                      fseek ( fp_Ar , 0 , SEEK_SET );
> @@ -307,8 +374,8 @@
>              if(Fhandler->fakefiles)
>              {
>                  fseek ( fp_B , 0 , SEEK_SET );
> -	  }
> -		fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
> +            }
> +            fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
>  
>  
>              Fhandler->b_empty_buffers.push(tobeWritten);
> @@ -356,7 +423,8 @@
>      //cout << "k" << flush;
>      //barrier
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -
> +
> +    {
>      type_buffElement* tmp;
>      while(!Fhandler->full_buffers.empty())
>      {
> @@ -404,6 +472,7 @@
>         Fhandler->b_empty_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
> +    }
>      }
>  
>  
> @@ -544,13 +613,9 @@
>  
>  void AIOwrapper::write_B(type_precision* B, int p, int blockSize)
>  {
> -     //int status;
> -    //int createstatus = 0;
> -   // cout << " b: full" << Fhandler->b_full_buffers.size() << "; epty" << Fhandler->b_empty_buffers.size() << endl;
>  
>      while(Fhandler->b_empty_buffers.empty())
>      {
> -
>          pthread_mutex_lock(&(Fhandler->m_more));
>          pthread_cond_signal( &(Fhandler->condition_more ));
>          pthread_mutex_unlock(&(Fhandler->m_more));
> @@ -568,7 +633,7 @@
>  
>  
>  
> -
> +        //cout << Fhandler->b_empty_buffers.size() << flush;
>          Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
>          Fhandler->b_empty_buffers.pop();
>  
> @@ -582,12 +647,9 @@
>  
>  
>  
> -    // cout << " b: full" << Fhandler->b_full_buffers.size() << "; epty" << Fhandler->b_empty_buffers.size() << endl;
> -
>      pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
>  
> -
>      pthread_mutex_lock(&(Fhandler->m_more));
>      pthread_cond_signal( &(Fhandler->condition_more ));
>      pthread_mutex_unlock(&(Fhandler->m_more));
> @@ -668,8 +730,12 @@
>  //        {
>  //            (tmp->buff)[i] = 0;
>  //        }
> -        Fhandler->b_empty_buffers.push(tmp);
> +        Fhandler->b_empty_buffers.push(tmp);
> +
> +//        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> +//        Fhandler->b_empty_buffers.pop();
>  
> +
>      }
>  
>  }
> @@ -819,17 +885,38 @@
>          (*AL) = Fhandler->AL;
>      }
>      else
> -    {
> +    {
>          FILE *fp;
>          fp = fopen((Fhandler->fnameAL+".fvd").c_str(), "rb");
>          if(fp == 0)
>          {
>              cout << "Error Reading File " << Fhandler->fnameAL << endl;
>              exit(1);
> -        }
> +        }
> +
> +        list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +        int chunk_size_buff;
> +        int buff_pos=0;
> +        int file_pos;
> +
> +        for (int i=0; i < Fhandler->l; i++)
> +        {
> +            for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +            {
> +                file_pos = i*Fhandler->n+ it->first;
> +                fseek ( fp , file_pos*sizeof(type_precision) , SEEK_SET );
> +                chunk_size_buff = it->second;
> +
> +                size_t result = fread (&(Fhandler->AL[buff_pos]),sizeof(type_precision),chunk_size_buff,fp); result++;
> +                buff_pos += chunk_size_buff;
> +            }
> +        }
>  
> -        size_t result = fread (Fhandler->AL,sizeof(type_precision),Fhandler->l*Fhandler->n,fp);
> -        result++;
> +
> +//        size_t result = fread (Fhandler->AL,sizeof(type_precision),Fhandler->l*Fhandler->n,fp);
> +//
> +//        result++;
>          fclose(fp);
>      }
>  
> @@ -862,6 +949,111 @@
>  {
>      delete []Fhandler->AL;
>  }
> +
> +
> +void AIOwrapper::read_excludeList(list< pair<int,int> >* excl, int &excl_count, int max_excl, string fname_excludeList)
> +{
> +
> +    ifstream fp_exL(fname_excludeList.c_str());
> +    if(fp_exL == 0)
> +    {
> +        cout << "Error reading exclude list file."<< endl;
> +        exit(1);
> +    }
> +
> +
> +    string line;
> +    excl_count = 0;
> +    bool early_EOF;
> +    int first,second, prev_second;
> +
> +     cout << "Excluding Ids: \n";
> +
> +    std::getline(fp_exL, line);
> +    std::istringstream iss(line);
> +    iss >> first;
> +    early_EOF = iss.eof();
> +    iss >> prev_second;
> +
> +    if(prev_second < first || early_EOF)
> +            prev_second = first;
> +
> +    second = prev_second;
> +
> +
> +    if(first > max_excl)
> +    {
> +        excl->push_back( make_pair(0,max_excl) );
> +        cout << "\nNothing to Exclude!\n";
> +    }
> +    else
> +    {
> +        if(second > max_excl)
> +        {
> +                excl_count += max_excl-first+1;
> +                excl->push_back( make_pair(0,first-1) );
> +                cout << first << "-" << second << ", ";
> +        }
> +        else
> +        {
> +
> +            cout << first << "-" << second << ", ";
> +            excl->push_back( make_pair(0,first - 1) );
> +            excl_count += second-(first-1);
> +
> +            while (std::getline(fp_exL, line) && second < max_excl )
> +            {
> +                std::istringstream iss(line);
> +
> +                iss >> first;
> +                early_EOF = iss.eof();
> +                iss >> second;
> +                if(prev_second >= first)
> +                {
> +                    cout << "\nPlease give an ordered Exlusion List!\n";
> +                    cout << "?? " << prev_second << "\n" << first << " ??"<< endl;
> +                    exit( 1 );
> +                }
> +
> +
> +                if(second < first || early_EOF )
> +                    second = first;
> +
> +
> +                if(second > max_excl)
> +                    excl_count += max_excl-first+1;
> +                else if(first < max_excl)
> +                    excl_count += second-(first-1);
> +
> +
> +
> +                cout << first << "-" << second << ", ";
> +
> +                excl->push_back( make_pair(prev_second,first - prev_second - 1) );
> +
> +
> +                prev_second = second;
> +
> +
> +            }
> +        }
> +    }
> +
> +
> +
> +
> +    if(excl_count >= max_excl)
> +    {
> +		cout << "\nExclusion List excluded all data!\n";
> +		cout << "Total Ids: " << max_excl << "\nExcluded Ids: " << excl_count << endl;
> +		exit( 1 );
> +	}
> +	 cout << "Excluded: "  << excl_count << " Using: " << max_excl-excl_count<< endl;
> +
> +
> +
> +}
> +
>  
>  void AIOwrapper::free_databel_fvi( struct databel_fvi **fvi )
>  {
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-15 08:17:29 UTC (rev 1763)
> @@ -2,7 +2,9 @@
>  #define AIOWRAPPER_H
>  
>  #include "Definitions.h"
> -#include "Utility.h"
> +#include "Utility.h"
> +#include <fstream>
> +#include <sstream>
>  
>  typedef struct BufferElement type_buffElement;
>  
> @@ -20,9 +22,15 @@
>  {
>      string fnameAL;
>      string fnameAR;
> -    string fnameY;
> -    string fnameOutB;
> +    string fnameY;
> +
>  
> +    string fnameOutB;
> +
> +
> +    list< pair<int,int> >* excl_List;
> +
> +
>      bool doublefileType;
>      bool fakefiles;
>  
> @@ -138,7 +146,9 @@
>  
>      protected:
>  
> -    private:
> +    private:
> +
> +        void read_excludeList(list< pair<int,int> >* excl, int &excl_count, int max_excl, string fname_excludeList);
>  
>  
>          void prepare_AR( int desired_blockSize, int n, int totalR, int columnsR);
> @@ -163,7 +173,7 @@
>  
>          void * fgls_malloc_impl( const char* file, long line, size_t size );
>  
> -        type_fileh FHandler;
> +            public: type_fileh FHandler;
>          type_fileh* Fhandler;
>  
>          FILE* fp_Ar;
> 
> Modified: pkg/OmicABELnoMM/src/Algorithm.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-15 08:17:29 UTC (rev 1763)
> @@ -1,5 +1,7 @@
> -#include "Algorithm.h"
> +#include "Algorithm.h"
> +
>  
> +
>  Algorithm::Algorithm()
>  {
>      // ctor
> @@ -51,41 +53,39 @@
>  }
>  
>  
> -void Algorithm::prepare_Bfinal(type_precision* bfinal, type_precision* top,
> -                               type_precision* bot, int dim1_b,
> -                               int dim2_b, int dim1_b_bot)
> +void Algorithm::prepare_Bfinal(type_precision* bfinal, type_precision* bsource, int a_amount, int y_amount, int p)
>  {
> -    // memcpy are faster version of the fors
> -    int i, k, w, top_idx, bot_idx;
> -    int size;
> -    top_idx = 0;
> -    bot_idx = 0;
> -    for (k = 0; k < dim2_b; k++)
> -    {
> -        size = k * dim1_b + (dim1_b - dim1_b_bot) - (k * dim1_b);
> -        memcpy( (type_precision*)&bfinal[k * dim1_b],
> -                (type_precision*)&top[top_idx],
> -                size * sizeof(type_precision) );
> -//        for (i = k*dim1_b; i < k*dim1_b+(dim1_b-dim1_b_bot); i++)
> -//        {
> -//            bfinal[i] = top[top_idx];
> -//            top_idx++;
> -//        }
> -        top_idx += size;
> -        i = k * dim1_b + size;
> -        w = i;
> -
> -        size = w + dim1_b_bot - w;
> -        memcpy( (type_precision*)&bfinal[w],
> -                (type_precision*)&bot[bot_idx],
> -                size * sizeof(type_precision) );
> -//        for (i = w; i < w+dim1_b_bot; i++)
> -//        {
> -//            bfinal[i] = bot[bot_idx];
> -//            bot_idx++;
> -//        }
> -        bot_idx += size;
> -    }
> +//    // memcpy are faster version of the fors
> +//    int i, k, w, top_idx, bot_idx;
> +//    int size;
> +//    top_idx = 0;
> +//    bot_idx = 0;
> +//    for (k = 0; k < dim2_b; k++)
> +//    {
> +//        size = k * dim1_b + (dim1_b - dim1_b_bot) - (k * dim1_b);
> +//        memcpy( (type_precision*)&bfinal[k * dim1_b],
> +//                (type_precision*)&top[top_idx],
> +//                size * sizeof(type_precision) );
> +////        for (i = k*dim1_b; i < k*dim1_b+(dim1_b-dim1_b_bot); i++)
> +////        {
> +////            bfinal[i] = top[top_idx];
> +////            top_idx++;
> +////        }
> +//        top_idx += size;
> +//        i = k * dim1_b + size;
> +//        w = i;
> +//
> +//        size = w + dim1_b_bot - w;
> +//        memcpy( (type_precision*)&bfinal[w],
> +//                (type_precision*)&bot[bot_idx],
> +//                size * sizeof(type_precision) );
> +////        for (i = w; i < w+dim1_b_bot; i++)
> +////        {
> +////            bfinal[i] = bot[bot_idx];
> +////            bot_idx++;
> +////        }
> +//        bot_idx += size;
> +//    }
>  }
>  
>  
> @@ -249,12 +249,13 @@
>  
>      lapack_int info = LAPACKE_sgels(STORAGE_TYPE, 'N', rowsA, colsA, rhs, A,
>                                      rowsA, ynew, rowsA);
> -    assert(info == 0, "Error Check");
> +    myassert(info == 0, "Error Check");
>  
>  
>      int index = 0;
>      int index_new = 0;
> -    for (i = 0; i < rhs; i++)
> +    for (i = 0; i < rhs; i++)
> +
>      {
>           copy_vec(&ynew[index], &new_sol[index_new], colsA);
>           index += rowsA;
> @@ -305,13 +306,14 @@
>  
>      srand(time(NULL));
>  
> -    blas_set_num_threads(max_threads);
> +    blas_set_num_threads(max_threads);
> +    omp_set_num_threads(max_threads);
>  
>  
>      //type_precision *Ytemp;
>      lapack_int info, n, lda, l, r, p;
>  
> -    cputime_type start_tick, start_tick2, end_tick;
> +    cputime_type start_tick, start_tick2, start_tick3, end_tick;
>  
>      AIOfile.initialize(params);
>  
> @@ -333,54 +335,70 @@
>  
>      for (int j = 0; j < y_iters && !params.ForceCheck; j++)
>      {
> -        if (y_iters >= 40 && (j%(y_iters/40)) == 0)
> +        if (!params.ForceCheck &&  y_iters >= 10 && (j%(y_iters/10)) == 0 )
>          {
>              cout << "*" << flush;
>          }
>  
>          for (int i = 0; i < a_iters; i++)
>          {
> -            for (int jj = 0; jj < y_block_size; jj++)
> -            {
> -                if (y_iters < 40 &&
> -                    (y_block_size < 3 || (jj%(y_block_size/3)) == 0)
> -                    )
> +
> +                if ( !params.ForceCheck &&  y_iters < 10 &&
> +                (  (a_iters >= 10 && (i%(a_iters/(10/y_iters))) == 0) || (a_iters < (10/y_iters)) ))
>                  {
>                      cout << "*" << flush;
>                  }
> -            }
> +
>          }
>      }
>  
>      if(!params.ForceCheck)
>          cout << endl;
>  
> -
> +    //add memalign
>  
>      //type_precision Stl[l*l];
>      //type_precision Str[l*r*a_block_size];
> -    type_precision* Stl = new type_precision[l*l];
> -    type_precision* Str = new type_precision[l*r*a_block_size];
> +    type_precision* Stl = new type_precision[l*l*1];
> +    type_precision* Str = new type_precision[l*r*a_block_size*1];
>  
>      type_precision* Sbr = new type_precision[r *  r * a_block_size];
> -    type_precision* Ay = new type_precision[p * a_block_size];
> +    type_precision* Ay = new type_precision[p * a_block_size];
> +    //type_precision* B_resorted = new type_precision[p * a_block_size*y_block_size];
>  
> -    type_precision* S = new type_precision[p * p];
> +    type_precision* S = new type_precision[p * p];
>  
> +
>      type_precision* Ay_top = new type_precision[l * y_amount];
>      type_precision* Ay_bot = new type_precision[y_block_size * a_block_size * r];
>  
> -    list<long int>* y_nan_idxs = new list<long int>[y_block_size];
> +    type_precision* y_residual = new type_precision[n * y_block_size ];
> +    type_precision* y_res_norms = new type_precision[a_block_size];
> +
> +    list<long int>* al_nan_idxs = new list<long int>[1];
> +    list<long int>* y_nan_idxs = new list<long int>[y_block_size];
> +    list<long int>* ar_nan_idxs = new list<long int>[a_block_size];
>  
> +
> +    type_precision* A = new type_precision[n * p * 1];
>      type_precision* AR = new type_precision[n * r * a_block_size * 1];
> -    type_precision* AL = new type_precision[n * l * 1];
> +//  type_precision* AL = new type_precision[n * l * 1];
> +    type_precision* AL = A;
> +
> +    type_precision* B = Ay;
> +
>      type_precision* backupAR;  // = new type_precision[n*r*a_block_size];
>      type_precision* backupAL;  // = new type_precision[n*l];
>  
>  
>      AIOfile.load_AL(&backupAL);
> -    //int total_al_nans = replace_nans(0, backupAL, n, l);
> -    replace_nans(0, backupAL, n, l);
> +
> +    //pthread_barrier_wait(&(AIOfile.Fhandler->finalize_barrier));
> +
> +    replace_nans(al_nan_idxs,1, backupAL, n, l);
> +    al_nan_idxs->push_back(1);
> +
> +    //LAPACKE_dgesdd()
>  
>      copy_vec(backupAL, AL, n*l);
>  
> @@ -394,176 +412,369 @@
>  
>      for (int j = 0; j < y_iters; j++)
>      {
> -        if (y_iters >= 40 && (j%(y_iters/40)) == 0 && !params.ForceCheck)
> +        if (!params.ForceCheck &&  y_iters >= 10 && (j%(y_iters/10)) == 0 && !params.ForceCheck)
>          {
>              cout << AIOfile.io_overhead << flush;
>              AIOfile.io_overhead = "*";
>          }
> +
> +        get_ticks(start_tick2);
>  
> -        AIOfile.load_Yblock(&Y, y_block_size);
> +        AIOfile.load_Yblock(&Y, y_block_size);
> +
> +        get_ticks(end_tick);
> +        out.acc_loady += ticks2sec(end_tick,start_tick2);
>  
> +        get_ticks(start_tick2);
> +        replace_nans(&y_nan_idxs[0],y_block_size, Y, n,1);
> +        //out.acc_other += ticks2sec(end_tick,start_tick2);
>  
>  
> -        //int total_y_nans = replace_nans(&y_nan_idxs[0], Y, n, y_block_size);
> -        replace_nans(&y_nan_idxs[0], Y, n, y_block_size);
> +        get_ticks(start_tick2);
>  
> -
>          //! Ay_top = AL'*Y
>          cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
>                      l, y_block_size, n, 1.0, AL, n, Y, n, 0.0,
> -                    &Ay_top[j * l * y_block_size], l);
> +                    &Ay_top[j * l * y_block_size], l);
> +
> +        get_ticks(end_tick);
> +        out.acc_gemm += ticks2sec(end_tick,start_tick2);
>  
>  
>          for (int i = 0; i < a_iters; i++)
> -        {
> -            AIOfile.load_ARblock(&backupAR, a_block_size);
> -            //int total_ar_nans = replace_nans(0, backupAR, n, a_block_size * r);
> -            replace_nans(0, backupAR, n, a_block_size * r);
> -            copy_vec(backupAR, AR, n *  r * a_block_size);
> +        {
> +
> +            if (!params.ForceCheck && y_iters < 10 &&
> +                (  (a_iters >= 10 && (i%(a_iters/(10/y_iters))) == 0) || (a_iters < (10/y_iters)) ))
> +            {
> +                cout << "*" << flush;
> +            }
> +
> +            get_ticks(start_tick2);
>  
> -            get_ticks(start_tick2);
> +            AIOfile.load_ARblock(&backupAR, a_block_size);
> +
> +            get_ticks(end_tick);
> +            out.acc_loadxr += ticks2sec(end_tick,start_tick2);
> +
> +            get_ticks(start_tick2);
> +
> +            replace_nans(&ar_nan_idxs[0],a_block_size, backupAR, n, r);
> +            replace_with_zeros(al_nan_idxs, backupAR,  n, r, a_block_size);
>  
> +            copy_vec(backupAR, AR, n *  r * a_block_size);
> +
>              get_ticks(end_tick);
> -            out.acc_pre += ticks2sec(end_tick-start_tick2, cpu_freq);
> +            //out.acc_other += ticks2sec(end_tick,start_tick2);
>  
> -            get_ticks(start_tick2);
> +
> +
> +            get_ticks(start_tick2);
> +
>              //! Ay_bot = AR'*Y
>              cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
>                          r * a_block_size, y_block_size, n, 1.0, AR, n, Y, n,
>                          0.0, Ay_bot, r * a_block_size);
>  
>              get_ticks(end_tick);
> -            out.firstloop += ticks2sec(end_tick - start_tick2, cpu_freq);
> +            out.acc_gemm += ticks2sec(end_tick,start_tick2);
> +
> +            int aL_idx =  0*l * n;
> +            int aR_idx = 0* r * n * a_block_size;
>  
> -            //#pragma omp parallel default(shared)
> -            {
> -                for (int jj = 0; jj < y_block_size; jj++)
> -                {
> -                    int thread_id = 0 * omp_get_thread_num();
> -                    int aL_idx = thread_id * l * n;
> -                    int aR_idx = thread_id * r * n * a_block_size;
> +            get_ticks(start_tick3);
> +            for (int jj = 0; jj < y_block_size; jj++)
> +            {
> +
> +                //int thread_id = 0 * omp_get_thread_num();//so far singel thread version only
>  
> -                    if (y_iters < 40 &&
> -                                  (y_block_size < 3 ||
> -                                                  (jj%(y_block_size/3)) == 0) && !params.ForceCheck)
> -                    {
> -                        cout << AIOfile.io_overhead << flush;
> -                        AIOfile.io_overhead = "*";
> -                    }
> +
> +                get_ticks(start_tick2);
> +
> +                copy_vec(backupAL, &AL[aL_idx], n * l);//try to remove!
>  
> -                    copy_vec(backupAL, &AL[aL_idx], n * l);
> +                replace_with_zeros(&y_nan_idxs[jj], &AL[aL_idx], n, l, 1);
> +
> +
> +                get_ticks(end_tick);//2%
> +                out.acc_other += ticks2sec(end_tick,start_tick2);
> +
> +                 get_ticks(start_tick2);
>  
> -                    replace_with_zeros(&y_nan_idxs[jj], &AL[aL_idx], n, l, 1);
> -                    //! Generate Stl
> -                    cblas_ssyrk(CblasColMajor, CblasUpper, CblasTrans,
> -                                l, n, 1.0, &AL[aL_idx], lda, 0.0, Stl, l);
> +                //! Generate Stl
> +                cblas_ssyrk(CblasColMajor, CblasUpper, CblasTrans,
> +                            l, n, 1.0, &AL[aL_idx], lda, 0.0, Stl, l);
> +
> +                get_ticks(end_tick);
> +                out.acc_stl += ticks2sec(end_tick,start_tick2);
> +
> +
> +                get_ticks(start_tick2);
> +
> +                copy_vec(backupAR,&AR[aR_idx], n*r*a_block_size);//!10%//try to remove!
> +
> +
> +                replace_with_zeros(&y_nan_idxs[jj], backupAR,  n, r, a_block_size);
>  
> -                    copy_vec(backupAR,&AR[aR_idx], n*r*a_block_size);
> -                    replace_with_zeros(&y_nan_idxs[jj], &AR[aR_idx],
> -                                       n, r, a_block_size);
> +
> +
> +                get_ticks(end_tick);
> +                out.acc_other += ticks2sec(end_tick,start_tick2);
> +
> +                get_ticks(start_tick2);
>  
> -                    get_ticks(start_tick2);
> -                    //! Generate Str
> -                    cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> -                                l, r * a_block_size, n, 1.0, &AL[aL_idx],
> -                                n, &AR[aR_idx], n, 0.0, Str, l);
> -                    get_ticks(end_tick);
> -                    out.acc_RTL_QLY += ticks2sec(end_tick - start_tick2,
> -                                                 cpu_freq);
> +                //! Generate Str
> +                cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> +                            l, r * a_block_size, n, 1.0, &AL[aL_idx],
> +                            n, &AR[aR_idx], n, 0.0, Str, l);//!45
>  
> +                get_ticks(end_tick);
> +                out.acc_str += ticks2sec(end_tick,start_tick2);
> +
>  
> -                    //type_precision Sbr[r *  r * a_block_size];
> -                    //type_precision Ay[p * a_block_size];
>  
> +                //type_precision Sbr[r *  r * a_block_size*1];
> +                //type_precision Ay[p * a_block_size*1];
> +
> +                blas_set_num_threads(1);
> +                omp_set_num_threads(max_threads);
> +
> +                #pragma omp parallel default(shared)
> +                {
>  
> -                    //#pragma omp for nowait  schedule(dynamic)
> -                    for (int ii= 0; ii < a_block_size; ii++)
> -                    {
> -                        get_ticks(start_tick2);
> -                        //! Generate Sbr
> -                        cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> -                                    r, r, n, 1.0, &AR[aR_idx+ii*r*n], n,
> -                                    &AR[aR_idx + ii * r * n], n, 0.0,
> -                                    &Sbr[ii * r * r], r);
> -                        get_ticks(end_tick);
> -                        out.acc_gemm += ticks2sec(end_tick - start_tick2,
> -                                                  cpu_freq);
> +                #pragma omp for nowait
> +                for (int ii= 0; ii < a_block_size; ii++)
> +                {
> +                    //cout << omp_get_thread_num() << endl << flush;
>  
> -                        get_ticks(start_tick2);
>  
> -                        copy_vec(&Ay_top[j*l*y_block_size+jj*l], &Ay[ii*p], l);
> -                        copy_vec(&Ay_bot[ii*r + jj*r*a_block_size],
> -                                 &Ay[l+ii*p], r);
> +                    get_ticks(start_tick2);
>  
> +                    //! Generate Sbr
> +                    cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> +                                r, r, n, 1.0, &AR[aR_idx+ii*r*n], n,
> +                                &AR[aR_idx + ii * r * n], n, 0.0,
> +                                &Sbr[ii * r * r], r);
> +
>  
> -                        //type_precision* B = Ay;
> -                        //type_precision S[p * p];
> +                    get_ticks(end_tick);
> +                    out.acc_sbr += ticks2sec(end_tick,start_tick2 );
> +
> +                    get_ticks(start_tick2);
>  
> +                    copy_vec(&Ay_top[j*l*y_block_size+jj*l], &Ay[ii*p], l);
> +                    copy_vec(&Ay_bot[ii*r + jj*r*a_block_size],
> +                             &Ay[l+ii*p], r);
> +
> +
> +                    type_precision S2[p * p];
>  
> -                        //! Rebuild S
> -                        build_S(S, Stl, &Str[ii*r*l], &Sbr[ii*r*r], l, r);
> -                        // matlab_print_matrix("S", p, p, S);
> +                    //! Rebuild S
> +                    build_S(S2, Stl, &Str[ii*r*l], &Sbr[ii*r*r], l, r);
> +                    // matlab_print_matrix("S", p, p, S);
>  
> -                        get_ticks(end_tick);
> -                        out.acc_b += ticks2sec(end_tick - start_tick2,
> -                                               cpu_freq);
> +
> +                    get_ticks(end_tick);//5%
> +                    out.acc_other += ticks2sec(end_tick,start_tick2);
>  
> -                        get_ticks(start_tick2);
> +                    get_ticks(start_tick2);
>  
> -                        //! b = S\Ay
> -                        info = LAPACKE_sposv(STORAGE_TYPE, 'U', p, 1, S, p,
> -                                             &Ay[ii*p], p);
>  
> -                        get_ticks(end_tick);
> -                        out.acc_loadxr += ticks2sec(end_tick - start_tick2,
> -                                                    cpu_freq);
> -                        assert(info == 0, "POSV");
> +                    //! b = S\Ay
> +                    info = LAPACKE_sposv(STORAGE_TYPE, 'U', p, 1, S2, p,
> +                                         &Ay[ii*p], p);
> +
> +                    myassert(info == 0, "S\\Ay");
> +
>  
> -                        if (params.ForceCheck)
> +                    get_ticks(end_tick);
> +                    out.acc_solve += ticks2sec(end_tick,start_tick2 );
> +
> +
> +                    if (params.ForceCheck)
> +                    {
> +                        #pragma omp critical
>                          {
> -                            #pragma omp critical
> -                            {
> -                                check_result(AL, &AR[aR_idx+ii*r*n], n, p,
> -                                             1, r, &Y[jj*n], &Ay[ii*p]);
> -                            }
> +                            check_result(AL, &AR[aR_idx+ii*r*n], n, p,
> +                                         1, r, &Y[jj*n], &Ay[ii*p]);
>                          }
>                      }
> +                }
> +                }
> +
> +                /************statistics**************/
> +//                type_precision* T;
> +//                type_precision* R2;
> +//                type_precision* P;
> +
> +                //hpc_statistics(n,A,a_block_size,Y,y_block_size,B,p,T,R2,P);
> +
> +                /**************************/
> +
> +                blas_set_num_threads(max_threads);
> +
> +                get_ticks(start_tick2);
>  
> -                    AIOfile.write_B(Ay, p, a_block_size);
> -                }
> -            }
> +                AIOfile.write_B(B, p, a_block_size);
> +
> +                get_ticks(end_tick);
> +                out.acc_storeb += ticks2sec(end_tick,start_tick2);
> +
> +
> +
> +
> +
> +            }
> +
> +            get_ticks(end_tick);
> +            out.acc_real_innerloops += ticks2sec(end_tick ,start_tick3);
> +
> +
>          }
>           AIOfile.reset_AR();
>      }
>  
> -    get_ticks(end_tick);
> -    out.duration = ticks2sec(end_tick - start_tick, cpu_freq);
> -    // out.gflops = a_amount/1000.0*(n*l*r+n*r*r+y_amount*(n*r+p*p*p)))/1000.0/1000.0;
> -    out.gflops = y_amount * (gemm_flops(l, n, 1, 0) +
> -                             a_amount * (gemm_flops(r, n, 1, 0) +
> -                                         gemm_flops(l, n, l, 0) +
> -                                         gemm_flops(l, n, r, 0) +
> -                                         gemm_flops(r, n, r, 0) +
> +    get_ticks(end_tick);
> +
> +    out.duration = ticks2sec(end_tick ,start_tick);
> +
> +
> +    out.gflops = y_iters * (gemm_flops(l, n, params.tb*1, 0) +
> +                             a_iters * ( gemm_flops(params.mb*r, n, params.tb*1, 0) +
> +                             params.tb *(  gemm_flops(l, 1*n, l, 0) + gemm_flops(l, n, params.mb *r, 0) +
> +                                    params.mb * (
> +                                         gemm_flops(r, 1*n, r, 0) +
>                                           (p * p * p / 3.0) /
> -                                         1000.0/1000.0/1000.0));
> [TRUNCATED]
> 
> To get the complete diff run:
>     svnlook diff /svnroot/genabel -r 1763
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140722/34cc1d63/attachment-0001.sig>

From alvaro.frank at rwth-aachen.de  Tue Jul 22 19:06:59 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Tue, 22 Jul 2014 17:06:59 +0000
Subject: [GenABEL-dev] [Genabel-commits] r1763 - in pkg/OmicABELnoMM:
	.	src tests
In-Reply-To: <53CE9461.8000702@karssen.org>
References: <20140715081730.1C04B1874A0@r-forge.r-project.org>,
 <53CE9461.8000702@karssen.org>
Message-ID: <244CF001646FF74FB34F372310A332C501152D7C@MBX-S2.rwth-ad.de>

Hi Lennart,

The code had to be reworked greatly to achieve usable performance. This changes will come next. 
I also have to figure a way to make the code work with non standard BLAS, like AMD ACML, since, the code will be turtle slow on older systems with no AVX instructions, like Opterons 6100. MKL for Intel systems is the no problematic version, AMCL requires a different kind of linking. 

A few parameter handling is also new but not synced yet. I am trying to make sure the results are correct after I changed the algorithm.

-Alvaro Frank

________________________________________
From: genabel-devel-bounces at lists.r-forge.r-project.org [genabel-devel-bounces at lists.r-forge.r-project.org] on behalf of L.C. Karssen [lennart at karssen.org]
Sent: Tuesday, July 22, 2014 6:42 PM
To: genabel-devel at lists.r-forge.r-project.org
Subject: Re: [GenABEL-dev] [Genabel-commits] r1763 - in pkg/OmicABELnoMM: .     src tests

Thanks Alvaro!

I ran only a quick check, and I noticed that after an 'autoreconf -i'
./configure, make and make check all work. Great!
However, running make distclean doesn't work. I'll commit a fix for that
later.

Now that the Makefile.am files are working fine, I'll delete the
tests/Makefile as well.


Thanks again,

Lennart.

On 15-07-14 10:17, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-15 10:17:29 +0200 (Tue, 15 Jul 2014)
> New Revision: 1763
>
> Added:
>    pkg/OmicABELnoMM/tests/Makefile.am
> Modified:
>    pkg/OmicABELnoMM/Makefile.am
>    pkg/OmicABELnoMM/configure.ac
>    pkg/OmicABELnoMM/src/AIOwrapper.cpp
>    pkg/OmicABELnoMM/src/AIOwrapper.h
>    pkg/OmicABELnoMM/src/Algorithm.cpp
>    pkg/OmicABELnoMM/src/Algorithm.h
>    pkg/OmicABELnoMM/src/Definitions.h
>    pkg/OmicABELnoMM/src/Utility.cpp
>    pkg/OmicABELnoMM/src/Utility.h
>    pkg/OmicABELnoMM/src/main.cpp
>    pkg/OmicABELnoMM/tests/Makefile
>    pkg/OmicABELnoMM/tests/test.cpp
> Log:
> Added R2, T-stat, Pval functions, but inactive atm. Added exclusion list support to remove individuals. Fixed propagation of missing values. Performance Improvements.
>
> Modified: pkg/OmicABELnoMM/Makefile.am
> ===================================================================
> --- pkg/OmicABELnoMM/Makefile.am      2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/Makefile.am      2014-07-15 08:17:29 UTC (rev 1763)
> @@ -2,7 +2,7 @@
>
>  oanomm_headers = src/Definitions.h src/AIOwrapper.h src/Algorithm.h  \
>   src/Utility.h
> -oanomm_cpp = src/AIOwrapper.cpp src/Algorithm.cpp src/Utility.cpp    \
> +oanomm_cpp = src/AIOwrapper.cpp src/Algorithm.cpp src/Utility.cpp \
>   src/main.cpp
>
>  omicabelnomm_SOURCES = $(oanomm_headers) $(oanomm_cpp)
>
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac     2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/configure.ac     2014-07-15 08:17:29 UTC (rev 1763)
> @@ -17,11 +17,12 @@
>  # Set some default compile flags
>  if test -z "$CXXFLAGS"; then
>     # User did not set CXXFLAGS, so we can put in our own defaults
> -    CXXFLAGS="-g -O2"
> +    CXXFLAGS="-O3"
>  fi
>  if test -z "$CPPFLAGS"; then
>     # User did not set CPPFLAGS, so we can put in our own defaults
>      CPPFLAGS="-Wall -g -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> +    #CPPFLAGS="-Wall"
>  fi
>  # If CXXFLAGS/CPPFLAGS are already set AC_PROG_CXX will not overwrite them
>  # with its own defaults
> @@ -45,13 +46,18 @@
>     AC_MSG_ERROR([Unable to find a Lapack library])
>  ])
>
> +#Boost
>
> +
> +
>  # Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
>  AC_OPENMP
>  AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>
>
>
> +
> +
>  # Checks for header files.
>  AC_CHECK_HEADERS([limits.h stdlib.h string.h sys/time.h unistd.h])
>
>
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.cpp       2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.cpp       2014-07-15 08:17:29 UTC (rev 1763)
> @@ -28,16 +28,21 @@
>      pthread_barrier_init(&(FHandler.finalize_barrier),NULL,2);
>
>
> -    Fhandler->fakefiles = params.use_fake_files;
> +    Fhandler->fakefiles = params.use_fake_files;
> +
> +
> +
>
>
> +
>      if(!Fhandler->fakefiles)
>      {
>          Fhandler->fnameAL = params.fnameAL;
>          Fhandler->fnameAR = params.fnameAR;
>          Fhandler->fnameY = params.fnameY;
> -        Fhandler->fnameOutB = params.fnameOutB;
> +        Fhandler->fnameOutB = params.fnameOutB;
>
> +
>          Yfvi  = load_databel_fvi( (Fhandler->fnameY+".fvi").c_str() );
>          ALfvi = load_databel_fvi( (Fhandler->fnameAL+".fvi").c_str() );
>          ARfvi = load_databel_fvi( (Fhandler->fnameAR+".fvi").c_str() );
> @@ -46,7 +51,7 @@
>          params.t = Yfvi->fvi_header.numVariables;
>          params.l = ALfvi->fvi_header.numVariables;
>
> -     int opt_tb = 1000;
> +        int opt_tb = 1000;
>          int opt_mb = 1000;
>
>          params.mb = min(params.m, opt_tb);
> @@ -56,7 +61,22 @@
>      else
>      {
>
> -    }
> +    }
> +
> +    //params.fname_excludelist = "exclfile.txt";
> +    int excl_count = 0;
> +    Fhandler->excl_List = new list< pair<int,int> >();
> +
> +    if(params.fname_excludelist.size()==0)
> +    {
> +        (Fhandler->excl_List)->push_back( make_pair(0,params.n) );
> +    }
> +    else
> +    {
> +        read_excludeList( Fhandler->excl_List,excl_count,params.n,params.fname_excludelist);
> +    }
> +
> +    params.n -= excl_count;
>
>      params.p = params.l + params.r;
>
> @@ -106,7 +126,9 @@
>      pthread_cond_destroy(&(Fhandler->condition_more));
>
>      pthread_mutex_destroy(&(Fhandler->m_read));
> -    pthread_cond_destroy(&(Fhandler->condition_read));
> +    pthread_cond_destroy(&(Fhandler->condition_read));
> +
> +    delete Fhandler->excl_List;
>
>
>  }
> @@ -186,12 +208,15 @@
>          }
>          //cout << "\nEnd preping files\n" << flush;
>
> -    }
> +    }
> +
> +    //pthread_barrier_wait(&(Fhandler->finalize_barrier));//for testing only
>
>
>      Fhandler->not_done = true;
> -    Fhandler->reset_wait = false;
> +    Fhandler->reset_wait = false;
>
> +
>      while(Fhandler->not_done)
>      {
>
> @@ -226,9 +251,30 @@
>                  Fhandler->seed += 75;
>              }
>              else
> -            {
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> -                result++;
> +            {
> +                list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +
> +                int chunk_size_buff;
> +                int buff_pos=0;
> +                int file_pos;
> +
> +                for(int i = 0; i < tmp_y_blockSize; i++)
> +                {
> +                    for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +                    {
> +                        file_pos = Fhandler->n*i + it->first;
> +                        fseek ( fp_Y , file_pos*sizeof(type_precision) , SEEK_SET );
> +                        chunk_size_buff = it->second;
> +
> +                        size_t result = fread (&tobeFilled->buff[buff_pos],sizeof(type_precision),chunk_size_buff,fp_Y); result++;
> +                        buff_pos += chunk_size_buff;
> +
> +
> +                    }
> +                }
> +//                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> +//                result++;
>                  if(Fhandler->y_to_readSize <= 0)
>                  {
>                      fseek ( fp_Y , 0 , SEEK_SET );
> @@ -267,17 +313,38 @@
>              if(Fhandler->fakefiles)
>              {
>                  fseek ( fp_Ar , 0 , SEEK_SET );
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -                result++;
> +                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar); result++;
>
>                  re_random_vec(tobeFilled->buff , Fhandler->n * tmp_ar_blockSize*Fhandler->r );
>                  re_random_vec_nan(tobeFilled->buff , Fhandler->n * tmp_ar_blockSize*Fhandler->r );
>
>              }
>              else
> -            {
> -                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -                result++;
> +            {
> +
> +                list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +                int chunk_size_buff;
> +                int buff_pos=0;
> +                int file_pos;
> +
> +                for(int i = 0; i < tmp_ar_blockSize*Fhandler->r; i++)
> +                {
> +                    for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +                    {
> +                        file_pos = Fhandler->n*i + it->first;
> +                        fseek ( fp_Ar , file_pos*sizeof(type_precision) , SEEK_SET );
> +
> +                        chunk_size_buff = it->second;
> +                        size_t result = fread (&tobeFilled->buff[buff_pos],sizeof(type_precision),chunk_size_buff,fp_Ar); result++;
> +                        buff_pos += chunk_size_buff;
> +
> +
> +                    }
> +                }
> +
> +//                size_t result = fread(tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> +//                result++;
>                  if (Fhandler->Ar_to_readSize <= 0)
>                  {
>                      fseek ( fp_Ar , 0 , SEEK_SET );
> @@ -307,8 +374,8 @@
>              if(Fhandler->fakefiles)
>              {
>                  fseek ( fp_B , 0 , SEEK_SET );
> -       }
> -             fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
> +            }
> +            fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
>
>
>              Fhandler->b_empty_buffers.push(tobeWritten);
> @@ -356,7 +423,8 @@
>      //cout << "k" << flush;
>      //barrier
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -
> +
> +    {
>      type_buffElement* tmp;
>      while(!Fhandler->full_buffers.empty())
>      {
> @@ -404,6 +472,7 @@
>         Fhandler->b_empty_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
> +    }
>      }
>
>
> @@ -544,13 +613,9 @@
>
>  void AIOwrapper::write_B(type_precision* B, int p, int blockSize)
>  {
> -     //int status;
> -    //int createstatus = 0;
> -   // cout << " b: full" << Fhandler->b_full_buffers.size() << "; epty" << Fhandler->b_empty_buffers.size() << endl;
>
>      while(Fhandler->b_empty_buffers.empty())
>      {
> -
>          pthread_mutex_lock(&(Fhandler->m_more));
>          pthread_cond_signal( &(Fhandler->condition_more ));
>          pthread_mutex_unlock(&(Fhandler->m_more));
> @@ -568,7 +633,7 @@
>
>
>
> -
> +        //cout << Fhandler->b_empty_buffers.size() << flush;
>          Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
>          Fhandler->b_empty_buffers.pop();
>
> @@ -582,12 +647,9 @@
>
>
>
> -    // cout << " b: full" << Fhandler->b_full_buffers.size() << "; epty" << Fhandler->b_empty_buffers.size() << endl;
> -
>      pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>
>
> -
>      pthread_mutex_lock(&(Fhandler->m_more));
>      pthread_cond_signal( &(Fhandler->condition_more ));
>      pthread_mutex_unlock(&(Fhandler->m_more));
> @@ -668,8 +730,12 @@
>  //        {
>  //            (tmp->buff)[i] = 0;
>  //        }
> -        Fhandler->b_empty_buffers.push(tmp);
> +        Fhandler->b_empty_buffers.push(tmp);
> +
> +//        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> +//        Fhandler->b_empty_buffers.pop();
>
> +
>      }
>
>  }
> @@ -819,17 +885,38 @@
>          (*AL) = Fhandler->AL;
>      }
>      else
> -    {
> +    {
>          FILE *fp;
>          fp = fopen((Fhandler->fnameAL+".fvd").c_str(), "rb");
>          if(fp == 0)
>          {
>              cout << "Error Reading File " << Fhandler->fnameAL << endl;
>              exit(1);
> -        }
> +        }
> +
> +        list< pair<int,int> >* excl_List = Fhandler->excl_List;
> +
> +        int chunk_size_buff;
> +        int buff_pos=0;
> +        int file_pos;
> +
> +        for (int i=0; i < Fhandler->l; i++)
> +        {
> +            for (list<  pair<int,int>  >::iterator it=excl_List->begin(); it != excl_List->end(); ++it)
> +            {
> +                file_pos = i*Fhandler->n+ it->first;
> +                fseek ( fp , file_pos*sizeof(type_precision) , SEEK_SET );
> +                chunk_size_buff = it->second;
> +
> +                size_t result = fread (&(Fhandler->AL[buff_pos]),sizeof(type_precision),chunk_size_buff,fp); result++;
> +                buff_pos += chunk_size_buff;
> +            }
> +        }
>
> -        size_t result = fread (Fhandler->AL,sizeof(type_precision),Fhandler->l*Fhandler->n,fp);
> -        result++;
> +
> +//        size_t result = fread (Fhandler->AL,sizeof(type_precision),Fhandler->l*Fhandler->n,fp);
> +//
> +//        result++;
>          fclose(fp);
>      }
>
> @@ -862,6 +949,111 @@
>  {
>      delete []Fhandler->AL;
>  }
> +
> +
> +void AIOwrapper::read_excludeList(list< pair<int,int> >* excl, int &excl_count, int max_excl, string fname_excludeList)
> +{
> +
> +    ifstream fp_exL(fname_excludeList.c_str());
> +    if(fp_exL == 0)
> +    {
> +        cout << "Error reading exclude list file."<< endl;
> +        exit(1);
> +    }
> +
> +
> +    string line;
> +    excl_count = 0;
> +    bool early_EOF;
> +    int first,second, prev_second;
> +
> +     cout << "Excluding Ids: \n";
> +
> +    std::getline(fp_exL, line);
> +    std::istringstream iss(line);
> +    iss >> first;
> +    early_EOF = iss.eof();
> +    iss >> prev_second;
> +
> +    if(prev_second < first || early_EOF)
> +            prev_second = first;
> +
> +    second = prev_second;
> +
> +
> +    if(first > max_excl)
> +    {
> +        excl->push_back( make_pair(0,max_excl) );
> +        cout << "\nNothing to Exclude!\n";
> +    }
> +    else
> +    {
> +        if(second > max_excl)
> +        {
> +                excl_count += max_excl-first+1;
> +                excl->push_back( make_pair(0,first-1) );
> +                cout << first << "-" << second << ", ";
> +        }
> +        else
> +        {
> +
> +            cout << first << "-" << second << ", ";
> +            excl->push_back( make_pair(0,first - 1) );
> +            excl_count += second-(first-1);
> +
> +            while (std::getline(fp_exL, line) && second < max_excl )
> +            {
> +                std::istringstream iss(line);
> +
> +                iss >> first;
> +                early_EOF = iss.eof();
> +                iss >> second;
> +                if(prev_second >= first)
> +                {
> +                    cout << "\nPlease give an ordered Exlusion List!\n";
> +                    cout << "?? " << prev_second << "\n" << first << " ??"<< endl;
> +                    exit( 1 );
> +                }
> +
> +
> +                if(second < first || early_EOF )
> +                    second = first;
> +
> +
> +                if(second > max_excl)
> +                    excl_count += max_excl-first+1;
> +                else if(first < max_excl)
> +                    excl_count += second-(first-1);
> +
> +
> +
> +                cout << first << "-" << second << ", ";
> +
> +                excl->push_back( make_pair(prev_second,first - prev_second - 1) );
> +
> +
> +                prev_second = second;
> +
> +
> +            }
> +        }
> +    }
> +
> +
> +
> +
> +    if(excl_count >= max_excl)
> +    {
> +             cout << "\nExclusion List excluded all data!\n";
> +             cout << "Total Ids: " << max_excl << "\nExcluded Ids: " << excl_count << endl;
> +             exit( 1 );
> +     }
> +      cout << "Excluded: "  << excl_count << " Using: " << max_excl-excl_count<< endl;
> +
> +
> +
> +}
> +
>
>  void AIOwrapper::free_databel_fvi( struct databel_fvi **fvi )
>  {
>
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.h 2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.h 2014-07-15 08:17:29 UTC (rev 1763)
> @@ -2,7 +2,9 @@
>  #define AIOWRAPPER_H
>
>  #include "Definitions.h"
> -#include "Utility.h"
> +#include "Utility.h"
> +#include <fstream>
> +#include <sstream>
>
>  typedef struct BufferElement type_buffElement;
>
> @@ -20,9 +22,15 @@
>  {
>      string fnameAL;
>      string fnameAR;
> -    string fnameY;
> -    string fnameOutB;
> +    string fnameY;
> +
>
> +    string fnameOutB;
> +
> +
> +    list< pair<int,int> >* excl_List;
> +
> +
>      bool doublefileType;
>      bool fakefiles;
>
> @@ -138,7 +146,9 @@
>
>      protected:
>
> -    private:
> +    private:
> +
> +        void read_excludeList(list< pair<int,int> >* excl, int &excl_count, int max_excl, string fname_excludeList);
>
>
>          void prepare_AR( int desired_blockSize, int n, int totalR, int columnsR);
> @@ -163,7 +173,7 @@
>
>          void * fgls_malloc_impl( const char* file, long line, size_t size );
>
> -        type_fileh FHandler;
> +            public: type_fileh FHandler;
>          type_fileh* Fhandler;
>
>          FILE* fp_Ar;
>
> Modified: pkg/OmicABELnoMM/src/Algorithm.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/Algorithm.cpp        2014-07-08 13:16:45 UTC (rev 1762)
> +++ pkg/OmicABELnoMM/src/Algorithm.cpp        2014-07-15 08:17:29 UTC (rev 1763)
> @@ -1,5 +1,7 @@
> -#include "Algorithm.h"
> +#include "Algorithm.h"
> +
>
> +
>  Algorithm::Algorithm()
>  {
>      // ctor
> @@ -51,41 +53,39 @@
>  }
>
>
> -void Algorithm::prepare_Bfinal(type_precision* bfinal, type_precision* top,
> -                               type_precision* bot, int dim1_b,
> -                               int dim2_b, int dim1_b_bot)
> +void Algorithm::prepare_Bfinal(type_precision* bfinal, type_precision* bsource, int a_amount, int y_amount, int p)
>  {
> -    // memcpy are faster version of the fors
> -    int i, k, w, top_idx, bot_idx;
> -    int size;
> -    top_idx = 0;
> -    bot_idx = 0;
> -    for (k = 0; k < dim2_b; k++)
> -    {
> -        size = k * dim1_b + (dim1_b - dim1_b_bot) - (k * dim1_b);
> -        memcpy( (type_precision*)&bfinal[k * dim1_b],
> -                (type_precision*)&top[top_idx],
> -                size * sizeof(type_precision) );
> -//        for (i = k*dim1_b; i < k*dim1_b+(dim1_b-dim1_b_bot); i++)
> -//        {
> -//            bfinal[i] = top[top_idx];
> -//            top_idx++;
> -//        }
> -        top_idx += size;
> -        i = k * dim1_b + size;
> -        w = i;
> -
> -        size = w + dim1_b_bot - w;
> -        memcpy( (type_precision*)&bfinal[w],
> -                (type_precision*)&bot[bot_idx],
> -                size * sizeof(type_precision) );
> -//        for (i = w; i < w+dim1_b_bot; i++)
> -//        {
> -//            bfinal[i] = bot[bot_idx];
> -//            bot_idx++;
> -//        }
> -        bot_idx += size;
> -    }
> +//    // memcpy are faster version of the fors
> +//    int i, k, w, top_idx, bot_idx;
> +//    int size;
> +//    top_idx = 0;
> +//    bot_idx = 0;
> +//    for (k = 0; k < dim2_b; k++)
> +//    {
> +//        size = k * dim1_b + (dim1_b - dim1_b_bot) - (k * dim1_b);
> +//        memcpy( (type_precision*)&bfinal[k * dim1_b],
> +//                (type_precision*)&top[top_idx],
> +//                size * sizeof(type_precision) );
> +////        for (i = k*dim1_b; i < k*dim1_b+(dim1_b-dim1_b_bot); i++)
> +////        {
> +////            bfinal[i] = top[top_idx];
> +////            top_idx++;
> +////        }
> +//        top_idx += size;
> +//        i = k * dim1_b + size;
> +//        w = i;
> +//
> +//        size = w + dim1_b_bot - w;
> +//        memcpy( (type_precision*)&bfinal[w],
> +//                (type_precision*)&bot[bot_idx],
> +//                size * sizeof(type_precision) );
> +////        for (i = w; i < w+dim1_b_bot; i++)
> +////        {
> +////            bfinal[i] = bot[bot_idx];
> +////            bot_idx++;
> +////        }
> +//        bot_idx += size;
> +//    }
>  }
>
>
> @@ -249,12 +249,13 @@
>
>      lapack_int info = LAPACKE_sgels(STORAGE_TYPE, 'N', rowsA, colsA, rhs, A,
>                                      rowsA, ynew, rowsA);
> -    assert(info == 0, "Error Check");
> +    myassert(info == 0, "Error Check");
>
>
>      int index = 0;
>      int index_new = 0;
> -    for (i = 0; i < rhs; i++)
> +    for (i = 0; i < rhs; i++)
> +
>      {
>           copy_vec(&ynew[index], &new_sol[index_new], colsA);
>           index += rowsA;
> @@ -305,13 +306,14 @@
>
>      srand(time(NULL));
>
> -    blas_set_num_threads(max_threads);
> +    blas_set_num_threads(max_threads);
> +    omp_set_num_threads(max_threads);
>
>
>      //type_precision *Ytemp;
>      lapack_int info, n, lda, l, r, p;
>
> -    cputime_type start_tick, start_tick2, end_tick;
> +    cputime_type start_tick, start_tick2, start_tick3, end_tick;
>
>      AIOfile.initialize(params);
>
> @@ -333,54 +335,70 @@
>
>      for (int j = 0; j < y_iters && !params.ForceCheck; j++)
>      {
> -        if (y_iters >= 40 && (j%(y_iters/40)) == 0)
> +        if (!params.ForceCheck &&  y_iters >= 10 && (j%(y_iters/10)) == 0 )
>          {
>              cout << "*" << flush;
>          }
>
>          for (int i = 0; i < a_iters; i++)
>          {
> -            for (int jj = 0; jj < y_block_size; jj++)
> -            {
> -                if (y_iters < 40 &&
> -                    (y_block_size < 3 || (jj%(y_block_size/3)) == 0)
> -                    )
> +
> +                if ( !params.ForceCheck &&  y_iters < 10 &&
> +                (  (a_iters >= 10 && (i%(a_iters/(10/y_iters))) == 0) || (a_iters < (10/y_iters)) ))
>                  {
>                      cout << "*" << flush;
>                  }
> -            }
> +
>          }
>      }
>
>      if(!params.ForceCheck)
>          cout << endl;
>
> -
> +    //add memalign
>
>      //type_precision Stl[l*l];
>      //type_precision Str[l*r*a_block_size];
> -    type_precision* Stl = new type_precision[l*l];
> -    type_precision* Str = new type_precision[l*r*a_block_size];
> +    type_precision* Stl = new type_precision[l*l*1];
> +    type_precision* Str = new type_precision[l*r*a_block_size*1];
>
>      type_precision* Sbr = new type_precision[r *  r * a_block_size];
> -    type_precision* Ay = new type_precision[p * a_block_size];
> +    type_precision* Ay = new type_precision[p * a_block_size];
> +    //type_precision* B_resorted = new type_precision[p * a_block_size*y_block_size];
>
> -    type_precision* S = new type_precision[p * p];
> +    type_precision* S = new type_precision[p * p];
>
> +
>      type_precision* Ay_top = new type_precision[l * y_amount];
>      type_precision* Ay_bot = new type_precision[y_block_size * a_block_size * r];
>
> -    list<long int>* y_nan_idxs = new list<long int>[y_block_size];
> +    type_precision* y_residual = new type_precision[n * y_block_size ];
> +    type_precision* y_res_norms = new type_precision[a_block_size];
> +
> +    list<long int>* al_nan_idxs = new list<long int>[1];
> +    list<long int>* y_nan_idxs = new list<long int>[y_block_size];
> +    list<long int>* ar_nan_idxs = new list<long int>[a_block_size];
>
> +
> +    type_precision* A = new type_precision[n * p * 1];
>      type_precision* AR = new type_precision[n * r * a_block_size * 1];
> -    type_precision* AL = new type_precision[n * l * 1];
> +//  type_precision* AL = new type_precision[n * l * 1];
> +    type_precision* AL = A;
> +
> +    type_precision* B = Ay;
> +
>      type_precision* backupAR;  // = new type_precision[n*r*a_block_size];
>      type_precision* backupAL;  // = new type_precision[n*l];
>
>
>      AIOfile.load_AL(&backupAL);
> -    //int total_al_nans = replace_nans(0, backupAL, n, l);
> -    replace_nans(0, backupAL, n, l);
> +
> +    //pthread_barrier_wait(&(AIOfile.Fhandler->finalize_barrier));
> +
> +    replace_nans(al_nan_idxs,1, backupAL, n, l);
> +    al_nan_idxs->push_back(1);
> +
> +    //LAPACKE_dgesdd()
>
>      copy_vec(backupAL, AL, n*l);
>
> @@ -394,176 +412,369 @@
>
>      for (int j = 0; j < y_iters; j++)
>      {
> -        if (y_iters >= 40 && (j%(y_iters/40)) == 0 && !params.ForceCheck)
> +        if (!params.ForceCheck &&  y_iters >= 10 && (j%(y_iters/10)) == 0 && !params.ForceCheck)
>          {
>              cout << AIOfile.io_overhead << flush;
>              AIOfile.io_overhead = "*";
>          }
> +
> +        get_ticks(start_tick2);
>
> -        AIOfile.load_Yblock(&Y, y_block_size);
> +        AIOfile.load_Yblock(&Y, y_block_size);
> +
> +        get_ticks(end_tick);
> +        out.acc_loady += ticks2sec(end_tick,start_tick2);
>
> +        get_ticks(start_tick2);
> +        replace_nans(&y_nan_idxs[0],y_block_size, Y, n,1);
> +        //out.acc_other += ticks2sec(end_tick,start_tick2);
>
>
> -        //int total_y_nans = replace_nans(&y_nan_idxs[0], Y, n, y_block_size);
> -        replace_nans(&y_nan_idxs[0], Y, n, y_block_size);
> +        get_ticks(start_tick2);
>
> -
>          //! Ay_top = AL'*Y
>          cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
>                      l, y_block_size, n, 1.0, AL, n, Y, n, 0.0,
> -                    &Ay_top[j * l * y_block_size], l);
> +                    &Ay_top[j * l * y_block_size], l);
> +
> +        get_ticks(end_tick);
> +        out.acc_gemm += ticks2sec(end_tick,start_tick2);
>
>
>          for (int i = 0; i < a_iters; i++)
> -        {
> -            AIOfile.load_ARblock(&backupAR, a_block_size);
> -            //int total_ar_nans = replace_nans(0, backupAR, n, a_block_size * r);
> -            replace_nans(0, backupAR, n, a_block_size * r);
> -            copy_vec(backupAR, AR, n *  r * a_block_size);
> +        {
> +
> +            if (!params.ForceCheck && y_iters < 10 &&
> +                (  (a_iters >= 10 && (i%(a_iters/(10/y_iters))) == 0) || (a_iters < (10/y_iters)) ))
> +            {
> +                cout << "*" << flush;
> +            }
> +
> +            get_ticks(start_tick2);
>
> -            get_ticks(start_tick2);
> +            AIOfile.load_ARblock(&backupAR, a_block_size);
> +
> +            get_ticks(end_tick);
> +            out.acc_loadxr += ticks2sec(end_tick,start_tick2);
> +
> +            get_ticks(start_tick2);
> +
> +            replace_nans(&ar_nan_idxs[0],a_block_size, backupAR, n, r);
> +            replace_with_zeros(al_nan_idxs, backupAR,  n, r, a_block_size);
>
> +            copy_vec(backupAR, AR, n *  r * a_block_size);
> +
>              get_ticks(end_tick);
> -            out.acc_pre += ticks2sec(end_tick-start_tick2, cpu_freq);
> +            //out.acc_other += ticks2sec(end_tick,start_tick2);
>
> -            get_ticks(start_tick2);
> +
> +
> +            get_ticks(start_tick2);
> +
>              //! Ay_bot = AR'*Y
>              cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
>                          r * a_block_size, y_block_size, n, 1.0, AR, n, Y, n,
>                          0.0, Ay_bot, r * a_block_size);
>
>              get_ticks(end_tick);
> -            out.firstloop += ticks2sec(end_tick - start_tick2, cpu_freq);
> +            out.acc_gemm += ticks2sec(end_tick,start_tick2);
> +
> +            int aL_idx =  0*l * n;
> +            int aR_idx = 0* r * n * a_block_size;
>
> -            //#pragma omp parallel default(shared)
> -            {
> -                for (int jj = 0; jj < y_block_size; jj++)
> -                {
> -                    int thread_id = 0 * omp_get_thread_num();
> -                    int aL_idx = thread_id * l * n;
> -                    int aR_idx = thread_id * r * n * a_block_size;
> +            get_ticks(start_tick3);
> +            for (int jj = 0; jj < y_block_size; jj++)
> +            {
> +
> +                //int thread_id = 0 * omp_get_thread_num();//so far singel thread version only
>
> -                    if (y_iters < 40 &&
> -                                  (y_block_size < 3 ||
> -                                                  (jj%(y_block_size/3)) == 0) && !params.ForceCheck)
> -                    {
> -                        cout << AIOfile.io_overhead << flush;
> -                        AIOfile.io_overhead = "*";
> -                    }
> +
> +                get_ticks(start_tick2);
> +
> +                copy_vec(backupAL, &AL[aL_idx], n * l);//try to remove!
>
> -                    copy_vec(backupAL, &AL[aL_idx], n * l);
> +                replace_with_zeros(&y_nan_idxs[jj], &AL[aL_idx], n, l, 1);
> +
> +
> +                get_ticks(end_tick);//2%
> +                out.acc_other += ticks2sec(end_tick,start_tick2);
> +
> +                 get_ticks(start_tick2);
>
> -                    replace_with_zeros(&y_nan_idxs[jj], &AL[aL_idx], n, l, 1);
> -                    //! Generate Stl
> -                    cblas_ssyrk(CblasColMajor, CblasUpper, CblasTrans,
> -                                l, n, 1.0, &AL[aL_idx], lda, 0.0, Stl, l);
> +                //! Generate Stl
> +                cblas_ssyrk(CblasColMajor, CblasUpper, CblasTrans,
> +                            l, n, 1.0, &AL[aL_idx], lda, 0.0, Stl, l);
> +
> +                get_ticks(end_tick);
> +                out.acc_stl += ticks2sec(end_tick,start_tick2);
> +
> +
> +                get_ticks(start_tick2);
> +
> +                copy_vec(backupAR,&AR[aR_idx], n*r*a_block_size);//!10%//try to remove!
> +
> +
> +                replace_with_zeros(&y_nan_idxs[jj], backupAR,  n, r, a_block_size);
>
> -                    copy_vec(backupAR,&AR[aR_idx], n*r*a_block_size);
> -                    replace_with_zeros(&y_nan_idxs[jj], &AR[aR_idx],
> -                                       n, r, a_block_size);
> +
> +
> +                get_ticks(end_tick);
> +                out.acc_other += ticks2sec(end_tick,start_tick2);
> +
> +                get_ticks(start_tick2);
>
> -                    get_ticks(start_tick2);
> -                    //! Generate Str
> -                    cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> -                                l, r * a_block_size, n, 1.0, &AL[aL_idx],
> -                                n, &AR[aR_idx], n, 0.0, Str, l);
> -                    get_ticks(end_tick);
> -                    out.acc_RTL_QLY += ticks2sec(end_tick - start_tick2,
> -                                                 cpu_freq);
> +                //! Generate Str
> +                cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> +                            l, r * a_block_size, n, 1.0, &AL[aL_idx],
> +                            n, &AR[aR_idx], n, 0.0, Str, l);//!45
>
> +                get_ticks(end_tick);
> +                out.acc_str += ticks2sec(end_tick,start_tick2);
> +
>
> -                    //type_precision Sbr[r *  r * a_block_size];
> -                    //type_precision Ay[p * a_block_size];
>
> +                //type_precision Sbr[r *  r * a_block_size*1];
> +                //type_precision Ay[p * a_block_size*1];
> +
> +                blas_set_num_threads(1);
> +                omp_set_num_threads(max_threads);
> +
> +                #pragma omp parallel default(shared)
> +                {
>
> -                    //#pragma omp for nowait  schedule(dynamic)
> -                    for (int ii= 0; ii < a_block_size; ii++)
> -                    {
> -                        get_ticks(start_tick2);
> -                        //! Generate Sbr
> -                        cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> -                                    r, r, n, 1.0, &AR[aR_idx+ii*r*n], n,
> -                                    &AR[aR_idx + ii * r * n], n, 0.0,
> -                                    &Sbr[ii * r * r], r);
> -                        get_ticks(end_tick);
> -                        out.acc_gemm += ticks2sec(end_tick - start_tick2,
> -                                                  cpu_freq);
> +                #pragma omp for nowait
> +                for (int ii= 0; ii < a_block_size; ii++)
> +                {
> +                    //cout << omp_get_thread_num() << endl << flush;
>
> -                        get_ticks(start_tick2);
>
> -                        copy_vec(&Ay_top[j*l*y_block_size+jj*l], &Ay[ii*p], l);
> -                        copy_vec(&Ay_bot[ii*r + jj*r*a_block_size],
> -                                 &Ay[l+ii*p], r);
> +                    get_ticks(start_tick2);
>
> +                    //! Generate Sbr
> +                    cblas_sgemm(CblasColMajor, CblasTrans, CblasNoTrans,
> +                                r, r, n, 1.0, &AR[aR_idx+ii*r*n], n,
> +                                &AR[aR_idx + ii * r * n], n, 0.0,
> +                                &Sbr[ii * r * r], r);
> +
>
> -                        //type_precision* B = Ay;
> -                        //type_precision S[p * p];
> +                    get_ticks(end_tick);
> +                    out.acc_sbr += ticks2sec(end_tick,start_tick2 );
> +
> +                    get_ticks(start_tick2);
>
> +                    copy_vec(&Ay_top[j*l*y_block_size+jj*l], &Ay[ii*p], l);
> +                    copy_vec(&Ay_bot[ii*r + jj*r*a_block_size],
> +                             &Ay[l+ii*p], r);
> +
> +
> +                    type_precision S2[p * p];
>
> -                        //! Rebuild S
> -                        build_S(S, Stl, &Str[ii*r*l], &Sbr[ii*r*r], l, r);
> -                        // matlab_print_matrix("S", p, p, S);
> +                    //! Rebuild S
> +                    build_S(S2, Stl, &Str[ii*r*l], &Sbr[ii*r*r], l, r);
> +                    // matlab_print_matrix("S", p, p, S);
>
> -                        get_ticks(end_tick);
> -                        out.acc_b += ticks2sec(end_tick - start_tick2,
> -                                               cpu_freq);
> +
> +                    get_ticks(end_tick);//5%
> +                    out.acc_other += ticks2sec(end_tick,start_tick2);
>
> -                        get_ticks(start_tick2);
> +                    get_ticks(start_tick2);
>
> -                        //! b = S\Ay
> -                        info = LAPACKE_sposv(STORAGE_TYPE, 'U', p, 1, S, p,
> -                                             &Ay[ii*p], p);
>
> -                        get_ticks(end_tick);
> -                        out.acc_loadxr += ticks2sec(end_tick - start_tick2,
> -                                                    cpu_freq);
> -                        assert(info == 0, "POSV");
> +                    //! b = S\Ay
> +                    info = LAPACKE_sposv(STORAGE_TYPE, 'U', p, 1, S2, p,
> +                                         &Ay[ii*p], p);
> +
> +                    myassert(info == 0, "S\\Ay");
> +
>
> -                        if (params.ForceCheck)
> +                    get_ticks(end_tick);
> +                    out.acc_solve += ticks2sec(end_tick,start_tick2 );
> +
> +
> +                    if (params.ForceCheck)
> +                    {
> +                        #pragma omp critical
>                          {
> -                            #pragma omp critical
> -                            {
> -                                check_result(AL, &AR[aR_idx+ii*r*n], n, p,
> -                                             1, r, &Y[jj*n], &Ay[ii*p]);
> -                            }
> +                            check_result(AL, &AR[aR_idx+ii*r*n], n, p,
> +                                         1, r, &Y[jj*n], &Ay[ii*p]);
>                          }
>                      }
> +                }
> +                }
> +
> +                /************statistics**************/
> +//                type_precision* T;
> +//                type_precision* R2;
> +//                type_precision* P;
> +
> +                //hpc_statistics(n,A,a_block_size,Y,y_block_size,B,p,T,R2,P);
> +
> +                /**************************/
> +
> +                blas_set_num_threads(max_threads);
> +
> +                get_ticks(start_tick2);
>
> -                    AIOfile.write_B(Ay, p, a_block_size);
> -                }
> -            }
> +                AIOfile.write_B(B, p, a_block_size);
> +
> +                get_ticks(end_tick);
> +                out.acc_storeb += ticks2sec(end_tick,start_tick2);
> +
> +
> +
> +
> +
> +            }
> +
> +            get_ticks(end_tick);
> +            out.acc_real_innerloops += ticks2sec(end_tick ,start_tick3);
> +
> +
>          }
>           AIOfile.reset_AR();
>      }
>
> -    get_ticks(end_tick);
> -    out.duration = ticks2sec(end_tick - start_tick, cpu_freq);
> -    // out.gflops = a_amount/1000.0*(n*l*r+n*r*r+y_amount*(n*r+p*p*p)))/1000.0/1000.0;
> -    out.gflops = y_amount * (gemm_flops(l, n, 1, 0) +
> -                             a_amount * (gemm_flops(r, n, 1, 0) +
> -                                         gemm_flops(l, n, l, 0) +
> -                                         gemm_flops(l, n, r, 0) +
> -                                         gemm_flops(r, n, r, 0) +
> +    get_ticks(end_tick);
> +
> +    out.duration = ticks2sec(end_tick ,start_tick);
> +
> +
> +    out.gflops = y_iters * (gemm_flops(l, n, params.tb*1, 0) +
> +                             a_iters * ( gemm_flops(params.mb*r, n, params.tb*1, 0) +
> +                             params.tb *(  gemm_flops(l, 1*n, l, 0) + gemm_flops(l, n, params.mb *r, 0) +
> +                                    params.mb * (
> +                                         gemm_flops(r, 1*n, r, 0) +
>                                           (p * p * p / 3.0) /
> -                                         1000.0/1000.0/1000.0));
> [TRUNCATED]
>
> To get the complete diff run:
>     svnlook diff /svnroot/genabel -r 1763
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
>

--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-


From yurii.aulchenko at gmail.com  Wed Jul 23 17:34:13 2014
From: yurii.aulchenko at gmail.com (Yury Aulchenko)
Date: Wed, 23 Jul 2014 17:34:13 +0200
Subject: [GenABEL-dev] Fwd: Patch for DatABEL for checkPackageVersionOnCRAN
References: <CAOPCvxup6KJyVb+DV2K8JFr0LUSiRB_xv64YoQeeoE3J=VJKOQ@mail.gmail.com>
Message-ID: <C1E898C2-9451-498A-84AB-401AF3DEBAA0@gmail.com>

Thank you very much, Marco!

Yurii

Begin forwarded message:

> From: Marco De Simone <marco.desimone at cbm.fvg.it>
> Subject: Patch for DatABEL for checkPackageVersionOnCRAN
> Date: July 23, 2014 at 17:09:13 GMT+2
> To: yurii at bionet.nsc.ru
> 
> Dear Yurii,
> we had a problem installing DatABEL since the check on the package
> version returned always a NULL string and so the install returned a
> non-zero value
> 
> ** testing if installed package can be loaded
> Error : .onAttach failed in attachNamespace() for 'DatABEL', details:
>  call: if (pkgVersion != cranVersion) {
>  error: argument is of length zero
> 
> I tried to debug the checkPackageVersionOnCRAN and I did a little
> change now it works:
> 
> the patch look for the html cell  Version: and select its siblings
> that contain the version string, than strips away the html td tags..
> 
> Hope it helps
> 
> kindly regards
> Marco
> -- 
> Marco De Simone
> 
> Servizi Informativi
> CBM scrl
> Area Science Park
> S.S. 14 km 163.5, 34149 Basovizza, Trieste
> tel. 040 375 7721
> fax 040 373 7710
> cell. 335 1298 065
> email: marco.desimone at cbm.fvg.it

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140723/b1248c14/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DatABEL.checkPackageVersionOnCRAN.patch
Type: text/x-patch
Size: 1142 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140723/b1248c14/attachment.bin>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140723/b1248c14/attachment-0001.html>

From alvaro.frank at rwth-aachen.de  Fri Jul 25 13:00:17 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Fri, 25 Jul 2014 11:00:17 +0000
Subject: [GenABEL-dev] ACML and make check
Message-ID: <244CF001646FF74FB34F372310A332C50115301D@MBX-S2.rwth-ad.de>

Hi Lennart,

I committed an ugly/nice hack to allow the usage of AMD's ACML library with omicabelnomm. This was required due to the program having VERY bad performance issues making it unusable on old Opteron systems prior to the 6200 versions. This are all that have non AVX instructions. I might have broken a few Jenkins warnings due to this, but those will be fixed later.
Normal OMICABEL should be tested against ACML this too. Using MKL on the Opteron System, with 10 traits and 400000 snps. the runtime takes over 10 hours. This does not seem like a normal behavior.

AMD ACMl also allows to use GPU without ANY change, the system just needs to support OpenCL, which NVIDIA and AMD do. I have no access to an Opteron/AMD system that has GPU so I cant test it yet.

The changes also allow for static linking now. So no need to have root access. The howtocompile file reflects this now.

I also have a very improtant request. Your change of make check will not display the output of the testing program as it runs. I previosly compiled the test file and manually called it, giving me a very quick workflow to test the system. But now my 1 cmd do it all workflow is broken, having to use make check to compile test.cpp and then having to abort it manually to later re-call it manually to be able to see the output in real time. Is there a way to change make check to allow for real time output? I tried gooooogoling this, but failed with the most common solutions.

A rewrite of Algorithm.cpp is coming next. it will break the validity/give wrong results perhaps at first.

Thank You.

Alvaro Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/2476f099/attachment.html>

From lennart at karssen.org  Fri Jul 25 15:38:07 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 15:38:07 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1768 - in pkg/OmicABELnoMM: .
	doc src tests
In-Reply-To: <20140725104804.2C8FF186938@r-forge.r-project.org>
References: <20140725104804.2C8FF186938@r-forge.r-project.org>
Message-ID: <53D25DBF.9010807@karssen.org>

Hi Alvaro,

Please see below.

On 25-07-14 12:48, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-25 12:48:03 +0200 (Fri, 25 Jul 2014)
> New Revision: 1768
> 
> Modified:
>    pkg/OmicABELnoMM/configure.ac
>    pkg/OmicABELnoMM/doc/howtocompile.txt
>    pkg/OmicABELnoMM/src/AIOwrapper.cpp
>    pkg/OmicABELnoMM/src/AIOwrapper.h
>    pkg/OmicABELnoMM/src/Algorithm.cpp
>    pkg/OmicABELnoMM/src/Definitions.h
>    pkg/OmicABELnoMM/src/Utility.cpp
>    pkg/OmicABELnoMM/src/main.cpp
>    pkg/OmicABELnoMM/test-driver
>    pkg/OmicABELnoMM/tests/test.cpp
> Log:
> Added AMD ACML 6 support for heteregenous systems (CPU + GPU). ACML fixes unusable performance issues under older Opteron (non AVX) systems. Several Bug fixes of IO. Added incomplete functionalities.
> 
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/configure.ac	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -3,8 +3,9 @@
>  
>  AC_PREREQ([2.67])
>  AC_INIT([OmicABELnoMM], [0.1.0], [genabel-devel at r-forge.wu-wien.ac.at])
> -AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> -AM_SILENT_RULES([yes])
> +AM_INIT_AUTOMAKE([])
> +#AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> +#AM_SILENT_RULES([yes])

I'm wondering why you commented these lines. Given that we have one
Makefile.am for the whole project, including subdirs, the subdir-objects
option needs to be enabled. autoreconf will also warn you about it:

$ autoreconf -i
Makefile.am:3: warning: source file 'src/AIOwrapper.cpp' is in a
subdirectory,
Makefile.am:3: but option 'subdir-objects' is disabled
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the
'subdir-objects'
automake: automake option hasn't been enabled.  For now, the
corresponding output
automake: object file(s) will be placed in the top-level directory.
However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same
subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option
throughout your
automake: project, to avoid future incompatibilities.
Makefile.am:3: warning: source file 'src/Algorithm.cpp' is in a
subdirectory,
Makefile.am:3: but option 'subdir-objects' is disabled
Makefile.am:3: warning: source file 'src/Utility.cpp' is in a subdirectory,
Makefile.am:3: but option 'subdir-objects' is disabled
Makefile.am:6: warning: source file 'src/main.cpp' is in a subdirectory,
Makefile.am:6: but option 'subdir-objects' is disabled
Makefile.am:12: warning: source file 'tests/test.cpp' is in a subdirectory,
Makefile.am:12: but option 'subdir-objects' is disabled


I guess you wanted to see the actual compiler options. If that's the
case you have two options:
run 'configure --disable-silent-rules'
or change the configure.ac line to no: AM_SILENT_RULES([no])


Best,

Lennart.

>  AC_CONFIG_SRCDIR([src/Utility.h])
>  AC_CONFIG_HEADERS([src/config.h])
>  
> @@ -17,12 +18,12 @@
>  # Set some default compile flags
>  if test -z "$CXXFLAGS"; then
>     # User did not set CXXFLAGS, so we can put in our own defaults
> -    CXXFLAGS="-O3"
> +    CXXFLAGS=""
>  fi
>  if test -z "$CPPFLAGS"; then
>     # User did not set CPPFLAGS, so we can put in our own defaults
> -    CPPFLAGS="-Wall -g -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> -    #CPPFLAGS="-Wall"
> +    #CPPFLAGS="-Wall -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> +    CPPFLAGS="-O0"
>  fi
>  # If CXXFLAGS/CPPFLAGS are already set AC_PROG_CXX will not overwrite them
>  # with its own defaults
> @@ -31,33 +32,49 @@
>  AC_PROG_CC
>  AC_PROG_CXX
>  
> +# Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
> +AC_OPENMP
> +AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>  
> +AM_CXXFLAGS="-static -O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
> +
>  # Checks for libraries.
>  # pthread library
>  AC_SEARCH_LIBS([pthread_mutex_init], [pthread], [], [
> -   AC_MSG_ERROR([Unable to find the pthread_mutex_init() function])
> +   AC_MSG_ERROR([Make sure pthread is available on the system])
>  ])
> -# Openblas
> -AC_SEARCH_LIBS([cblas_sgemm], [openblas], [], [
> -   AC_MSG_ERROR([Unable to find the openblas library])
> -])
> -# Lapack
> -AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> -   AC_MSG_ERROR([Unable to find a Lapack library])
> -])
>  
> -#Boost
> +if test -z "$LDFLAGS"; then
> +	LDFLAGS="-L./libs/lib/ -L../libs/lib/"
> +fi
>  
> +found_blas=0
>  
> +AC_SEARCH_LIBS([__iso_c_binding_c_f_pointer_l4],[gfortran])
>  
> -# Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
> -AC_OPENMP
> -AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
> +# ACML
> +AC_SEARCH_LIBS(dgemm, acml_mp,[found_blas=1 AM_CXXFLAGS="-D_acml_ $AM_CXXFLAGS"], [
> +   AC_MSG_NOTICE([NOT using AMD the ACML  library],[-lgfortran])
> +])
>  
>  
>  
> +if test "$found_blas" -eq 0
> +	then
>  
> +# Openblas
> +AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> +   AC_MSG_ERROR([OpenBLAS  library NOT found])
> +])
> + #Lapack
> +AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> +   AC_MSG_ERROR([Unable to find a Lapack library])
> +])
> +AC_MSG_NOTICE([Using OpenBLAS  library])
> +fi
>  
> +
> +
>  # Checks for header files.
>  AC_CHECK_HEADERS([limits.h stdlib.h string.h sys/time.h unistd.h])
>  
> @@ -73,6 +90,7 @@
>  # Files to be generated by autotools
>  AC_CONFIG_FILES([
>          Makefile
> +
>  ])
>  
>  AC_OUTPUT
> 
> Modified: pkg/OmicABELnoMM/doc/howtocompile.txt
> ===================================================================
> --- pkg/OmicABELnoMM/doc/howtocompile.txt	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/doc/howtocompile.txt	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -1,4 +1,7 @@
> -
> +
> +-------------------------Install BLAS----------------------------
> +#STEP 1:
> +#a) Install OPENBLAS:
>  mkdir GWAS_PROJECT
>  cd GWAS_PROJECT
>  
> @@ -10,21 +13,52 @@
>  #make sure g++ its 4.8 or above!on rwth cluster module load gcc/4.8, choose 32 or 64 also
>  make all HOSTCC=g++ FC=gfortran USE_OPENMP=1
>  
> -sudo make install PREFIX="/usr"
> +make install PREFIX="path_to_/OmicABELnoMM/libs/"
>  
> -sudo ldconfig
> +ldconfig
>  
> -cd ..
> -#BLAS END
>  
> +cd ..
> +
> +-------------------------AMD ACML Alternative Version-------------------------
> +#STEP 1:
> +# b)
> +mkdir OmicABELnoMM/libs/
> +
> +cd OmicABELnoMM/libs/
> +
> +You can use AMD ACML. Download from:
> +http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/acml-downloads-resources/
> +
> + and copy the supplied binary 
> + libraries to "OmicABELnoMM/libs/"
> +
> +IF both libraries are present (Openblas + ACML), the system will use ACML.
> +
> +cd ../../
> +
> +---------------------------------#BLAS END----------------------------------------------------
> +
> +#STEP 2:
> +
> +
> +ON the folder GWAS_PROJECT
> +
>  svn checkout svn+ssh://developername at svn.r-forge.r-project.org/svnroot/genabel/OmicABELnoMM
>  
>  cd OmicABELnoMM
> +
> +autoreconf -fi
>  
> -./configure LDFLAGS="-L/usr/lib/"
> +./configure
>  
> -make
> +make
>  
> +make check
> +
> +#DONE
> +
> +------------------------------Example--------------------------------
>  #test it
>  ./omicabelnomm -c examples/XL --geno examples/XR -p examples/Y -o examples/B -n 2 -t 2
>  
> @@ -42,12 +76,19 @@
>  
>  
>  
> --------------------------Alternative Version-------------------------
> +-------------------------Alternative Version of BLAS-------------------------
>  
> -Install all required libraries for your system: 
> +#Make sure autoconf is installed:
>  
> --Iinux ubuntu:
> +sudo apt-get install autoconf
> +autoreconf -fi
> +autoconf
>  
> +#Install all required libraries for your system, 
> +#but they will not work, 99% of the time, due to not having openmp support: 
> +
> +#Iinux ubuntu:
> +
>  sudo apt-get install libopenblas-dev
>  sudo apt-get install libopenblas-base
>  sudo apt-get install liblapack3gf
> @@ -56,11 +97,7 @@
>  sudo apt-get install liblapacke
>  sudo apt-get install liblapacke-dev
>  
> --Make sure autoconf is installed:
>  
> -sudo apt-get install autoconf
> -autoreconf -fi
> -autoconf
>  
>  
>  
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -40,7 +40,7 @@
>          Fhandler->fnameAL = params.fnameAL;
>          Fhandler->fnameAR = params.fnameAR;
>          Fhandler->fnameY = params.fnameY;
> -        Fhandler->fnameOutB = params.fnameOutB;
> +        Fhandler->fnameOutFiles = params.fnameOutFiles;
>  
>  
>          Yfvi  = load_databel_fvi( (Fhandler->fnameY+".fvi").c_str() );
> @@ -91,7 +91,7 @@
>  
>      prepare_AL(params.l,params.n);
>      prepare_AR(  params.mb,  params.n,  params.m,  params.r);
> -    prepare_B(params.tb, params.l+params.r);
> +    prepare_OutFiles(params.mb, params.l+params.r);
>      prepare_Y(params.tb, params.n, params.t);
>  
>  
> @@ -118,7 +118,7 @@
>      finalize_Y();
>      finalize_AR();
>      finalize_AL();
> -    finalize_B();
> +    finalize_OutFiles();
>  
>      pthread_attr_destroy(&(Fhandler->attr));
>  
> @@ -135,7 +135,7 @@
>  
>  
>  
> -void AIOwrapper::finalize_B()
> +void AIOwrapper::finalize_OutFiles()
>  {
>  
>  }
> @@ -149,7 +149,10 @@
>  
>      struct timespec timeToWait;
>      FILE*  fp_Y;
> -    FILE*  fp_B;
> +    FILE*  fp_B;
> +    FILE*  fp_R;
> +    FILE*  fp_SD2;
> +    FILE*  fp_P;
>      FILE*  fp_Ar;
>      if(!Fhandler->fakefiles)
>      {
> @@ -167,11 +170,29 @@
>              exit(1);
>          }
>  
> -        fp_B = fopen((Fhandler->fnameOutB+".fvd").c_str(), "w+b");
> +        fp_B = fopen((Fhandler->fnameOutFiles+"_B.fvd").c_str(), "w+b");
>          if(fp_B == 0)
>          {
> -            cout << "Error Opening File B " << Fhandler->fnameOutB << endl;
> +            cout << "Error Opening File B " << Fhandler->fnameOutFiles << "_B" << endl;
>              exit(1);
> +        }
> +        fp_R = fopen((Fhandler->fnameOutFiles+"_R.fvd").c_str(), "w+b");
> +        if(fp_R == 0)
> +        {
> +            cout << "Error Opening File R " << Fhandler->fnameOutFiles << "_R" << endl;
> +            exit(1);
> +        }
> +        fp_SD2 = fopen((Fhandler->fnameOutFiles+"_SD2.fvd").c_str(), "w+b");
> +        if(fp_SD2 == 0)
> +        {
> +            cout << "Error Opening File SD2 " << Fhandler->fnameOutFiles << "_SD2" << endl;
> +            exit(1);
> +        }
> +        fp_P = fopen((Fhandler->fnameOutFiles+"_P.fvd").c_str(), "w+b");
> +        if(fp_P == 0)
> +        {
> +            cout << "Error Opening File P " << Fhandler->fnameOutFiles << "_P" << endl;
> +            exit(1);
>          }
>      }
>      else
> @@ -205,6 +226,24 @@
>          {
>              cout << "Error setting up temp File B " << endl;
>              exit(1);
> +        }
> +        fp_R = fopen("tempR.bin", "w+b");
> +        if(fp_R == 0)
> +        {
> +            cout << "Error setting up temp File R " << endl;
> +            exit(1);
> +        }
> +        fp_SD2 = fopen("tempSD2.bin", "w+b");
> +        if(fp_SD2 == 0)
> +        {
> +            cout << "Error setting up temp File SD2 " << endl;
> +            exit(1);
> +        }
> +        fp_P = fopen("tempP.bin", "w+b");
> +        if(fp_P == 0)
> +        {
> +            cout << "Error setting up temp File P " << endl;
> +            exit(1);
>          }
>          //cout << "\nEnd preping files\n" << flush;
>  
> @@ -229,14 +268,16 @@
>  
>              Fhandler->y_to_readSize -= tmp_y_blockSize;
>              size_buff = Fhandler->n * tmp_y_blockSize;
> -            //cout << Fhandler->y_to_readSize << endl;
> +
>  
> +
>              pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -            //cout << " pre;" << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
> +
> +
>              type_buffElement* tobeFilled = Fhandler->empty_buffers.front();
>              Fhandler->empty_buffers.pop();
> -            //pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
> +
>              tobeFilled->size = tmp_y_blockSize;
>  
>              if(Fhandler->fakefiles)
> @@ -273,8 +314,8 @@
>  
>                      }
>                  }
> -//                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> -//                result++;
> +
> +
>                  if(Fhandler->y_to_readSize <= 0)
>                  {
>                      fseek ( fp_Y , 0 , SEEK_SET );
> @@ -282,10 +323,9 @@
>              }
>  
>  
> -            //pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +
>              Fhandler->full_buffers.push(tobeFilled);
> -            //  cout << "\nStoring " << tobeFilled << endl;
> -            //cout << " post;" << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
> +
>              pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
>              pthread_mutex_lock(&(Fhandler->m_read));
> @@ -294,8 +334,12 @@
>  
>          }
>  
> -        while(!Fhandler->ar_empty_buffers.empty() && Fhandler->Ar_to_readSize)
> -        {
> +        while(!Fhandler->ar_empty_buffers.empty() && Fhandler->Ar_to_readSize )
> +        {
> +
> +
> +
> +
>              tmp_ar_blockSize = Fhandler->Ar_blockSize;
>              if(Fhandler->Ar_to_readSize < Fhandler->Ar_blockSize)
>                  tmp_ar_blockSize = Fhandler->Ar_to_readSize;
> @@ -343,12 +387,13 @@
>                      }
>                  }
>  
> -//                size_t result = fread(tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -//                result++;
> -                if (Fhandler->Ar_to_readSize <= 0)
> -                {
> -                    fseek ( fp_Ar , 0 , SEEK_SET );
> -                }
> +
> +            }
> +
> +            if(Fhandler->Ar_to_readSize <= 0)
> +            {
> +                Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +                fseek ( fp_Ar , 0 , SEEK_SET );
>              }
>  
>              Fhandler->ar_full_buffers.push(tobeFilled);
> @@ -362,23 +407,29 @@
>          }
>          //B write
>  
> -        while(!Fhandler->b_full_buffers.empty())
> +        while(!Fhandler->write_full_buffers.empty())
>          {
>  
>  
>              pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -            type_buffElement* tobeWritten = Fhandler->b_full_buffers.front();
> -            Fhandler->b_full_buffers.pop();
> +            type_buffElement* tobeWritten = Fhandler->write_full_buffers.front();
> +            Fhandler->write_full_buffers.pop();
>              int size = Fhandler->p*Fhandler->b_blockSize;
>  
>              if(Fhandler->fakefiles)
>              {
> -                fseek ( fp_B , 0 , SEEK_SET );
> +                fseek ( fp_B , 0 , SEEK_SET );
> +                fseek ( fp_R , 0 , SEEK_SET );
> +                fseek ( fp_SD2 , 0 , SEEK_SET );
> +                fseek ( fp_P , 0 , SEEK_SET );
>              }
> -            fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
> +            fwrite (&(tobeWritten->buff[0]),sizeof(type_precision),size,fp_B);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*Fhandler->p]),sizeof(type_precision),Fhandler->b_blockSize,fp_R);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*(Fhandler->p+1)]),sizeof(type_precision),Fhandler->b_blockSize,fp_SD2);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*(Fhandler->p+2)]),sizeof(type_precision),size,fp_P);
>  
>  
> -            Fhandler->b_empty_buffers.push(tobeWritten);
> +            Fhandler->write_empty_buffers.push(tobeWritten);
>              //  cout << "\nStoring " << tobeWritten << endl;
>              pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
> @@ -411,12 +462,31 @@
>          pthread_cond_signal( &(Fhandler->condition_read ));
>          pthread_mutex_unlock(&(Fhandler->m_read));
>  
> -        if(Fhandler->reset_wait)
> -        {
> -            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -            //wait for main thread to reset everything
> -            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -        }
> +//        if(Fhandler->reset_wait)
> +//        {
> +//            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//            //wait for main thread to reset everything
> +//
> +//            pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +//            Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +//
> +//            if(Fhandler->Ar_currentReadBuff)
> +//            {
> +//                Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +//                Fhandler->Ar_currentReadBuff=0;
> +//            }
> +//            while(!Fhandler->ar_full_buffers.empty())
> +//            {
> +//                Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> +//                Fhandler->ar_full_buffers.pop();
> +//            }
> +//            pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +//
> +//            Fhandler->reset_wait = false;
> +//
> +//
> +//            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//        }
>  
>  
>      }
> @@ -425,7 +495,14 @@
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
>  
>      {
> -    type_buffElement* tmp;
> +    type_buffElement* tmp;
> +
> +    if(Fhandler->currentReadBuff)
> +    {
> +        Fhandler->full_buffers.push(Fhandler->currentReadBuff);
> +        Fhandler->currentReadBuff=0;
> +    }
> +
>      while(!Fhandler->full_buffers.empty())
>      {
>         tmp= Fhandler->full_buffers.front();
> @@ -438,8 +515,15 @@
>      {
>         tmp= Fhandler->empty_buffers.front();
>         Fhandler->empty_buffers.pop();
> -       delete []tmp->buff;
> -       delete tmp;
> +        delete []tmp->buff;
> +        delete tmp;
> +
> +    }
> +
> +    if(Fhandler->Ar_currentReadBuff)
> +    {
> +        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +        Fhandler->Ar_currentReadBuff=0;
>      }
>  
>      while(!Fhandler->ar_full_buffers.empty())
> @@ -458,18 +542,18 @@
>         delete tmp;
>      }
>  
> -    while(!Fhandler->b_full_buffers.empty())
> +    while(!Fhandler->write_full_buffers.empty())
>      {
> -       tmp= Fhandler->b_full_buffers.front();
> -       Fhandler->b_full_buffers.pop();
> +       tmp= Fhandler->write_full_buffers.front();
> +       Fhandler->write_full_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
>      }
>  
> -    while(!Fhandler->b_empty_buffers.empty())
> +    while(!Fhandler->write_empty_buffers.empty())
>      {
> -       tmp= Fhandler->b_empty_buffers.front();
> -       Fhandler->b_empty_buffers.pop();
> +       tmp= Fhandler->write_empty_buffers.front();
> +       Fhandler->write_empty_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
>      }
> @@ -482,6 +566,9 @@
>          fclose(fp_Y);
>          fclose(fp_Ar);
>          fclose(fp_B);
> +        fclose(fp_R);
> +        fclose(fp_SD2);
> +        fclose(fp_P);
>  
>          //cout << "\nexited io\n";
>  
> @@ -514,15 +601,16 @@
>  
>  
>      //!read new rdy buffer
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -        if(Fhandler->Ar_currentReadBuff)
> -        {
> -            Fhandler->ar_empty_buffers.push(Fhandler->Ar_currentReadBuff);
> -        }
> +    pthread_mutex_lock(&(Fhandler->m_buff_upd));
>  
> -        Fhandler->Ar_currentReadBuff = Fhandler->ar_full_buffers.front();
> -        Fhandler->ar_full_buffers.pop();
> +    if(Fhandler->Ar_currentReadBuff)
> +    {
> +        Fhandler->ar_empty_buffers.push(Fhandler->Ar_currentReadBuff);
> +    }
>  
> +    Fhandler->Ar_currentReadBuff = Fhandler->ar_full_buffers.front();
> +    Fhandler->ar_full_buffers.pop();
> +
>      //cout << "\nReading " << Fhandler->Ar_currentReadBuff << endl;
>      Fhandler->Ar = Fhandler->Ar_currentReadBuff->buff;
>      Ar_blockSize = Fhandler->Ar_currentReadBuff->size;
> @@ -580,24 +668,23 @@
>  
>      //!read new rdy buffer
>      pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -    //cout << " pre," << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
>  
> +
>          if(Fhandler->currentReadBuff)
>          {
> -            //memset(Fhandler->currentReadBuff->buff,0,y_blockSize);
>              Fhandler->empty_buffers.push(Fhandler->currentReadBuff);
>          }
>          Fhandler->currentReadBuff = Fhandler->full_buffers.front();
>          Fhandler->full_buffers.pop();
>  
> -    //cout << "\nReading " << Fhandler->currentReadBuff << endl;
> +
>      Fhandler->Yb = Fhandler->currentReadBuff->buff;
>      y_blockSize = Fhandler->currentReadBuff->size;
>  
>      (*Y) = Fhandler->Yb;
>  
> -     //cout << " post," << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
>  
> +
>      pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
>  
> @@ -611,50 +698,7 @@
>  
>  }
>  
> -void AIOwrapper::write_B(type_precision* B, int p, int blockSize)
> -{
> -
> -    while(Fhandler->b_empty_buffers.empty())
> -    {
> -        pthread_mutex_lock(&(Fhandler->m_more));
> -        pthread_cond_signal( &(Fhandler->condition_more ));
> -        pthread_mutex_unlock(&(Fhandler->m_more));
> -
> -        io_overhead = "b";
> -
> -        pthread_mutex_lock(&(Fhandler->m_read));
> -        pthread_cond_wait( &(Fhandler->condition_read), &(Fhandler->m_read ));
> -        pthread_mutex_unlock(&(Fhandler->m_read));
> -
> -    }
> -
> -
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -
> -
> -
> -        //cout << Fhandler->b_empty_buffers.size() << flush;
> -        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> -        Fhandler->b_empty_buffers.pop();
> -
> -
> -
> -    Fhandler->B = Fhandler->currentWriteBuff->buff;
> -    Fhandler->b_blockSize = blockSize;
> -    copy_vec(B,Fhandler->B,p*blockSize);
> -
> -    Fhandler->b_full_buffers.push(Fhandler->currentWriteBuff);
> -
> -
> -
> -    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> -
> -
> -    pthread_mutex_lock(&(Fhandler->m_more));
> -    pthread_cond_signal( &(Fhandler->condition_more ));
> -    pthread_mutex_unlock(&(Fhandler->m_more));
> -}
> -
> +
>  void AIOwrapper::prepare_Y(int y_blockSize, int n, int totalY)
>  {
>      //for fake files
> @@ -679,10 +723,6 @@
>          tmp = new type_buffElement();
>          tmp->buff = new type_precision[Fhandler->n*Fhandler->y_blockSize];
>          tmp->size = y_blockSize;
> -//        for( int i = 0; i < Fhandler->n*Fhandler->y_blockSize; i++)
> -//        {
> -//            (tmp->buff)[i] = 0;
> -//        }
>          Fhandler->empty_buffers.push(tmp);
>          Fhandler->Yb = tmp->buff;
>      }
> @@ -701,52 +741,98 @@
>  
>  
>  
> -}
> -
> -void AIOwrapper::prepare_B(int b_blockSize, int p)
> +}
> +
> +void AIOwrapper::getCurrentWriteBuffers(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P)
> +{
> +    B = &(Fhandler->currentWriteBuff->buff[0]);
> +    R = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*Fhandler->p]);
> +    SD2 = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*(Fhandler->p+1)]);
> +    P = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*(Fhandler->p+2)]);
> +}
> +
> +void AIOwrapper::write_OutFiles(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P,  int blockSize)
>  {
> -    //for fake files
>  
> +    while(Fhandler->write_empty_buffers.empty())
> +    {
> +        pthread_mutex_lock(&(Fhandler->m_more));
> +        pthread_cond_signal( &(Fhandler->condition_more ));
> +        pthread_mutex_unlock(&(Fhandler->m_more));
>  
> -    Fhandler->b_blockSize = b_blockSize;
> +        io_overhead = "W";
>  
> -    Fhandler->p=p;
> +        pthread_mutex_lock(&(Fhandler->m_read));
> +        pthread_cond_wait( &(Fhandler->condition_read), &(Fhandler->m_read ));
> +        pthread_mutex_unlock(&(Fhandler->m_read));
> +    }
>  
>  
> -    int buff_count = 4;
> +    pthread_mutex_lock(&(Fhandler->m_buff_upd));
>  
> -    Fhandler->currentWriteBuff = 0;
>  
> -    type_buffElement* tmp;
> +    Fhandler->write_full_buffers.push(Fhandler->currentWriteBuff);
> +    Fhandler->b_blockSize = blockSize;
> +
> +
> +    Fhandler->currentWriteBuff = Fhandler->write_empty_buffers.front();
> +    Fhandler->write_empty_buffers.pop();
> +
> +    B = &(Fhandler->currentWriteBuff->buff[0]);
> +    R = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*Fhandler->p]);
> +    SD2 = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*(Fhandler->p+1)]);
> +    P = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*(Fhandler->p+2)]);
>  
>  
> +    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +
> +
> +    pthread_mutex_lock(&(Fhandler->m_more));
> +    pthread_cond_signal( &(Fhandler->condition_more ));
> +    pthread_mutex_unlock(&(Fhandler->m_more));
> +}
> +
> +
> +
> +
> +
> +void AIOwrapper::prepare_OutFiles(int max_b_blockSize, int p)
> +{
> +
> +    Fhandler->max_b_blockSize = max_b_blockSize;
> +    Fhandler->p=p;
> +    int buff_count = 4;
> +
> +    type_buffElement* tmp;
> +
> +
>      for(int i = 0; i< buff_count  ; i++)
>      {
> -
>          tmp = new type_buffElement();
> -        tmp->buff = new type_precision[Fhandler->p*Fhandler->b_blockSize];
> -        tmp->size = b_blockSize;
> -//        for( int i = 0; i < Fhandler->n*Fhandler->b_blockSize; i++)
> -//        {
> -//            (tmp->buff)[i] = 0;
> -//        }
> -        Fhandler->b_empty_buffers.push(tmp);
> -
> -//        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> -//        Fhandler->b_empty_buffers.pop();
> +        tmp->buff = new type_precision[Fhandler->max_b_blockSize*(2*Fhandler->p+2)];
> +        tmp->size = max_b_blockSize;
> +        Fhandler->write_empty_buffers.push(tmp);
> +    }
> +    Fhandler->currentWriteBuff = Fhandler->write_empty_buffers.front();
> +    Fhandler->write_empty_buffers.pop();
>  
>  
> -    }
> -
>  }
>  
> +
> + void AIOwrapper::write_significantValues(int Y, int X_R, float R, float SD2, float P)
> + {
> +
> + }
> +
> +
>  void AIOwrapper::reset_Y()
>  {
>      //void *status;
>  
>      Fhandler->seed = 1337;
>  
> -    //cout << "ry" << flush;
> +    cout << "ry" << flush;
>  
>      Fhandler->reset_wait = true;
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
> @@ -789,38 +875,38 @@
>  
>      //cout << "ra" << flush;
>  
> -    Fhandler->reset_wait = true;
> -    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//    Fhandler->reset_wait = true;
> +//    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//
> +////    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +////    Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +////
> +////    if(Fhandler->Ar_currentReadBuff)
> +////    {
> +////        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +////        Fhandler->Ar_currentReadBuff=0;
> +////    }
> +////
> +////    while(!Fhandler->ar_full_buffers.empty())
> +////    {
> +////        Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> +//////        for( int i = 0; i < Fhandler->n*Fhandler->r*Fhandler->Ar_blockSize; i++)
> +//////        {
> +//////            ((Fhandler->ar_full_buffers.front())->buff)[i] = 0;
> +//////        }
> +////        Fhandler->ar_full_buffers.pop();
> +////    }
> +////    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +////
> +////    Fhandler->reset_wait = false;
> +//
> +//    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//
> +//    pthread_mutex_lock(&(Fhandler->m_more));
> +//    pthread_cond_signal( &(Fhandler->condition_more ));
> +//    pthread_mutex_unlock(&(Fhandler->m_more));
>  
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -    Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
>  
> -    if(Fhandler->Ar_currentReadBuff)
> -    {
> -        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> -        Fhandler->Ar_currentReadBuff=0;
> -    }
> -
> -    while(!Fhandler->ar_full_buffers.empty())
> -    {
> -        Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> -//        for( int i = 0; i < Fhandler->n*Fhandler->r*Fhandler->Ar_blockSize; i++)
> -//        {
> -//            ((Fhandler->ar_full_buffers.front())->buff)[i] = 0;
> -//        }
> -        Fhandler->ar_full_buffers.pop();
> -    }
> -    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> -
> -    Fhandler->reset_wait = false;
> -
> -    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -
> -    pthread_mutex_lock(&(Fhandler->m_more));
> -    pthread_cond_signal( &(Fhandler->condition_more ));
> -    pthread_mutex_unlock(&(Fhandler->m_more));
> -
> -
>  }
>  
>  void AIOwrapper::finalize_Y()
> @@ -838,7 +924,7 @@
>      Fhandler->Ar_Amount = totalR;
>      Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
>  
> -    int buff_count = min(3,(totalR+ desired_blockSize - 1)/desired_blockSize);
> +    int buff_count = 4;
>  
>      Fhandler->Ar_currentReadBuff = 0;
>      type_buffElement* tmp;
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -25,7 +25,7 @@
>      string fnameY;
>  
>  
> -    string fnameOutB;
> +    string fnameOutFiles;
>  
>  
>      list< pair<int,int> >* excl_List;
> @@ -46,8 +46,8 @@
>      queue<type_buffElement*> empty_buffers;
>      queue<type_buffElement*> full_buffers;
>  
> -    queue<type_buffElement*> b_empty_buffers;
> -    queue<type_buffElement*> b_full_buffers;
> +    queue<type_buffElement*> write_empty_buffers;
> +    queue<type_buffElement*> write_full_buffers;
>  
>      queue<type_buffElement*> ar_empty_buffers;
>      queue<type_buffElement*> ar_full_buffers;
> @@ -67,7 +67,8 @@
>      int y_blockSize;
>      int y_to_readSize;
>  
> -    int b_blockSize;
> +    int b_blockSize;
> +    int max_b_blockSize;
>  
>      bool not_done;
>      bool reset_wait;
> @@ -139,7 +140,11 @@
>          void reset_Y();
>          void reset_AR();
>  
> -        void write_B(type_precision* B, int p, int blockSize);
> +        void getCurrentWriteBuffers(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P);
> +
> +        void write_OutFiles(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P,  int blockSize);
> +
> +        void write_significantValues(int Y, int X_R, float R, float SD2, float P);
>  
>          string io_overhead;
>  
> @@ -161,8 +166,8 @@
>          void prepare_AL( int columns, int n);
>          void finalize_AL();
>  
> -        void prepare_B(int b_blockSize, int p);
> -        void finalize_B();
> +        void prepare_OutFiles(int max_b_blockSize, int p);
> +        void finalize_OutFiles();
>  
>  
>          static void* async_io(void *ptr );
> 
> Modified: pkg/OmicABELnoMM/src/Algorithm.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -621,7 +621,7 @@
>  
>                  get_ticks(start_tick2);
>  
> -                AIOfile.write_B(B, p, a_block_size);
> +                //AIOfile.write_B(B, p, a_block_size);
>  
>                  get_ticks(end_tick);
>                  out.acc_storeb += ticks2sec(end_tick,start_tick2);
> 
> Modified: pkg/OmicABELnoMM/src/Definitions.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/Definitions.h	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/Definitions.h	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -5,41 +5,110 @@
>      #define LINUX
>  #else
>      #define WINDOWS
> -#endif
> +#endif
>  
> +#include <unistd.h>
> +#include <limits.h>
> +#include <queue>
> +#include <iostream>
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <time.h>       /* time */
>  #include <cstring>
>  #include <math.h>
> -#include <omp.h>
> +#include <omp.h>
> +#include <pthread.h>
>  
> -
> -
> +
>  #ifdef WINDOWS
>      #include <windows.h>
> -    #include <cblas.h>
>  #else
> -    //#include "mpi.h"
> -    //#define cpu_freq 3.0
> -    #define cpu_freq 3.2
> -    #include "cblas.h"
> +
> +#endif
> +
> +//!For intel use propetary MKL, it will be preferred over others
> +#ifdef __INTEL_MKL__
> +    #pragma message("MKL will Probably NOT compile")
> +    #include "mkl.h"
> +    #include "cblas.h"
> +    #include <lapacke.h>
> +    #define blas_set_num_threads(n) mkl_set_num_threads(n)
> +    #define STORAGE_TYPE LAPACK_COL_MAJOR
> +#else
> +
> +    //!For AMD systems use the proper ACML library, preferred over openblas ON AMD
> +    #ifdef _acml_
> +        #pragma message("Compiled with AMD ACML")
> +        #define blas_set_num_threads(n) omp_set_num_threads(n)
> +
> +        #include <acml.h>
> +
> +        #define lapack_int int
> +
> +        #define CblasTrans 'T'
> +        #define CblasNoTrans 'N'
> +        #define CblasUpper 'U'
> +        #define CblasColMajor 1
> +
> +
> +        #define STORAGE_TYPE CblasColMajor
> +
> +
> +        #define cblas_snrm2 snrm2
> +        #define cblas_saxpy saxpy
> +
> +        #ifndef BLASdefs_H_INCLUDED
> +        #define BLASdefs_H_INCLUDED
> +
> +        inline  void cblas_sgemm(int storage, char transa, char transb, int m, int n, int k, float alpha, float *a, int lda, float *b, int ldb, float beta, float *c, int ldc)
> +        {
> +           sgemm(transa, transb, m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
> +        }
> +
> +        inline  lapack_int LAPACKE_sposv( int matrix_order, char uplo, lapack_int n,  lapack_int nrhs, float* a, lapack_int lda, float* b, lapack_int ldb )
> +        {
> +            int info;
> +            sposv( uplo, n,  nrhs, a, lda, b,  ldb, &info);
> +            return info;
> +        }
> +
> +        inline  void cblas_ssyrk(int Order, char uplo, char Trans,
> +		 int N, int K, float alpha, float *A, int lda,  float beta, float *C, int ldc)
> +        {
> +            ssyrk(uplo, Trans, N, K, alpha, A, lda, beta, C, ldc);
> +        }
> +
> +        inline  lapack_int LAPACKE_sgels( int matrix_order, char trans, lapack_int m, lapack_int n, lapack_int nrhs, float* a,  lapack_int lda, float* b, lapack_int ldb )
> +        {
> +            int info;
> +            sgels(trans, m, n, nrhs, a, lda, b, ldb,&info);
> +            return info;
> +        }
> +
> +        #endif
> +
> +
> +
> +
> +
> +    #else
> +
> +        //!IF MKL is not present on INTEL, use openblas
> +        #ifdef _openblas_
> +            #pragma message("Compiled with OPENBLAS")
> +            #define STORAGE_TYPE LAPACK_COL_MAJOR
> +            #include "cblas.h"
> +            #include <lapacke.h>
> +            extern "C" void openblas_set_num_threads(int num_threads);
> +            #define blas_set_num_threads(n) openblas_set_num_threads(n)
> +        #endif
> +
> +    #endif
> +
>  #endif
> -#ifdef __INTEL_MKL__
> -    #include "mkl.h"
> -    #define blas_set_num_threads(n) mkl_set_num_threads(n)
> -#else
> -    extern "C" void openblas_set_num_threads(int num_threads);
> -    #define blas_set_num_threads(n) openblas_set_num_threads(n)
> -#endif
>  
> -#include <unistd.h>
> -#include <pthread.h>
> -#include <limits.h>
> -#include <queue>
> -#include <iostream>
> -#include <lapacke.h>
>  
> +
>  //!SETTINGS
>  
>  #define EXTENDEDTEST 0
> @@ -47,7 +116,7 @@
>  
>  #define OUTPUT 0
>  
> -#define STORAGE_TYPE LAPACK_COL_MAJOR
> +
>  #define type_precision float
>  
>  #define MIN(a,b) (((a)<(b))?(a):(b))
> @@ -56,86 +125,8 @@
>  #define _10MB 10*_1MB
>  #define _1GB 1024*1024*1024
>  
> -//!for CPU speed!
>  
> -//#ifdef WIN32
> -//#define WIN32_LEAN_AND_MEAN
> -//#include <windows.h>
> -//typedef unsigned __int64 usCount;
> -//static usCount GetUsCount()
> -//{
> -//    static LARGE_INTEGER ticksPerSec;
> -//    static double scalefactor;
> -//    LARGE_INTEGER val;
> -//    if (!scalefactor)
> -//    {
> -//        if (QueryPerformanceFrequency(&ticksPerSec))
> -//            scalefactor=ticksPerSec.QuadPart/1000000000000.0;
> -//        else
> -//            scalefactor=1;
> -//    }
> -//    if (!QueryPerformanceCounter(&val))
> -//        return (usCount) GetTickCount() * 1000000000;
> -//    return (usCount) (val.QuadPart/scalefactor);
> -//}
> -//#else
> -//#include <sys/time.h>
> -//#include <time.h>
> -//#include <sched.h>
> -//typedef unsigned long long usCount;
> -//static usCount GetUsCount()
> -//{
> -//#ifdef CLOCK_MONOTONIC
> -//    struct timespec ts;
> -//    clock_gettime(CLOCK_MONOTONIC, &ts);
> -//    return ((usCount) ts.tv_sec*1000000000000LL)+ts.tv_nsec*1000LL;
> -//#else
> -//    struct timeval tv;
> -//    gettimeofday(&tv, 0);
> -//    return ((usCount) tv.tv_sec*1000000000000LL)+tv.tv_usec*1000000LL;
> -//#endif
> -//}
> -//#endif
> -//static usCount usCountOverhead;
> -//#ifdef __GNUC__
> -//#include "x86intrin.h"
> -//#define __rdtsc() __builtin_ia32_rdtsc()
> -//#endif
>  
> -//static usCount GetClockSpeed()
> -//{
> -//    int n;
> -//    usCount start, end, start_tsc, end_tsc;
> -//    if (!usCountOverhead)
> -//    {
> -//        usCount foo = 0;
> -//        start=GetUsCount();
> -//        for (n = 0; n < 1000000; n++)
> -//        {
> -//            foo += GetUsCount();
> -//        }
> -//        end = GetUsCount();
> -//        usCountOverhead = (end - start)/n;
> -//    }
> -//
> -//    start = GetUsCount();
> -//    start_tsc = __rdtsc();
> -//    for (n = 0; n <1000; n++)
> -//    {
> -//#ifdef WIN32
> -//        Sleep(0);
> -//#else
> -//        sched_yield();
> -//#endif
> -//    }
> -//
> -//    end_tsc = __rdtsc();
> -//    end = GetUsCount();
> -//    return(usCount)((1000000000000.0 * (end_tsc - start_tsc)) /
> -//                    (end - start - usCountOverhead));
> -//}
> -
> -
>  using namespace std;
>  
>  
> @@ -149,7 +140,9 @@
>      int p;
>      int tb;
>      int mb;
> -    int id;
> +    int id;
> +
> +    float sig_threshold;
>  
>      int threads;
>  
> @@ -158,7 +151,7 @@
>      string fnameAL;
>      string fnameAR;
>      string fnameY;
> -    string fnameOutB;
> +    string fnameOutFiles;
>      string fname_excludelist;
>  
> [TRUNCATED]
> 
> To get the complete diff run:
>     svnlook diff /svnroot/genabel -r 1768
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/4020affa/attachment-0001.sig>

From lennart at karssen.org  Fri Jul 25 15:47:12 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 15:47:12 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1768 - in pkg/OmicABELnoMM: .
	doc src tests
In-Reply-To: <20140725104804.2C8FF186938@r-forge.r-project.org>
References: <20140725104804.2C8FF186938@r-forge.r-project.org>
Message-ID: <53D25FE0.3070408@karssen.org>

Hi Alvaro,

Please find a few other comments below.

On 25-07-14 12:48, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-25 12:48:03 +0200 (Fri, 25 Jul 2014)
> New Revision: 1768
> 
> Modified:
>    pkg/OmicABELnoMM/configure.ac
>    pkg/OmicABELnoMM/doc/howtocompile.txt
>    pkg/OmicABELnoMM/src/AIOwrapper.cpp
>    pkg/OmicABELnoMM/src/AIOwrapper.h
>    pkg/OmicABELnoMM/src/Algorithm.cpp
>    pkg/OmicABELnoMM/src/Definitions.h
>    pkg/OmicABELnoMM/src/Utility.cpp
>    pkg/OmicABELnoMM/src/main.cpp
>    pkg/OmicABELnoMM/test-driver
>    pkg/OmicABELnoMM/tests/test.cpp
> Log:
> Added AMD ACML 6 support for heteregenous systems (CPU + GPU). ACML fixes unusable performance issues under older Opteron (non AVX) systems. Several Bug fixes of IO. Added incomplete functionalities.
> 
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/configure.ac	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -3,8 +3,9 @@
>  
>  AC_PREREQ([2.67])
>  AC_INIT([OmicABELnoMM], [0.1.0], [genabel-devel at r-forge.wu-wien.ac.at])
> -AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> -AM_SILENT_RULES([yes])
> +AM_INIT_AUTOMAKE([])
> +#AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> +#AM_SILENT_RULES([yes])
>  AC_CONFIG_SRCDIR([src/Utility.h])
>  AC_CONFIG_HEADERS([src/config.h])
>  
> @@ -17,12 +18,12 @@
>  # Set some default compile flags
>  if test -z "$CXXFLAGS"; then
>     # User did not set CXXFLAGS, so we can put in our own defaults
> -    CXXFLAGS="-O3"
> +    CXXFLAGS=""

Is there a special reason why you remove optimisations here and set -O0
below? You seem to later enable O3 with AM_CXXFLAGS.

>  fi
>  if test -z "$CPPFLAGS"; then
>     # User did not set CPPFLAGS, so we can put in our own defaults
> -    CPPFLAGS="-Wall -g -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> -    #CPPFLAGS="-Wall"
> +    #CPPFLAGS="-Wall -pedantic -Wunused-result -Wmaybe-uninitialized -Wformat"
> +    CPPFLAGS="-O0"
>  fi
>  # If CXXFLAGS/CPPFLAGS are already set AC_PROG_CXX will not overwrite them
>  # with its own defaults
> @@ -31,33 +32,49 @@
>  AC_PROG_CC
>  AC_PROG_CXX
>  
> +# Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
> +AC_OPENMP
> +AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>  
> +AM_CXXFLAGS="-static -O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
> +
>  # Checks for libraries.
>  # pthread library
>  AC_SEARCH_LIBS([pthread_mutex_init], [pthread], [], [
> -   AC_MSG_ERROR([Unable to find the pthread_mutex_init() function])
> +   AC_MSG_ERROR([Make sure pthread is available on the system])
>  ])
> -# Openblas
> -AC_SEARCH_LIBS([cblas_sgemm], [openblas], [], [
> -   AC_MSG_ERROR([Unable to find the openblas library])
> -])
> -# Lapack
> -AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> -   AC_MSG_ERROR([Unable to find a Lapack library])
> -])
>  
> -#Boost
> +if test -z "$LDFLAGS"; then
> +	LDFLAGS="-L./libs/lib/ -L../libs/lib/"
> +fi
>  
> +found_blas=0
>  
> +AC_SEARCH_LIBS([__iso_c_binding_c_f_pointer_l4],[gfortran])
>  
> -# Check for openMP. If found the OPENMP_CXXFLAGS is set automatically
> -AC_OPENMP
> -AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
> +# ACML
> +AC_SEARCH_LIBS(dgemm, acml_mp,[found_blas=1 AM_CXXFLAGS="-D_acml_ $AM_CXXFLAGS"], [
> +   AC_MSG_NOTICE([NOT using AMD the ACML  library],[-lgfortran])
> +])
>  
>  
>  
> +if test "$found_blas" -eq 0
> +	then
>  
> +# Openblas
> +AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> +   AC_MSG_ERROR([OpenBLAS  library NOT found])
> +])
> + #Lapack
> +AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> +   AC_MSG_ERROR([Unable to find a Lapack library])
> +])
> +AC_MSG_NOTICE([Using OpenBLAS  library])
> +fi
>  

Could you add a bit of indentation in the lines above. It took me a
while to see that the if was closed by the fi a few lines below it.



Thanks,

Lennart.

> +
> +
>  # Checks for header files.
>  AC_CHECK_HEADERS([limits.h stdlib.h string.h sys/time.h unistd.h])
>  
> @@ -73,6 +90,7 @@
>  # Files to be generated by autotools
>  AC_CONFIG_FILES([
>          Makefile
> +
>  ])
>  
>  AC_OUTPUT
> 
> Modified: pkg/OmicABELnoMM/doc/howtocompile.txt
> ===================================================================
> --- pkg/OmicABELnoMM/doc/howtocompile.txt	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/doc/howtocompile.txt	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -1,4 +1,7 @@
> -
> +
> +-------------------------Install BLAS----------------------------
> +#STEP 1:
> +#a) Install OPENBLAS:
>  mkdir GWAS_PROJECT
>  cd GWAS_PROJECT
>  
> @@ -10,21 +13,52 @@
>  #make sure g++ its 4.8 or above!on rwth cluster module load gcc/4.8, choose 32 or 64 also
>  make all HOSTCC=g++ FC=gfortran USE_OPENMP=1
>  
> -sudo make install PREFIX="/usr"
> +make install PREFIX="path_to_/OmicABELnoMM/libs/"
>  
> -sudo ldconfig
> +ldconfig
>  
> -cd ..
> -#BLAS END
>  
> +cd ..
> +
> +-------------------------AMD ACML Alternative Version-------------------------
> +#STEP 1:
> +# b)
> +mkdir OmicABELnoMM/libs/
> +
> +cd OmicABELnoMM/libs/
> +
> +You can use AMD ACML. Download from:
> +http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/acml-downloads-resources/
> +
> + and copy the supplied binary 
> + libraries to "OmicABELnoMM/libs/"
> +
> +IF both libraries are present (Openblas + ACML), the system will use ACML.
> +
> +cd ../../
> +
> +---------------------------------#BLAS END----------------------------------------------------
> +
> +#STEP 2:
> +
> +
> +ON the folder GWAS_PROJECT
> +
>  svn checkout svn+ssh://developername at svn.r-forge.r-project.org/svnroot/genabel/OmicABELnoMM
>  
>  cd OmicABELnoMM
> +
> +autoreconf -fi
>  
> -./configure LDFLAGS="-L/usr/lib/"
> +./configure
>  
> -make
> +make
>  
> +make check
> +
> +#DONE
> +
> +------------------------------Example--------------------------------
>  #test it
>  ./omicabelnomm -c examples/XL --geno examples/XR -p examples/Y -o examples/B -n 2 -t 2
>  
> @@ -42,12 +76,19 @@
>  
>  
>  
> --------------------------Alternative Version-------------------------
> +-------------------------Alternative Version of BLAS-------------------------
>  
> -Install all required libraries for your system: 
> +#Make sure autoconf is installed:
>  
> --Iinux ubuntu:
> +sudo apt-get install autoconf
> +autoreconf -fi
> +autoconf
>  
> +#Install all required libraries for your system, 
> +#but they will not work, 99% of the time, due to not having openmp support: 
> +
> +#Iinux ubuntu:
> +
>  sudo apt-get install libopenblas-dev
>  sudo apt-get install libopenblas-base
>  sudo apt-get install liblapack3gf
> @@ -56,11 +97,7 @@
>  sudo apt-get install liblapacke
>  sudo apt-get install liblapacke-dev
>  
> --Make sure autoconf is installed:
>  
> -sudo apt-get install autoconf
> -autoreconf -fi
> -autoconf
>  
>  
>  
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.cpp	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -40,7 +40,7 @@
>          Fhandler->fnameAL = params.fnameAL;
>          Fhandler->fnameAR = params.fnameAR;
>          Fhandler->fnameY = params.fnameY;
> -        Fhandler->fnameOutB = params.fnameOutB;
> +        Fhandler->fnameOutFiles = params.fnameOutFiles;
>  
>  
>          Yfvi  = load_databel_fvi( (Fhandler->fnameY+".fvi").c_str() );
> @@ -91,7 +91,7 @@
>  
>      prepare_AL(params.l,params.n);
>      prepare_AR(  params.mb,  params.n,  params.m,  params.r);
> -    prepare_B(params.tb, params.l+params.r);
> +    prepare_OutFiles(params.mb, params.l+params.r);
>      prepare_Y(params.tb, params.n, params.t);
>  
>  
> @@ -118,7 +118,7 @@
>      finalize_Y();
>      finalize_AR();
>      finalize_AL();
> -    finalize_B();
> +    finalize_OutFiles();
>  
>      pthread_attr_destroy(&(Fhandler->attr));
>  
> @@ -135,7 +135,7 @@
>  
>  
>  
> -void AIOwrapper::finalize_B()
> +void AIOwrapper::finalize_OutFiles()
>  {
>  
>  }
> @@ -149,7 +149,10 @@
>  
>      struct timespec timeToWait;
>      FILE*  fp_Y;
> -    FILE*  fp_B;
> +    FILE*  fp_B;
> +    FILE*  fp_R;
> +    FILE*  fp_SD2;
> +    FILE*  fp_P;
>      FILE*  fp_Ar;
>      if(!Fhandler->fakefiles)
>      {
> @@ -167,11 +170,29 @@
>              exit(1);
>          }
>  
> -        fp_B = fopen((Fhandler->fnameOutB+".fvd").c_str(), "w+b");
> +        fp_B = fopen((Fhandler->fnameOutFiles+"_B.fvd").c_str(), "w+b");
>          if(fp_B == 0)
>          {
> -            cout << "Error Opening File B " << Fhandler->fnameOutB << endl;
> +            cout << "Error Opening File B " << Fhandler->fnameOutFiles << "_B" << endl;
>              exit(1);
> +        }
> +        fp_R = fopen((Fhandler->fnameOutFiles+"_R.fvd").c_str(), "w+b");
> +        if(fp_R == 0)
> +        {
> +            cout << "Error Opening File R " << Fhandler->fnameOutFiles << "_R" << endl;
> +            exit(1);
> +        }
> +        fp_SD2 = fopen((Fhandler->fnameOutFiles+"_SD2.fvd").c_str(), "w+b");
> +        if(fp_SD2 == 0)
> +        {
> +            cout << "Error Opening File SD2 " << Fhandler->fnameOutFiles << "_SD2" << endl;
> +            exit(1);
> +        }
> +        fp_P = fopen((Fhandler->fnameOutFiles+"_P.fvd").c_str(), "w+b");
> +        if(fp_P == 0)
> +        {
> +            cout << "Error Opening File P " << Fhandler->fnameOutFiles << "_P" << endl;
> +            exit(1);
>          }
>      }
>      else
> @@ -205,6 +226,24 @@
>          {
>              cout << "Error setting up temp File B " << endl;
>              exit(1);
> +        }
> +        fp_R = fopen("tempR.bin", "w+b");
> +        if(fp_R == 0)
> +        {
> +            cout << "Error setting up temp File R " << endl;
> +            exit(1);
> +        }
> +        fp_SD2 = fopen("tempSD2.bin", "w+b");
> +        if(fp_SD2 == 0)
> +        {
> +            cout << "Error setting up temp File SD2 " << endl;
> +            exit(1);
> +        }
> +        fp_P = fopen("tempP.bin", "w+b");
> +        if(fp_P == 0)
> +        {
> +            cout << "Error setting up temp File P " << endl;
> +            exit(1);
>          }
>          //cout << "\nEnd preping files\n" << flush;
>  
> @@ -229,14 +268,16 @@
>  
>              Fhandler->y_to_readSize -= tmp_y_blockSize;
>              size_buff = Fhandler->n * tmp_y_blockSize;
> -            //cout << Fhandler->y_to_readSize << endl;
> +
>  
> +
>              pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -            //cout << " pre;" << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
> +
> +
>              type_buffElement* tobeFilled = Fhandler->empty_buffers.front();
>              Fhandler->empty_buffers.pop();
> -            //pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
> +
>              tobeFilled->size = tmp_y_blockSize;
>  
>              if(Fhandler->fakefiles)
> @@ -273,8 +314,8 @@
>  
>                      }
>                  }
> -//                size_t result = fread (tobeFilled->buff,sizeof(type_precision),size_buff,fp_Y);
> -//                result++;
> +
> +
>                  if(Fhandler->y_to_readSize <= 0)
>                  {
>                      fseek ( fp_Y , 0 , SEEK_SET );
> @@ -282,10 +323,9 @@
>              }
>  
>  
> -            //pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +
>              Fhandler->full_buffers.push(tobeFilled);
> -            //  cout << "\nStoring " << tobeFilled << endl;
> -            //cout << " post;" << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
> +
>              pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
>              pthread_mutex_lock(&(Fhandler->m_read));
> @@ -294,8 +334,12 @@
>  
>          }
>  
> -        while(!Fhandler->ar_empty_buffers.empty() && Fhandler->Ar_to_readSize)
> -        {
> +        while(!Fhandler->ar_empty_buffers.empty() && Fhandler->Ar_to_readSize )
> +        {
> +
> +
> +
> +
>              tmp_ar_blockSize = Fhandler->Ar_blockSize;
>              if(Fhandler->Ar_to_readSize < Fhandler->Ar_blockSize)
>                  tmp_ar_blockSize = Fhandler->Ar_to_readSize;
> @@ -343,12 +387,13 @@
>                      }
>                  }
>  
> -//                size_t result = fread(tobeFilled->buff,sizeof(type_precision),size_buff,fp_Ar);
> -//                result++;
> -                if (Fhandler->Ar_to_readSize <= 0)
> -                {
> -                    fseek ( fp_Ar , 0 , SEEK_SET );
> -                }
> +
> +            }
> +
> +            if(Fhandler->Ar_to_readSize <= 0)
> +            {
> +                Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +                fseek ( fp_Ar , 0 , SEEK_SET );
>              }
>  
>              Fhandler->ar_full_buffers.push(tobeFilled);
> @@ -362,23 +407,29 @@
>          }
>          //B write
>  
> -        while(!Fhandler->b_full_buffers.empty())
> +        while(!Fhandler->write_full_buffers.empty())
>          {
>  
>  
>              pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -            type_buffElement* tobeWritten = Fhandler->b_full_buffers.front();
> -            Fhandler->b_full_buffers.pop();
> +            type_buffElement* tobeWritten = Fhandler->write_full_buffers.front();
> +            Fhandler->write_full_buffers.pop();
>              int size = Fhandler->p*Fhandler->b_blockSize;
>  
>              if(Fhandler->fakefiles)
>              {
> -                fseek ( fp_B , 0 , SEEK_SET );
> +                fseek ( fp_B , 0 , SEEK_SET );
> +                fseek ( fp_R , 0 , SEEK_SET );
> +                fseek ( fp_SD2 , 0 , SEEK_SET );
> +                fseek ( fp_P , 0 , SEEK_SET );
>              }
> -            fwrite (tobeWritten->buff,sizeof(type_precision),size,fp_B);
> +            fwrite (&(tobeWritten->buff[0]),sizeof(type_precision),size,fp_B);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*Fhandler->p]),sizeof(type_precision),Fhandler->b_blockSize,fp_R);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*(Fhandler->p+1)]),sizeof(type_precision),Fhandler->b_blockSize,fp_SD2);
> +            fwrite (&(tobeWritten->buff[Fhandler->max_b_blockSize*(Fhandler->p+2)]),sizeof(type_precision),size,fp_P);
>  
>  
> -            Fhandler->b_empty_buffers.push(tobeWritten);
> +            Fhandler->write_empty_buffers.push(tobeWritten);
>              //  cout << "\nStoring " << tobeWritten << endl;
>              pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
> @@ -411,12 +462,31 @@
>          pthread_cond_signal( &(Fhandler->condition_read ));
>          pthread_mutex_unlock(&(Fhandler->m_read));
>  
> -        if(Fhandler->reset_wait)
> -        {
> -            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -            //wait for main thread to reset everything
> -            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -        }
> +//        if(Fhandler->reset_wait)
> +//        {
> +//            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//            //wait for main thread to reset everything
> +//
> +//            pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +//            Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +//
> +//            if(Fhandler->Ar_currentReadBuff)
> +//            {
> +//                Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +//                Fhandler->Ar_currentReadBuff=0;
> +//            }
> +//            while(!Fhandler->ar_full_buffers.empty())
> +//            {
> +//                Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> +//                Fhandler->ar_full_buffers.pop();
> +//            }
> +//            pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +//
> +//            Fhandler->reset_wait = false;
> +//
> +//
> +//            pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//        }
>  
>  
>      }
> @@ -425,7 +495,14 @@
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
>  
>      {
> -    type_buffElement* tmp;
> +    type_buffElement* tmp;
> +
> +    if(Fhandler->currentReadBuff)
> +    {
> +        Fhandler->full_buffers.push(Fhandler->currentReadBuff);
> +        Fhandler->currentReadBuff=0;
> +    }
> +
>      while(!Fhandler->full_buffers.empty())
>      {
>         tmp= Fhandler->full_buffers.front();
> @@ -438,8 +515,15 @@
>      {
>         tmp= Fhandler->empty_buffers.front();
>         Fhandler->empty_buffers.pop();
> -       delete []tmp->buff;
> -       delete tmp;
> +        delete []tmp->buff;
> +        delete tmp;
> +
> +    }
> +
> +    if(Fhandler->Ar_currentReadBuff)
> +    {
> +        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +        Fhandler->Ar_currentReadBuff=0;
>      }
>  
>      while(!Fhandler->ar_full_buffers.empty())
> @@ -458,18 +542,18 @@
>         delete tmp;
>      }
>  
> -    while(!Fhandler->b_full_buffers.empty())
> +    while(!Fhandler->write_full_buffers.empty())
>      {
> -       tmp= Fhandler->b_full_buffers.front();
> -       Fhandler->b_full_buffers.pop();
> +       tmp= Fhandler->write_full_buffers.front();
> +       Fhandler->write_full_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
>      }
>  
> -    while(!Fhandler->b_empty_buffers.empty())
> +    while(!Fhandler->write_empty_buffers.empty())
>      {
> -       tmp= Fhandler->b_empty_buffers.front();
> -       Fhandler->b_empty_buffers.pop();
> +       tmp= Fhandler->write_empty_buffers.front();
> +       Fhandler->write_empty_buffers.pop();
>         delete []tmp->buff;
>         delete tmp;
>      }
> @@ -482,6 +566,9 @@
>          fclose(fp_Y);
>          fclose(fp_Ar);
>          fclose(fp_B);
> +        fclose(fp_R);
> +        fclose(fp_SD2);
> +        fclose(fp_P);
>  
>          //cout << "\nexited io\n";
>  
> @@ -514,15 +601,16 @@
>  
>  
>      //!read new rdy buffer
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -        if(Fhandler->Ar_currentReadBuff)
> -        {
> -            Fhandler->ar_empty_buffers.push(Fhandler->Ar_currentReadBuff);
> -        }
> +    pthread_mutex_lock(&(Fhandler->m_buff_upd));
>  
> -        Fhandler->Ar_currentReadBuff = Fhandler->ar_full_buffers.front();
> -        Fhandler->ar_full_buffers.pop();
> +    if(Fhandler->Ar_currentReadBuff)
> +    {
> +        Fhandler->ar_empty_buffers.push(Fhandler->Ar_currentReadBuff);
> +    }
>  
> +    Fhandler->Ar_currentReadBuff = Fhandler->ar_full_buffers.front();
> +    Fhandler->ar_full_buffers.pop();
> +
>      //cout << "\nReading " << Fhandler->Ar_currentReadBuff << endl;
>      Fhandler->Ar = Fhandler->Ar_currentReadBuff->buff;
>      Ar_blockSize = Fhandler->Ar_currentReadBuff->size;
> @@ -580,24 +668,23 @@
>  
>      //!read new rdy buffer
>      pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -    //cout << " pre," << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
>  
> +
>          if(Fhandler->currentReadBuff)
>          {
> -            //memset(Fhandler->currentReadBuff->buff,0,y_blockSize);
>              Fhandler->empty_buffers.push(Fhandler->currentReadBuff);
>          }
>          Fhandler->currentReadBuff = Fhandler->full_buffers.front();
>          Fhandler->full_buffers.pop();
>  
> -    //cout << "\nReading " << Fhandler->currentReadBuff << endl;
> +
>      Fhandler->Yb = Fhandler->currentReadBuff->buff;
>      y_blockSize = Fhandler->currentReadBuff->size;
>  
>      (*Y) = Fhandler->Yb;
>  
> -     //cout << " post," << Fhandler->full_buffers.size() << ";" << Fhandler->empty_buffers.size() << endl;
>  
> +
>      pthread_mutex_unlock(&(Fhandler->m_buff_upd));
>  
>  
> @@ -611,50 +698,7 @@
>  
>  }
>  
> -void AIOwrapper::write_B(type_precision* B, int p, int blockSize)
> -{
> -
> -    while(Fhandler->b_empty_buffers.empty())
> -    {
> -        pthread_mutex_lock(&(Fhandler->m_more));
> -        pthread_cond_signal( &(Fhandler->condition_more ));
> -        pthread_mutex_unlock(&(Fhandler->m_more));
> -
> -        io_overhead = "b";
> -
> -        pthread_mutex_lock(&(Fhandler->m_read));
> -        pthread_cond_wait( &(Fhandler->condition_read), &(Fhandler->m_read ));
> -        pthread_mutex_unlock(&(Fhandler->m_read));
> -
> -    }
> -
> -
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -
> -
> -
> -        //cout << Fhandler->b_empty_buffers.size() << flush;
> -        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> -        Fhandler->b_empty_buffers.pop();
> -
> -
> -
> -    Fhandler->B = Fhandler->currentWriteBuff->buff;
> -    Fhandler->b_blockSize = blockSize;
> -    copy_vec(B,Fhandler->B,p*blockSize);
> -
> -    Fhandler->b_full_buffers.push(Fhandler->currentWriteBuff);
> -
> -
> -
> -    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> -
> -
> -    pthread_mutex_lock(&(Fhandler->m_more));
> -    pthread_cond_signal( &(Fhandler->condition_more ));
> -    pthread_mutex_unlock(&(Fhandler->m_more));
> -}
> -
> +
>  void AIOwrapper::prepare_Y(int y_blockSize, int n, int totalY)
>  {
>      //for fake files
> @@ -679,10 +723,6 @@
>          tmp = new type_buffElement();
>          tmp->buff = new type_precision[Fhandler->n*Fhandler->y_blockSize];
>          tmp->size = y_blockSize;
> -//        for( int i = 0; i < Fhandler->n*Fhandler->y_blockSize; i++)
> -//        {
> -//            (tmp->buff)[i] = 0;
> -//        }
>          Fhandler->empty_buffers.push(tmp);
>          Fhandler->Yb = tmp->buff;
>      }
> @@ -701,52 +741,98 @@
>  
>  
>  
> -}
> -
> -void AIOwrapper::prepare_B(int b_blockSize, int p)
> +}
> +
> +void AIOwrapper::getCurrentWriteBuffers(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P)
> +{
> +    B = &(Fhandler->currentWriteBuff->buff[0]);
> +    R = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*Fhandler->p]);
> +    SD2 = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*(Fhandler->p+1)]);
> +    P = &(Fhandler->currentWriteBuff->buff[Fhandler->max_b_blockSize*(Fhandler->p+2)]);
> +}
> +
> +void AIOwrapper::write_OutFiles(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P,  int blockSize)
>  {
> -    //for fake files
>  
> +    while(Fhandler->write_empty_buffers.empty())
> +    {
> +        pthread_mutex_lock(&(Fhandler->m_more));
> +        pthread_cond_signal( &(Fhandler->condition_more ));
> +        pthread_mutex_unlock(&(Fhandler->m_more));
>  
> -    Fhandler->b_blockSize = b_blockSize;
> +        io_overhead = "W";
>  
> -    Fhandler->p=p;
> +        pthread_mutex_lock(&(Fhandler->m_read));
> +        pthread_cond_wait( &(Fhandler->condition_read), &(Fhandler->m_read ));
> +        pthread_mutex_unlock(&(Fhandler->m_read));
> +    }
>  
>  
> -    int buff_count = 4;
> +    pthread_mutex_lock(&(Fhandler->m_buff_upd));
>  
> -    Fhandler->currentWriteBuff = 0;
>  
> -    type_buffElement* tmp;
> +    Fhandler->write_full_buffers.push(Fhandler->currentWriteBuff);
> +    Fhandler->b_blockSize = blockSize;
> +
> +
> +    Fhandler->currentWriteBuff = Fhandler->write_empty_buffers.front();
> +    Fhandler->write_empty_buffers.pop();
> +
> +    B = &(Fhandler->currentWriteBuff->buff[0]);
> +    R = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*Fhandler->p]);
> +    SD2 = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*(Fhandler->p+1)]);
> +    P = &(Fhandler->currentWriteBuff->buff[Fhandler->b_blockSize*(Fhandler->p+2)]);
>  
>  
> +    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +
> +
> +    pthread_mutex_lock(&(Fhandler->m_more));
> +    pthread_cond_signal( &(Fhandler->condition_more ));
> +    pthread_mutex_unlock(&(Fhandler->m_more));
> +}
> +
> +
> +
> +
> +
> +void AIOwrapper::prepare_OutFiles(int max_b_blockSize, int p)
> +{
> +
> +    Fhandler->max_b_blockSize = max_b_blockSize;
> +    Fhandler->p=p;
> +    int buff_count = 4;
> +
> +    type_buffElement* tmp;
> +
> +
>      for(int i = 0; i< buff_count  ; i++)
>      {
> -
>          tmp = new type_buffElement();
> -        tmp->buff = new type_precision[Fhandler->p*Fhandler->b_blockSize];
> -        tmp->size = b_blockSize;
> -//        for( int i = 0; i < Fhandler->n*Fhandler->b_blockSize; i++)
> -//        {
> -//            (tmp->buff)[i] = 0;
> -//        }
> -        Fhandler->b_empty_buffers.push(tmp);
> -
> -//        Fhandler->currentWriteBuff = Fhandler->b_empty_buffers.front();
> -//        Fhandler->b_empty_buffers.pop();
> +        tmp->buff = new type_precision[Fhandler->max_b_blockSize*(2*Fhandler->p+2)];
> +        tmp->size = max_b_blockSize;
> +        Fhandler->write_empty_buffers.push(tmp);
> +    }
> +    Fhandler->currentWriteBuff = Fhandler->write_empty_buffers.front();
> +    Fhandler->write_empty_buffers.pop();
>  
>  
> -    }
> -
>  }
>  
> +
> + void AIOwrapper::write_significantValues(int Y, int X_R, float R, float SD2, float P)
> + {
> +
> + }
> +
> +
>  void AIOwrapper::reset_Y()
>  {
>      //void *status;
>  
>      Fhandler->seed = 1337;
>  
> -    //cout << "ry" << flush;
> +    cout << "ry" << flush;
>  
>      Fhandler->reset_wait = true;
>      pthread_barrier_wait(&(Fhandler->finalize_barrier));
> @@ -789,38 +875,38 @@
>  
>      //cout << "ra" << flush;
>  
> -    Fhandler->reset_wait = true;
> -    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//    Fhandler->reset_wait = true;
> +//    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//
> +////    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> +////    Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
> +////
> +////    if(Fhandler->Ar_currentReadBuff)
> +////    {
> +////        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> +////        Fhandler->Ar_currentReadBuff=0;
> +////    }
> +////
> +////    while(!Fhandler->ar_full_buffers.empty())
> +////    {
> +////        Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> +//////        for( int i = 0; i < Fhandler->n*Fhandler->r*Fhandler->Ar_blockSize; i++)
> +//////        {
> +//////            ((Fhandler->ar_full_buffers.front())->buff)[i] = 0;
> +//////        }
> +////        Fhandler->ar_full_buffers.pop();
> +////    }
> +////    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> +////
> +////    Fhandler->reset_wait = false;
> +//
> +//    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> +//
> +//    pthread_mutex_lock(&(Fhandler->m_more));
> +//    pthread_cond_signal( &(Fhandler->condition_more ));
> +//    pthread_mutex_unlock(&(Fhandler->m_more));
>  
> -    pthread_mutex_lock(&(Fhandler->m_buff_upd));
> -    Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
>  
> -    if(Fhandler->Ar_currentReadBuff)
> -    {
> -        Fhandler->ar_full_buffers.push(Fhandler->Ar_currentReadBuff);
> -        Fhandler->Ar_currentReadBuff=0;
> -    }
> -
> -    while(!Fhandler->ar_full_buffers.empty())
> -    {
> -        Fhandler->ar_empty_buffers.push(Fhandler->ar_full_buffers.front());
> -//        for( int i = 0; i < Fhandler->n*Fhandler->r*Fhandler->Ar_blockSize; i++)
> -//        {
> -//            ((Fhandler->ar_full_buffers.front())->buff)[i] = 0;
> -//        }
> -        Fhandler->ar_full_buffers.pop();
> -    }
> -    pthread_mutex_unlock(&(Fhandler->m_buff_upd));
> -
> -    Fhandler->reset_wait = false;
> -
> -    pthread_barrier_wait(&(Fhandler->finalize_barrier));
> -
> -    pthread_mutex_lock(&(Fhandler->m_more));
> -    pthread_cond_signal( &(Fhandler->condition_more ));
> -    pthread_mutex_unlock(&(Fhandler->m_more));
> -
> -
>  }
>  
>  void AIOwrapper::finalize_Y()
> @@ -838,7 +924,7 @@
>      Fhandler->Ar_Amount = totalR;
>      Fhandler->Ar_to_readSize = Fhandler->Ar_Amount;
>  
> -    int buff_count = min(3,(totalR+ desired_blockSize - 1)/desired_blockSize);
> +    int buff_count = 4;
>  
>      Fhandler->Ar_currentReadBuff = 0;
>      type_buffElement* tmp;
> 
> Modified: pkg/OmicABELnoMM/src/AIOwrapper.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/AIOwrapper.h	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -25,7 +25,7 @@
>      string fnameY;
>  
>  
> -    string fnameOutB;
> +    string fnameOutFiles;
>  
>  
>      list< pair<int,int> >* excl_List;
> @@ -46,8 +46,8 @@
>      queue<type_buffElement*> empty_buffers;
>      queue<type_buffElement*> full_buffers;
>  
> -    queue<type_buffElement*> b_empty_buffers;
> -    queue<type_buffElement*> b_full_buffers;
> +    queue<type_buffElement*> write_empty_buffers;
> +    queue<type_buffElement*> write_full_buffers;
>  
>      queue<type_buffElement*> ar_empty_buffers;
>      queue<type_buffElement*> ar_full_buffers;
> @@ -67,7 +67,8 @@
>      int y_blockSize;
>      int y_to_readSize;
>  
> -    int b_blockSize;
> +    int b_blockSize;
> +    int max_b_blockSize;
>  
>      bool not_done;
>      bool reset_wait;
> @@ -139,7 +140,11 @@
>          void reset_Y();
>          void reset_AR();
>  
> -        void write_B(type_precision* B, int p, int blockSize);
> +        void getCurrentWriteBuffers(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P);
> +
> +        void write_OutFiles(type_precision* &B,type_precision* &R,type_precision* &SD2,type_precision* &P,  int blockSize);
> +
> +        void write_significantValues(int Y, int X_R, float R, float SD2, float P);
>  
>          string io_overhead;
>  
> @@ -161,8 +166,8 @@
>          void prepare_AL( int columns, int n);
>          void finalize_AL();
>  
> -        void prepare_B(int b_blockSize, int p);
> -        void finalize_B();
> +        void prepare_OutFiles(int max_b_blockSize, int p);
> +        void finalize_OutFiles();
>  
>  
>          static void* async_io(void *ptr );
> 
> Modified: pkg/OmicABELnoMM/src/Algorithm.cpp
> ===================================================================
> --- pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/Algorithm.cpp	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -621,7 +621,7 @@
>  
>                  get_ticks(start_tick2);
>  
> -                AIOfile.write_B(B, p, a_block_size);
> +                //AIOfile.write_B(B, p, a_block_size);
>  
>                  get_ticks(end_tick);
>                  out.acc_storeb += ticks2sec(end_tick,start_tick2);
> 
> Modified: pkg/OmicABELnoMM/src/Definitions.h
> ===================================================================
> --- pkg/OmicABELnoMM/src/Definitions.h	2014-07-23 13:50:12 UTC (rev 1767)
> +++ pkg/OmicABELnoMM/src/Definitions.h	2014-07-25 10:48:03 UTC (rev 1768)
> @@ -5,41 +5,110 @@
>      #define LINUX
>  #else
>      #define WINDOWS
> -#endif
> +#endif
>  
> +#include <unistd.h>
> +#include <limits.h>
> +#include <queue>
> +#include <iostream>
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <time.h>       /* time */
>  #include <cstring>
>  #include <math.h>
> -#include <omp.h>
> +#include <omp.h>
> +#include <pthread.h>
>  
> -
> -
> +
>  #ifdef WINDOWS
>      #include <windows.h>
> -    #include <cblas.h>
>  #else
> -    //#include "mpi.h"
> -    //#define cpu_freq 3.0
> -    #define cpu_freq 3.2
> -    #include "cblas.h"
> +
> +#endif
> +
> +//!For intel use propetary MKL, it will be preferred over others
> +#ifdef __INTEL_MKL__
> +    #pragma message("MKL will Probably NOT compile")
> +    #include "mkl.h"
> +    #include "cblas.h"
> +    #include <lapacke.h>
> +    #define blas_set_num_threads(n) mkl_set_num_threads(n)
> +    #define STORAGE_TYPE LAPACK_COL_MAJOR
> +#else
> +
> +    //!For AMD systems use the proper ACML library, preferred over openblas ON AMD
> +    #ifdef _acml_
> +        #pragma message("Compiled with AMD ACML")
> +        #define blas_set_num_threads(n) omp_set_num_threads(n)
> +
> +        #include <acml.h>
> +
> +        #define lapack_int int
> +
> +        #define CblasTrans 'T'
> +        #define CblasNoTrans 'N'
> +        #define CblasUpper 'U'
> +        #define CblasColMajor 1
> +
> +
> +        #define STORAGE_TYPE CblasColMajor
> +
> +
> +        #define cblas_snrm2 snrm2
> +        #define cblas_saxpy saxpy
> +
> +        #ifndef BLASdefs_H_INCLUDED
> +        #define BLASdefs_H_INCLUDED
> +
> +        inline  void cblas_sgemm(int storage, char transa, char transb, int m, int n, int k, float alpha, float *a, int lda, float *b, int ldb, float beta, float *c, int ldc)
> +        {
> +           sgemm(transa, transb, m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
> +        }
> +
> +        inline  lapack_int LAPACKE_sposv( int matrix_order, char uplo, lapack_int n,  lapack_int nrhs, float* a, lapack_int lda, float* b, lapack_int ldb )
> +        {
> +            int info;
> +            sposv( uplo, n,  nrhs, a, lda, b,  ldb, &info);
> +            return info;
> +        }
> +
> +        inline  void cblas_ssyrk(int Order, char uplo, char Trans,
> +		 int N, int K, float alpha, float *A, int lda,  float beta, float *C, int ldc)
> +        {
> +            ssyrk(uplo, Trans, N, K, alpha, A, lda, beta, C, ldc);
> +        }
> +
> +        inline  lapack_int LAPACKE_sgels( int matrix_order, char trans, lapack_int m, lapack_int n, lapack_int nrhs, float* a,  lapack_int lda, float* b, lapack_int ldb )
> +        {
> +            int info;
> +            sgels(trans, m, n, nrhs, a, lda, b, ldb,&info);
> +            return info;
> +        }
> +
> +        #endif
> +
> +
> +
> +
> +
> +    #else
> +
> +        //!IF MKL is not present on INTEL, use openblas
> +        #ifdef _openblas_
> +            #pragma message("Compiled with OPENBLAS")
> +            #define STORAGE_TYPE LAPACK_COL_MAJOR
> +            #include "cblas.h"
> +            #include <lapacke.h>
> +            extern "C" void openblas_set_num_threads(int num_threads);
> +            #define blas_set_num_threads(n) openblas_set_num_threads(n)
> +        #endif
> +
> +    #endif
> +
>  #endif
> -#ifdef __INTEL_MKL__
> -    #include "mkl.h"
> -    #define blas_set_num_threads(n) mkl_set_num_threads(n)
> -#else
> -    extern "C" void openblas_set_num_threads(int num_threads);
> -    #define blas_set_num_threads(n) openblas_set_num_threads(n)
> -#endif
>  
> -#include <unistd.h>
> -#include <pthread.h>
> -#include <limits.h>
> -#include <queue>
> -#include <iostream>
> -#include <lapacke.h>
>  
> +
>  //!SETTINGS
>  
>  #define EXTENDEDTEST 0
> @@ -47,7 +116,7 @@
>  
>  #define OUTPUT 0
>  
> -#define STORAGE_TYPE LAPACK_COL_MAJOR
> +
>  #define type_precision float
>  
>  #define MIN(a,b) (((a)<(b))?(a):(b))
> @@ -56,86 +125,8 @@
>  #define _10MB 10*_1MB
>  #define _1GB 1024*1024*1024
>  
> -//!for CPU speed!
>  
> -//#ifdef WIN32
> -//#define WIN32_LEAN_AND_MEAN
> -//#include <windows.h>
> -//typedef unsigned __int64 usCount;
> -//static usCount GetUsCount()
> -//{
> -//    static LARGE_INTEGER ticksPerSec;
> -//    static double scalefactor;
> -//    LARGE_INTEGER val;
> -//    if (!scalefactor)
> -//    {
> -//        if (QueryPerformanceFrequency(&ticksPerSec))
> -//            scalefactor=ticksPerSec.QuadPart/1000000000000.0;
> -//        else
> -//            scalefactor=1;
> -//    }
> -//    if (!QueryPerformanceCounter(&val))
> -//        return (usCount) GetTickCount() * 1000000000;
> -//    return (usCount) (val.QuadPart/scalefactor);
> -//}
> -//#else
> -//#include <sys/time.h>
> -//#include <time.h>
> -//#include <sched.h>
> -//typedef unsigned long long usCount;
> -//static usCount GetUsCount()
> -//{
> -//#ifdef CLOCK_MONOTONIC
> -//    struct timespec ts;
> -//    clock_gettime(CLOCK_MONOTONIC, &ts);
> -//    return ((usCount) ts.tv_sec*1000000000000LL)+ts.tv_nsec*1000LL;
> -//#else
> -//    struct timeval tv;
> -//    gettimeofday(&tv, 0);
> -//    return ((usCount) tv.tv_sec*1000000000000LL)+tv.tv_usec*1000000LL;
> -//#endif
> -//}
> -//#endif
> -//static usCount usCountOverhead;
> -//#ifdef __GNUC__
> -//#include "x86intrin.h"
> -//#define __rdtsc() __builtin_ia32_rdtsc()
> -//#endif
>  
> -//static usCount GetClockSpeed()
> -//{
> -//    int n;
> -//    usCount start, end, start_tsc, end_tsc;
> -//    if (!usCountOverhead)
> -//    {
> -//        usCount foo = 0;
> -//        start=GetUsCount();
> -//        for (n = 0; n < 1000000; n++)
> -//        {
> -//            foo += GetUsCount();
> -//        }
> -//        end = GetUsCount();
> -//        usCountOverhead = (end - start)/n;
> -//    }
> -//
> -//    start = GetUsCount();
> -//    start_tsc = __rdtsc();
> -//    for (n = 0; n <1000; n++)
> -//    {
> -//#ifdef WIN32
> -//        Sleep(0);
> -//#else
> -//        sched_yield();
> -//#endif
> -//    }
> -//
> -//    end_tsc = __rdtsc();
> -//    end = GetUsCount();
> -//    return(usCount)((1000000000000.0 * (end_tsc - start_tsc)) /
> -//                    (end - start - usCountOverhead));
> -//}
> -
> -
>  using namespace std;
>  
>  
> @@ -149,7 +140,9 @@
>      int p;
>      int tb;
>      int mb;
> -    int id;
> +    int id;
> +
> +    float sig_threshold;
>  
>      int threads;
>  
> @@ -158,7 +151,7 @@
>      string fnameAL;
>      string fnameAR;
>      string fnameY;
> -    string fnameOutB;
> +    string fnameOutFiles;
>      string fname_excludelist;
>  
> [TRUNCATED]
> 
> To get the complete diff run:
>     svnlook diff /svnroot/genabel -r 1768
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/caf54e2e/attachment-0001.sig>

From lennart at karssen.org  Fri Jul 25 15:59:56 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 15:59:56 +0200
Subject: [GenABEL-dev] ACML and make check
In-Reply-To: <244CF001646FF74FB34F372310A332C50115301D@MBX-S2.rwth-ad.de>
References: <244CF001646FF74FB34F372310A332C50115301D@MBX-S2.rwth-ad.de>
Message-ID: <53D262DC.4010800@karssen.org>

Hi Alvaro,

On 25-07-14 13:00, Frank, Alvaro Jesus wrote:
> Hi Lennart,
> 
> I committed an ugly/nice hack to allow the usage of AMD's ACML library
> with omicabelnomm. This was required due to the program having VERY bad
> performance issues making it unusable on old Opteron systems prior to
> the 6200 versions. This are all that have non AVX instructions. I might
> have broken a few Jenkins warnings due to this, but those will be fixed
> later.
> Normal OMICABEL should be tested against ACML this too. Using MKL on the
> Opteron System, with 10 traits and 400000 snps. the runtime takes over
> 10 hours. This does not seem like a normal behavior.
> 
> AMD ACMl also allows to use GPU without ANY change, the system just
> needs to support OpenCL, which NVIDIA and AMD do. I have no access to an
> Opteron/AMD system that has GPU so I cant test it yet.

That's cool! Unfortunately I don't have access to an AMD GPU either :-(.

> 
> The changes also allow for static linking now. So no need to have root
> access. The howtocompile file reflects this now.
> 
> I also have a very improtant request. Your change of make check will not
> display the output of the testing program as it runs. I previosly
> compiled the test file and manually called it, giving me a very quick
> workflow to test the system. But now my 1 cmd do it all workflow is
> broken, having to use make check to compile test.cpp and then having to
> abort it manually to later re-call it manually to be able to see the
> output in real time. Is there a way to change make check to allow for
> real time output? I tried gooooogoling this, but failed with the most
> common solutions.
> 

The default way of running tests has changed in a recent automake
update. It now allows tests to be run in parallel (assuming more than
one exists). That's why the screen output isn't printed to screen anymore.

The actual screen output of the test can be found in the file
tests/test_omicabelnomm.log. So you can check that after the 'make
check' run.

Of course, nothing prevents you from running 'tests/test_omicablenomm'
after you've run './configure' followed by 'make'. 'make check' makes
sense only if the exit status of the tests returns 0 or 1 for success or
failure, respectively.

> A rewrite of Algorithm.cpp is coming next. it will break the
> validity/give wrong results perhaps at first.

Actually, the current HEAD already gives me compile errors:

++ -static -O3 -I../libs/include/ -I./libs/include/ -fopenmp
-D_openblas_   -L./libs/lib/ -L../libs/lib/ -o omicabelnomm
src/AIOwrapper.o src/Algorithm.o src/Utility.o src/main.o  -llapacke
-lopenblas -lgfortran
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
In function `LAPACKE_sgels_work':
(.text+0x1a3): undefined reference to `sgels_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
In function `LAPACKE_sgels_work':
(.text+0x28c): undefined reference to `sgels_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
In function `LAPACKE_sposv_work':
(.text+0x151): undefined reference to `sposv_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
In function `LAPACKE_sposv_work':
(.text+0x240): undefined reference to `sposv_'
collect2: error: ld returned 1 exit status
make: *** [omicabelnomm] Error 1


Did your commit change anything in the lapack requirements?


Thanks,

Lennart.



> 
> Thank You.
> 
> Alvaro Frank

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/e059f455/attachment.sig>

From alvaro.frank at rwth-aachen.de  Fri Jul 25 16:15:37 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Fri, 25 Jul 2014 14:15:37 +0000
Subject: [GenABEL-dev] ACML and make check
In-Reply-To: <53D262DC.4010800@karssen.org>
References: <244CF001646FF74FB34F372310A332C50115301D@MBX-S2.rwth-ad.de>,
 <53D262DC.4010800@karssen.org>
Message-ID: <244CF001646FF74FB34F372310A332C50115307A@MBX-S2.rwth-ad.de>

I do not recommend using distributed libraries. Use self compiled libraries. It will not fail if done so. Yours may compile, but no one guarantees that OPENMP=1 was used in the compilation for it.

All errors pointed out were not present in series of 3 different environments. 3 machines, 3 libraries each. All different architectures. I corrected configure.ac anyway. The LAPACKE_sposv_work are from -static linking with unsuitable libraries. Therefore self-compilation. 

We could, just like with normal omicabel, supply static precompiled versions, with both AMD and a proper OpenBLAS OPENMP=1, DYNAMIC_ARCH=0, etc. Committed the changes anyway. There is no more static linking and the user is responsible for having the libraries available at run-time with sudo rights. 

I don't have an environment where this commit fails.

P.S: I still need to compile the test without having to abort it manually. Screen output in real time is still a requirement for me.

-A Frank
________________________________________
From: L.C. Karssen [lennart at karssen.org]
Sent: Friday, July 25, 2014 3:59 PM
To: Frank, Alvaro Jesus; genabel-devel at lists.r-forge.r-project.org
Subject: Re: ACML and make check

Hi Alvaro,

On 25-07-14 13:00, Frank, Alvaro Jesus wrote:
> Hi Lennart,
>
> I committed an ugly/nice hack to allow the usage of AMD's ACML library
> with omicabelnomm. This was required due to the program having VERY bad
> performance issues making it unusable on old Opteron systems prior to
> the 6200 versions. This are all that have non AVX instructions. I might
> have broken a few Jenkins warnings due to this, but those will be fixed
> later.
> Normal OMICABEL should be tested against ACML this too. Using MKL on the
> Opteron System, with 10 traits and 400000 snps. the runtime takes over
> 10 hours. This does not seem like a normal behavior.
>
> AMD ACMl also allows to use GPU without ANY change, the system just
> needs to support OpenCL, which NVIDIA and AMD do. I have no access to an
> Opteron/AMD system that has GPU so I cant test it yet.

That's cool! Unfortunately I don't have access to an AMD GPU either :-(.

>
> The changes also allow for static linking now. So no need to have root
> access. The howtocompile file reflects this now.
>
> I also have a very improtant request. Your change of make check will not
> display the output of the testing program as it runs. I previosly
> compiled the test file and manually called it, giving me a very quick
> workflow to test the system. But now my 1 cmd do it all workflow is
> broken, having to use make check to compile test.cpp and then having to
> abort it manually to later re-call it manually to be able to see the
> output in real time. Is there a way to change make check to allow for
> real time output? I tried gooooogoling this, but failed with the most
> common solutions.
>

The default way of running tests has changed in a recent automake
update. It now allows tests to be run in parallel (assuming more than
one exists). That's why the screen output isn't printed to screen anymore.

The actual screen output of the test can be found in the file
tests/test_omicabelnomm.log. So you can check that after the 'make
check' run.

Of course, nothing prevents you from running 'tests/test_omicablenomm'
after you've run './configure' followed by 'make'. 'make check' makes
sense only if the exit status of the tests returns 0 or 1 for success or
failure, respectively.

> A rewrite of Algorithm.cpp is coming next. it will break the
> validity/give wrong results perhaps at first.

Actually, the current HEAD already gives me compile errors:

++ -static -O3 -I../libs/include/ -I./libs/include/ -fopenmp
-D_openblas_   -L./libs/lib/ -L../libs/lib/ -o omicabelnomm
src/AIOwrapper.o src/Algorithm.o src/Utility.o src/main.o  -llapacke
-lopenblas -lgfortran
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
In function `LAPACKE_sgels_work':
(.text+0x1a3): undefined reference to `sgels_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
In function `LAPACKE_sgels_work':
(.text+0x28c): undefined reference to `sgels_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
In function `LAPACKE_sposv_work':
(.text+0x151): undefined reference to `sposv_'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
In function `LAPACKE_sposv_work':
(.text+0x240): undefined reference to `sposv_'
collect2: error: ld returned 1 exit status
make: *** [omicabelnomm] Error 1


Did your commit change anything in the lapack requirements?


Thanks,

Lennart.



>
> Thank You.
>
> Alvaro Frank

--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-


From lennart at karssen.org  Fri Jul 25 16:24:26 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 16:24:26 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1770 - pkg/OmicABELnoMM
In-Reply-To: <20140725141256.2C4D0183E26@r-forge.r-project.org>
References: <20140725141256.2C4D0183E26@r-forge.r-project.org>
Message-ID: <53D2689A.2080503@karssen.org>

Thanks for implementing the suggestions.

I still get the compile errors when I run make. I'll try to find out if
it's specific to my enviromnent.


Lennart.

On 25-07-14 16:12, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-25 16:12:55 +0200 (Fri, 25 Jul 2014)
> New Revision: 1770
> 
> Modified:
>    pkg/OmicABELnoMM/configure.ac
> Log:
> Rest of the changes on configure.ac as requested. Have no envireoment where it fails. Need third party to check.
> 
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac	2014-07-25 14:05:25 UTC (rev 1769)
> +++ pkg/OmicABELnoMM/configure.ac	2014-07-25 14:12:55 UTC (rev 1770)
> @@ -4,8 +4,8 @@
>  AC_PREREQ([2.67])
>  AC_INIT([OmicABELnoMM], [0.1.0], [genabel-devel at r-forge.wu-wien.ac.at])
>  AM_INIT_AUTOMAKE([])
> -AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> -AM_SILENT_RULES([no])
> +#AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> +#AM_SILENT_RULES([yes])
>  AC_CONFIG_SRCDIR([src/Utility.h])
>  AC_CONFIG_HEADERS([src/config.h])
>  
> @@ -36,7 +36,7 @@
>  AC_OPENMP
>  AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>  
> -AM_CXXFLAGS="-static -O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
> +AM_CXXFLAGS="-O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
>  
>  # Checks for libraries.
>  # pthread library
> @@ -62,15 +62,15 @@
>  if test "$found_blas" -eq 0
>  	then
>  
> -	# Openblas
> -	AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> -	   AC_MSG_ERROR([OpenBLAS  library NOT found])
> -	])
> -	 #Lapack
> -	AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> -	   AC_MSG_ERROR([Unable to find a Lapack library])
> -	])
> -	AC_MSG_NOTICE([Using OpenBLAS  library])
> +# Openblas
> +AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> +   AC_MSG_ERROR([OpenBLAS  library NOT found])
> +])
> + #Lapack
> +AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> +   AC_MSG_ERROR([Unable to find a Lapack library])
> +])
> +AC_MSG_NOTICE([Using OpenBLAS  library])
>  fi
>  
>  
> 
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/581cc839/attachment.sig>

From lennart at karssen.org  Fri Jul 25 16:25:55 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 16:25:55 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1770 - pkg/OmicABELnoMM
In-Reply-To: <20140725141256.2C4D0183E26@r-forge.r-project.org>
References: <20140725141256.2C4D0183E26@r-forge.r-project.org>
Message-ID: <53D268F3.8070601@karssen.org>

Hi Alvaro,

I just accidentally commented on the previous commit. This commit
wrongly reintroduces the older changes.


Lennart.

On 25-07-14 16:12, noreply at r-forge.r-project.org wrote:
> Author: afrank
> Date: 2014-07-25 16:12:55 +0200 (Fri, 25 Jul 2014)
> New Revision: 1770
> 
> Modified:
>    pkg/OmicABELnoMM/configure.ac
> Log:
> Rest of the changes on configure.ac as requested. Have no envireoment where it fails. Need third party to check.
> 
> Modified: pkg/OmicABELnoMM/configure.ac
> ===================================================================
> --- pkg/OmicABELnoMM/configure.ac	2014-07-25 14:05:25 UTC (rev 1769)
> +++ pkg/OmicABELnoMM/configure.ac	2014-07-25 14:12:55 UTC (rev 1770)
> @@ -4,8 +4,8 @@
>  AC_PREREQ([2.67])
>  AC_INIT([OmicABELnoMM], [0.1.0], [genabel-devel at r-forge.wu-wien.ac.at])
>  AM_INIT_AUTOMAKE([])
> -AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> -AM_SILENT_RULES([no])
> +#AM_INIT_AUTOMAKE([silent-rules subdir-objects])
> +#AM_SILENT_RULES([yes])
>  AC_CONFIG_SRCDIR([src/Utility.h])
>  AC_CONFIG_HEADERS([src/config.h])
>  
> @@ -36,7 +36,7 @@
>  AC_OPENMP
>  AC_SUBST(AM_CXXFLAGS, "$OPENMP_CFLAGS")
>  
> -AM_CXXFLAGS="-static -O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
> +AM_CXXFLAGS="-O3 -I../libs/include/ -I./libs/include/ $AM_CXXFLAGS"
>  
>  # Checks for libraries.
>  # pthread library
> @@ -62,15 +62,15 @@
>  if test "$found_blas" -eq 0
>  	then
>  
> -	# Openblas
> -	AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> -	   AC_MSG_ERROR([OpenBLAS  library NOT found])
> -	])
> -	 #Lapack
> -	AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> -	   AC_MSG_ERROR([Unable to find a Lapack library])
> -	])
> -	AC_MSG_NOTICE([Using OpenBLAS  library])
> +# Openblas
> +AC_SEARCH_LIBS([cblas_sgemm], [openblas], [AM_CXXFLAGS="$AM_CXXFLAGS -D_openblas_"], [
> +   AC_MSG_ERROR([OpenBLAS  library NOT found])
> +])
> + #Lapack
> +AC_SEARCH_LIBS([LAPACKE_sgeqrf], [lapack, lapacke], [], [
> +   AC_MSG_ERROR([Unable to find a Lapack library])
> +])
> +AC_MSG_NOTICE([Using OpenBLAS  library])
>  fi
>  
>  
> 
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/e855063e/attachment.sig>

From lennart at karssen.org  Fri Jul 25 20:12:19 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Fri, 25 Jul 2014 20:12:19 +0200
Subject: [GenABEL-dev] ACML and make check
In-Reply-To: <244CF001646FF74FB34F372310A332C50115307A@MBX-S2.rwth-ad.de>
References: <244CF001646FF74FB34F372310A332C50115301D@MBX-S2.rwth-ad.de>,
 <53D262DC.4010800@karssen.org>
 <244CF001646FF74FB34F372310A332C50115307A@MBX-S2.rwth-ad.de>
Message-ID: <53D29E03.9000302@karssen.org>



On 25-07-14 16:15, Frank, Alvaro Jesus wrote:
> I do not recommend using distributed libraries. Use self compiled libraries. It will not fail if done so. Yours may compile, but no one guarantees that OPENMP=1 was used in the compilation for it.
> 
> All errors pointed out were not present in series of 3 different environments. 3 machines, 3 libraries each. All different architectures. I corrected configure.ac anyway. The LAPACKE_sposv_work are from -static linking with unsuitable libraries. Therefore self-compilation. 
> 
> We could, just like with normal omicabel, supply static precompiled versions, with both AMD and a proper OpenBLAS OPENMP=1, DYNAMIC_ARCH=0, etc. Committed the changes anyway. There is no more static linking and the user is responsible for having the libraries available at run-time with sudo rights. 
> 
> I don't have an environment where this commit fails.

OK, thanks, I'll try that.

> 
> P.S: I still need to compile the test without having to abort it manually. Screen output in real time is still a requirement for me.

You could treat test_omicablenomm as a regular binary (at least for now)
instead of a test file, so it would get compile just like omicablenomm.


Lennart


> 
> -A Frank
> ________________________________________
> From: L.C. Karssen [lennart at karssen.org]
> Sent: Friday, July 25, 2014 3:59 PM
> To: Frank, Alvaro Jesus; genabel-devel at lists.r-forge.r-project.org
> Subject: Re: ACML and make check
> 
> Hi Alvaro,
> 
> On 25-07-14 13:00, Frank, Alvaro Jesus wrote:
>> Hi Lennart,
>>
>> I committed an ugly/nice hack to allow the usage of AMD's ACML library
>> with omicabelnomm. This was required due to the program having VERY bad
>> performance issues making it unusable on old Opteron systems prior to
>> the 6200 versions. This are all that have non AVX instructions. I might
>> have broken a few Jenkins warnings due to this, but those will be fixed
>> later.
>> Normal OMICABEL should be tested against ACML this too. Using MKL on the
>> Opteron System, with 10 traits and 400000 snps. the runtime takes over
>> 10 hours. This does not seem like a normal behavior.
>>
>> AMD ACMl also allows to use GPU without ANY change, the system just
>> needs to support OpenCL, which NVIDIA and AMD do. I have no access to an
>> Opteron/AMD system that has GPU so I cant test it yet.
> 
> That's cool! Unfortunately I don't have access to an AMD GPU either :-(.
> 
>>
>> The changes also allow for static linking now. So no need to have root
>> access. The howtocompile file reflects this now.
>>
>> I also have a very improtant request. Your change of make check will not
>> display the output of the testing program as it runs. I previosly
>> compiled the test file and manually called it, giving me a very quick
>> workflow to test the system. But now my 1 cmd do it all workflow is
>> broken, having to use make check to compile test.cpp and then having to
>> abort it manually to later re-call it manually to be able to see the
>> output in real time. Is there a way to change make check to allow for
>> real time output? I tried gooooogoling this, but failed with the most
>> common solutions.
>>
> 
> The default way of running tests has changed in a recent automake
> update. It now allows tests to be run in parallel (assuming more than
> one exists). That's why the screen output isn't printed to screen anymore.
> 
> The actual screen output of the test can be found in the file
> tests/test_omicabelnomm.log. So you can check that after the 'make
> check' run.
> 
> Of course, nothing prevents you from running 'tests/test_omicablenomm'
> after you've run './configure' followed by 'make'. 'make check' makes
> sense only if the exit status of the tests returns 0 or 1 for success or
> failure, respectively.
> 
>> A rewrite of Algorithm.cpp is coming next. it will break the
>> validity/give wrong results perhaps at first.
> 
> Actually, the current HEAD already gives me compile errors:
> 
> ++ -static -O3 -I../libs/include/ -I./libs/include/ -fopenmp
> -D_openblas_   -L./libs/lib/ -L../libs/lib/ -o omicabelnomm
> src/AIOwrapper.o src/Algorithm.o src/Utility.o src/main.o  -llapacke
> -lopenblas -lgfortran
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
> In function `LAPACKE_sgels_work':
> (.text+0x1a3): undefined reference to `sgels_'
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sgels_work.o):
> In function `LAPACKE_sgels_work':
> (.text+0x28c): undefined reference to `sgels_'
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
> In function `LAPACKE_sposv_work':
> (.text+0x151): undefined reference to `sposv_'
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/liblapacke.a(lapacke_sposv_work.o):
> In function `LAPACKE_sposv_work':
> (.text+0x240): undefined reference to `sposv_'
> collect2: error: ld returned 1 exit status
> make: *** [omicabelnomm] Error 1
> 
> 
> Did your commit change anything in the lapack requirements?
> 
> 
> Thanks,
> 
> Lennart.
> 
> 
> 
>>
>> Thank You.
>>
>> Alvaro Frank
> 
> --
> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
> L.C. Karssen
> Utrecht
> The Netherlands
> 
> lennart at karssen.org
> http://blog.karssen.org
> GPG key ID: A88F554A
> -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140725/ca0d6ff3/attachment-0001.sig>

From lennart at karssen.org  Mon Jul 28 12:07:01 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Mon, 28 Jul 2014 12:07:01 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1772 - pkg/DatABEL/R
In-Reply-To: <20140728094146.C37AA1839E7@r-forge.r-project.org>
References: <20140728094146.C37AA1839E7@r-forge.r-project.org>
Message-ID: <53D620C5.1040906@karssen.org>

Hi Maksim,


Thanks for the quick fix!


Lennart


On 28-07-14 11:41, noreply at r-forge.r-project.org wrote:
> Author: maksim
> Date: 2014-07-28 11:41:46 +0200 (Mon, 28 Jul 2014)
> New Revision: 1772
> 
> Modified:
>    pkg/DatABEL/R/checkPackageVersionOnCRAN.R
> Log:
> Fixed version check from CRAN site
> 
> Modified: pkg/DatABEL/R/checkPackageVersionOnCRAN.R
> ===================================================================
> --- pkg/DatABEL/R/checkPackageVersionOnCRAN.R	2014-07-25 14:29:59 UTC (rev 1771)
> +++ pkg/DatABEL/R/checkPackageVersionOnCRAN.R	2014-07-28 09:41:46 UTC (rev 1772)
> @@ -26,35 +26,67 @@
>  checkPackageVersionOnCRAN <- function(packageName,baseUrlCRAN="http://cran.r-project.org/web/packages/", 
>  		timeout = 2)
>  {
> -	# change default timout
> -	svtmo <- options("timeout")
> -	options("timeout"=timeout)
> -	# page to check is
> -	pageAddress <- paste(baseUrlCRAN,packageName,sep="/")
> -	# establish connection to the CRAN page of the package
> -	suppressWarnings(
> -			conn <- try( url(pageAddress) , silent=TRUE )
> -	)
> -	# if connection ok, read full page, store the results in pageContent; if failed, pageContent <- "try-error"
> -	if ( all( class(conn) != "try-error") ) {
> -		suppressWarnings(
> -				pageContent <- try( readLines(conn) , silent=TRUE )
> -		)
> -		close(conn)
> -	} else {
> -		pageContent <- "try-error"
> -		class(pageContent) <- "try-error"
> -	}
> -	# restore default timeout
> -	options("timeout"=svtmo)
> -	# if failed in reading (pageContent is "try-error"), return NULL
> -	if (class(pageContent) == "try-error") return(NULL)
> -	# parse the page and get string starting with "Package source:"
> -	targetLine <- pageContent[grep("source:",pageContent)]
> -	# split the string at "Package_" and ".tar.gz"; the element before the last will contain the version
> -	splitPattern <- paste(packageName,"_|.tar.gz",sep="")
> -	stringSplit <- strsplit(targetLine,splitPattern)
> -	cranVersion <- stringSplit[[1]][length(stringSplit[[1]])-1]
> -	# return version
> -	return(cranVersion)
> +
> +CRAN_mirror_ref <- "http://cran.xl-mirror.nl/src/contrib" # Amsterdam mirror
> +
> +all_cran_packs <- available.packages(contriburl=CRAN_mirror_ref)
> +all_cran_packs_df <- as.data.frame(all_cran_packs)
> +
> +databel_info <- all_cran_packs_df[all_cran_packs_df$Package == "DatABEL",]
> +cranVersion <- as.character(databel_info$Version)
> +
> +return(cranVersion)
> +
> +
> +
> +# Below is an old verion which depends on CRAN site content that is unreliable.
> +
> +#	# change default timout
> +#	svtmo <- options("timeout")
> +#	options("timeout"=timeout)
> +#	# page to check is
> +#	pageAddress <- paste(baseUrlCRAN,packageName,sep="/")
> +#	# establish connection to the CRAN page of the package
> +#	suppressWarnings(
> +#			conn <- try( url(pageAddress) , silent=TRUE )
> +#	)
> +#	# if connection ok, read full page, store the results in pageContent; if failed, pageContent <- "try-error"
> +#	if ( all( class(conn) != "try-error") ) {
> +#		suppressWarnings(
> +#				pageContent <- try( readLines(conn) , silent=TRUE )
> +#		)
> +#		close(conn)
> +#	} else {
> +#		pageContent <- "try-error"
> +#		class(pageContent) <- "try-error"
> +#	}
> +#	# restore default timeout
> +#	options("timeout"=svtmo)
> +#	# if failed in reading (pageContent is "try-error"), return NULL
> +#	if (class(pageContent) == "try-error") return(NULL)
> +#	# parse the page and get string starting with "Package source:"
> +#	targetLine <- pageContent[grep("source:",pageContent)]
> +#	# split the string at "Package_" and ".tar.gz"; the element before the last will contain the version
> +#	splitPattern <- paste(packageName,"_|.tar.gz",sep="")
> +#	stringSplit <- strsplit(targetLine,splitPattern)
> +#	cranVersion <- stringSplit[[1]][length(stringSplit[[1]])-1]
> +#	# return version
> +#  return(cranVersion)
> +
> +
> +
> +# Below is a patch received from Marco De Simone <marco.desimone at cbm.fvg.it>. This version is, however, use content if CRAN that is aso unreliable.
> +#
> +#options("timeout"=svtmo)
> +## if failed in reading (pageContent is "try-error"), return NULL
> +#if (class(pageContent) == "try-error") return(NULL)
> +## parse the table inside the page for the cell with the content "Version:" and select the following cell that contains the version"
> +#targetLine <- pageContent[grep("Version:",pageContent)+1]
> +## Strip the html tags <td> and </td> and get the content value of version
> +#splitPattern <- paste("<td>|</td>",sep="")
> +#
> +#cranVersion <- stringSplit[[1]][length(stringSplit[[1]])]
> +## return version
> +#return(cranVersion)
> +
>  }
> 
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140728/57e144ed/attachment.sig>

From lennart at karssen.org  Mon Jul 28 12:10:10 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Mon, 28 Jul 2014 12:10:10 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1773 - pkg/DatABEL
In-Reply-To: <20140728100451.63272187620@r-forge.r-project.org>
References: <20140728100451.63272187620@r-forge.r-project.org>
Message-ID: <53D62182.1070306@karssen.org>

Hi Maksim,

On 28-07-14 12:04, noreply at r-forge.r-project.org wrote:
> Author: maksim
> Date: 2014-07-28 12:04:50 +0200 (Mon, 28 Jul 2014)
> New Revision: 1773
> 
> Modified:
>    pkg/DatABEL/DESCRIPTION
> Log:
> Prepare for submission to CRAN
> 
> Modified: pkg/DatABEL/DESCRIPTION
> ===================================================================
> --- pkg/DatABEL/DESCRIPTION	2014-07-28 09:41:46 UTC (rev 1772)
> +++ pkg/DatABEL/DESCRIPTION	2014-07-28 10:04:50 UTC (rev 1773)
> @@ -1,9 +1,9 @@
>  Package: DatABEL
>  Type: Package
>  Title: file-based access to large matrices stored on HDD in binary format
> -Version: 0.9-4
> -Date: 2013-03-12
> -Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar
> +Version: 0.9-4.1

I'm not sure if CRAN allows this type of numbering (with a 4th digit). I
seem to remember it was not allowed.
My suggestion would be to go for 0.9-5.


Also, I hope my messing around with the documentation (which I still
haven't finished :-(, unfortunately) will not result in big problems
when running the CRAN checks.

Let me know if you need help.


Best,

Lennart.

> +Date: 2014-07-28
> +Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar, Maksim Struchalin
>  Maintainer: Yurii Aulchenko <yurii at bionet.nsc.ru>
>  Depends:
>      R (>= 2.4.0),
> 
> _______________________________________________
> Genabel-commits mailing list
> Genabel-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140728/68e63469/attachment.sig>

From alvaro.frank at rwth-aachen.de  Mon Jul 28 18:03:12 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Mon, 28 Jul 2014 16:03:12 +0000
Subject: [GenABEL-dev] databel float type
Message-ID: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>

Hi all,

what is wrong with this? R:
---------------------------------------------------------------------------------------------


library(DatABEL)

n = 100 # number of individuals
l = 2    # number of covariates
m = 10 # number of snps
t = 5  # number of traits

set.seed(1001)
runif(3)

XL <- matrix(rnorm((l+1)*n),ncol=(l+1)) # first column should be ones (intercept)
for(i in 1:n*l){ if(sample(1:10,1) > 5) XL[i]=0/0}

colnames(XL) <- c("intercept", paste("cov",1:l,sep=""))
rownames(XL) <- paste("ind",1:n,sep="")
XL_db <- matrix2databel(XL,filename="XL",type="FLOAT")



---------------------------------------------------------------------------------------------
The error that I get is this:

file contains data of unknown type 5
Error in !.Call("assignDoubleMatrix", x at data, newi, newj, as.double(value),  :
  invalid argument type

I cannot create any more databel flaot files....


A Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140728/c43c330d/attachment.html>

From alvaro.frank at rwth-aachen.de  Mon Jul 28 18:12:03 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Mon, 28 Jul 2014 16:12:03 +0000
Subject: [GenABEL-dev] databel float type
In-Reply-To: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>
References: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>
Message-ID: <244CF001646FF74FB34F372310A332C501153310@MBX-S2.rwth-ad.de>

So yes, it seems utterly broken.
I am gonna take a small break until it is fixed.

-A Frank

________________________________
From: genabel-devel-bounces at lists.r-forge.r-project.org [genabel-devel-bounces at lists.r-forge.r-project.org] on behalf of Frank, Alvaro Jesus [alvaro.frank at rwth-aachen.de]
Sent: Monday, July 28, 2014 6:03 PM
To: genabel-devel at lists.r-forge.r-project.org
Subject: [GenABEL-dev] databel float type

Hi all,

what is wrong with this? R:
---------------------------------------------------------------------------------------------


library(DatABEL)

n = 100 # number of individuals
l = 2    # number of covariates
m = 10 # number of snps
t = 5  # number of traits

set.seed(1001)
runif(3)

XL <- matrix(rnorm((l+1)*n),ncol=(l+1)) # first column should be ones (intercept)
for(i in 1:n*l){ if(sample(1:10,1) > 5) XL[i]=0/0}

colnames(XL) <- c("intercept", paste("cov",1:l,sep=""))
rownames(XL) <- paste("ind",1:n,sep="")
XL_db <- matrix2databel(XL,filename="XL",type="FLOAT")



---------------------------------------------------------------------------------------------
The error that I get is this:

file contains data of unknown type 5
Error in !.Call("assignDoubleMatrix", x at data, newi, newj, as.double(value),  :
  invalid argument type

I cannot create any more databel flaot files....


A Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140728/18e35cf1/attachment.html>

From m.v.struchalin at mail.ru  Mon Jul 28 18:29:53 2014
From: m.v.struchalin at mail.ru (Maksim Struchalin)
Date: Mon, 28 Jul 2014 23:29:53 +0700
Subject: [GenABEL-dev] databel float type
In-Reply-To: <244CF001646FF74FB34F372310A332C501153310@MBX-S2.rwth-ad.de>
References: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>
 <244CF001646FF74FB34F372310A332C501153310@MBX-S2.rwth-ad.de>
Message-ID: <53D67A81.7020003@mail.ru>

Dear Frank,

Try to install the latest version of DatABEL from 
https://app.box.com/s/4rngaf6dp9rkn18i29e2 and run your script again. 
This version was submited on CRAN today and will be publically available 
soon.

Sorry for that your break was so small :-).
Maksim


On 28/07/2014 23:12, Frank, Alvaro Jesus wrote:
> So yes, it seems utterly broken.
> I am gonna take a small break until it is fixed.
>
> -A Frank
>
> ------------------------------------------------------------------------
> *From:* genabel-devel-bounces at lists.r-forge.r-project.org 
> [genabel-devel-bounces at lists.r-forge.r-project.org] on behalf of 
> Frank, Alvaro Jesus [alvaro.frank at rwth-aachen.de]
> *Sent:* Monday, July 28, 2014 6:03 PM
> *To:* genabel-devel at lists.r-forge.r-project.org
> *Subject:* [GenABEL-dev] databel float type
>
> Hi all,
>
> what is wrong with this? R:
> ---------------------------------------------------------------------------------------------
>
>
> library(DatABEL)
>
> n = 100 # number of individuals
> l = 2    # number of covariates
> m = 10 # number of snps
> t = 5  # number of traits
>
> set.seed(1001)
> runif(3)
>
> XL <- matrix(rnorm((l+1)*n),ncol=(l+1)) # first column should be ones 
> (intercept)
> for(i in 1:n*l){ if(sample(1:10,1) > 5) XL[i]=0/0}
>
> colnames(XL) <- c("intercept", paste("cov",1:l,sep=""))
> rownames(XL) <- paste("ind",1:n,sep="")
> XL_db <- matrix2databel(XL,filename="XL",type="FLOAT")
>
>
>
> ---------------------------------------------------------------------------------------------
> The error that I get is this:
>
> file contains data of unknown type 5
> Error in !.Call("assignDoubleMatrix", x at data, newi, newj, 
> as.double(value),  :
>   invalid argument type
>
> I cannot create any more databel flaot files....
>
>
> A Frank
>
>
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140728/9ebec2e7/attachment-0001.html>

From m.v.struchalin at mail.ru  Tue Jul 29 03:43:35 2014
From: m.v.struchalin at mail.ru (Maksim Struchalin)
Date: Tue, 29 Jul 2014 08:43:35 +0700
Subject: [GenABEL-dev] [Genabel-commits] r1773 - pkg/DatABEL
In-Reply-To: <53D62182.1070306@karssen.org>
References: <20140728100451.63272187620@r-forge.r-project.org>
 <53D62182.1070306@karssen.org>
Message-ID: <53D6FC47.5030107@mail.ru>

Hi Lennart,

I changed the numbering. Looks like the forth digit is for internal use 
during 'R CMD check'.

There were no 'NOTE's relating documentation so it should be ok I guess.

best,
Maksim


On 28/07/2014 17:10, L.C. Karssen wrote:
> Hi Maksim,
>
> On 28-07-14 12:04, noreply at r-forge.r-project.org wrote:
>> Author: maksim
>> Date: 2014-07-28 12:04:50 +0200 (Mon, 28 Jul 2014)
>> New Revision: 1773
>>
>> Modified:
>>     pkg/DatABEL/DESCRIPTION
>> Log:
>> Prepare for submission to CRAN
>>
>> Modified: pkg/DatABEL/DESCRIPTION
>> ===================================================================
>> --- pkg/DatABEL/DESCRIPTION	2014-07-28 09:41:46 UTC (rev 1772)
>> +++ pkg/DatABEL/DESCRIPTION	2014-07-28 10:04:50 UTC (rev 1773)
>> @@ -1,9 +1,9 @@
>>   Package: DatABEL
>>   Type: Package
>>   Title: file-based access to large matrices stored on HDD in binary format
>> -Version: 0.9-4
>> -Date: 2013-03-12
>> -Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar
>> +Version: 0.9-4.1
> I'm not sure if CRAN allows this type of numbering (with a 4th digit). I
> seem to remember it was not allowed.
> My suggestion would be to go for 0.9-5.
>
>
> Also, I hope my messing around with the documentation (which I still
> haven't finished :-(, unfortunately) will not result in big problems
> when running the CRAN checks.
>
> Let me know if you need help.
>
>
> Best,
>
> Lennart.
>
>> +Date: 2014-07-28
>> +Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar, Maksim Struchalin
>>   Maintainer: Yurii Aulchenko <yurii at bionet.nsc.ru>
>>   Depends:
>>       R (>= 2.4.0),
>>
>> _______________________________________________
>> Genabel-commits mailing list
>> Genabel-commits at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
>>
>
>
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140729/cbfbc398/attachment.html>

From lennart at karssen.org  Tue Jul 29 10:08:41 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Tue, 29 Jul 2014 10:08:41 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1773 - pkg/DatABEL
In-Reply-To: <53D6FC47.5030107@mail.ru>
References: <20140728100451.63272187620@r-forge.r-project.org>
 <53D62182.1070306@karssen.org> <53D6FC47.5030107@mail.ru>
Message-ID: <53D75689.5010005@karssen.org>

Hi Maksim,

On 29-07-14 03:43, Maksim Struchalin wrote:
> Hi Lennart,
> 
> I changed the numbering. Looks like the forth digit is for internal use
> during 'R CMD check'.

Aha. I didn't know that.

> 
> There were no 'NOTE's relating documentation so it should be ok I guess.

Great! Let's see what CRAN has to say.


Thanks for uploading the fix so quickly,

Lennart.

> 
> best,
> Maksim
> 
> 
> On 28/07/2014 17:10, L.C. Karssen wrote:
>> Hi Maksim,
>>
>> On 28-07-14 12:04, noreply at r-forge.r-project.org wrote:
>>> Author: maksim
>>> Date: 2014-07-28 12:04:50 +0200 (Mon, 28 Jul 2014)
>>> New Revision: 1773
>>>
>>> Modified:
>>>    pkg/DatABEL/DESCRIPTION
>>> Log:
>>> Prepare for submission to CRAN
>>>
>>> Modified: pkg/DatABEL/DESCRIPTION
>>> ===================================================================
>>> --- pkg/DatABEL/DESCRIPTION	2014-07-28 09:41:46 UTC (rev 1772)
>>> +++ pkg/DatABEL/DESCRIPTION	2014-07-28 10:04:50 UTC (rev 1773)
>>> @@ -1,9 +1,9 @@
>>>  Package: DatABEL
>>>  Type: Package
>>>  Title: file-based access to large matrices stored on HDD in binary format
>>> -Version: 0.9-4
>>> -Date: 2013-03-12
>>> -Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar
>>> +Version: 0.9-4.1
>> I'm not sure if CRAN allows this type of numbering (with a 4th digit). I
>> seem to remember it was not allowed.
>> My suggestion would be to go for 0.9-5.
>>
>>
>> Also, I hope my messing around with the documentation (which I still
>> haven't finished :-(, unfortunately) will not result in big problems
>> when running the CRAN checks.
>>
>> Let me know if you need help.
>>
>>
>> Best,
>>
>> Lennart.
>>
>>> +Date: 2014-07-28
>>> +Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar, Maksim Struchalin
>>>  Maintainer: Yurii Aulchenko <yurii at bionet.nsc.ru>
>>>  Depends:
>>>      R (>= 2.4.0),
>>>
>>> _______________________________________________
>>> Genabel-commits mailing list
>>> Genabel-commits at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
>>>
>>
>>
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 
> 
> 
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140729/e8d162d2/attachment.sig>

From alvaro.frank at rwth-aachen.de  Tue Jul 29 10:26:51 2014
From: alvaro.frank at rwth-aachen.de (Frank, Alvaro Jesus)
Date: Tue, 29 Jul 2014 08:26:51 +0000
Subject: [GenABEL-dev] databel float type
In-Reply-To: <53D67A81.7020003@mail.ru>
References: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>
 <244CF001646FF74FB34F372310A332C501153310@MBX-S2.rwth-ad.de>,
 <53D67A81.7020003@mail.ru>
Message-ID: <244CF001646FF74FB34F372310A332C501154741@MBX-W1.rwth-ad.de>

Hi Maksim,

Thank you for the info. Sadly I have no idea how to do it.

A Frank
________________________________
From: Maksim Struchalin [m.v.struchalin at mail.ru]
Sent: Monday, July 28, 2014 6:29 PM
To: Frank, Alvaro Jesus; genabel-devel at lists.r-forge.r-project.org
Subject: Re: [GenABEL-dev] databel float type

Dear Frank,

Try to install the latest version of DatABEL from https://app.box.com/s/4rngaf6dp9rkn18i29e2 and run your script again. This version was submited on CRAN today and will be publically available soon.

Sorry for that your break was so small :-).
Maksim


On 28/07/2014 23:12, Frank, Alvaro Jesus wrote:
So yes, it seems utterly broken.
I am gonna take a small break until it is fixed.

-A Frank

________________________________
From: genabel-devel-bounces at lists.r-forge.r-project.org<mailto:genabel-devel-bounces at lists.r-forge.r-project.org> [genabel-devel-bounces at lists.r-forge.r-project.org<mailto:genabel-devel-bounces at lists.r-forge.r-project.org>] on behalf of Frank, Alvaro Jesus [alvaro.frank at rwth-aachen.de<mailto:alvaro.frank at rwth-aachen.de>]
Sent: Monday, July 28, 2014 6:03 PM
To: genabel-devel at lists.r-forge.r-project.org<mailto:genabel-devel at lists.r-forge.r-project.org>
Subject: [GenABEL-dev] databel float type

Hi all,

what is wrong with this? R:
---------------------------------------------------------------------------------------------


library(DatABEL)

n = 100 # number of individuals
l = 2    # number of covariates
m = 10 # number of snps
t = 5  # number of traits

set.seed(1001)
runif(3)

XL <- matrix(rnorm((l+1)*n),ncol=(l+1)) # first column should be ones (intercept)
for(i in 1:n*l){ if(sample(1:10,1) > 5) XL[i]=0/0}

colnames(XL) <- c("intercept", paste("cov",1:l,sep=""))
rownames(XL) <- paste("ind",1:n,sep="")
XL_db <- matrix2databel(XL,filename="XL",type="FLOAT")



---------------------------------------------------------------------------------------------
The error that I get is this:

file contains data of unknown type 5
Error in !.Call("assignDoubleMatrix", x at data, newi, newj, as.double(value),  :
  invalid argument type

I cannot create any more databel flaot files....


A Frank



_______________________________________________
genabel-devel mailing list
genabel-devel at lists.r-forge.r-project.org<mailto:genabel-devel at lists.r-forge.r-project.org>
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140729/3d1e796e/attachment.html>

From lennart at karssen.org  Tue Jul 29 10:45:50 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Tue, 29 Jul 2014 10:45:50 +0200
Subject: [GenABEL-dev] databel float type
In-Reply-To: <244CF001646FF74FB34F372310A332C501154741@MBX-W1.rwth-ad.de>
References: <244CF001646FF74FB34F372310A332C5011532FC@MBX-S2.rwth-ad.de>
 <244CF001646FF74FB34F372310A332C501153310@MBX-S2.rwth-ad.de>,
 <53D67A81.7020003@mail.ru>
 <244CF001646FF74FB34F372310A332C501154741@MBX-W1.rwth-ad.de>
Message-ID: <53D75F3E.9090506@karssen.org>

Hi Alvaro,

On 29-07-14 10:26, Frank, Alvaro Jesus wrote:
> Hi Maksim,
> 
> Thank you for the info. Sadly I have no idea how to do it.

You mean how to install the tar.gz Maksim provided? This can be done
e.g. by R CMD INSTALL file.tar.gz. See man R.


Lennart.


> 
> A Frank
> ------------------------------------------------------------------------
> *From:* Maksim Struchalin [m.v.struchalin at mail.ru]
> *Sent:* Monday, July 28, 2014 6:29 PM
> *To:* Frank, Alvaro Jesus; genabel-devel at lists.r-forge.r-project.org
> *Subject:* Re: [GenABEL-dev] databel float type
> 
> Dear Frank,
> 
> Try to install the latest version of DatABEL from
> https://app.box.com/s/4rngaf6dp9rkn18i29e2 and run your script again.
> This version was submited on CRAN today and will be publically available
> soon.
> 
> Sorry for that your break was so small :-).
> Maksim
> 
> 
> On 28/07/2014 23:12, Frank, Alvaro Jesus wrote:
>> So yes, it seems utterly broken.
>> I am gonna take a small break until it is fixed.
>>
>> -A Frank
>>
>> ------------------------------------------------------------------------
>> *From:* genabel-devel-bounces at lists.r-forge.r-project.org
>> [genabel-devel-bounces at lists.r-forge.r-project.org] on behalf of
>> Frank, Alvaro Jesus [alvaro.frank at rwth-aachen.de]
>> *Sent:* Monday, July 28, 2014 6:03 PM
>> *To:* genabel-devel at lists.r-forge.r-project.org
>> *Subject:* [GenABEL-dev] databel float type
>>
>> Hi all,
>>
>> what is wrong with this? R:
>> ---------------------------------------------------------------------------------------------
>>
>>
>> library(DatABEL)
>>
>> n = 100 # number of individuals
>> l = 2    # number of covariates
>> m = 10 # number of snps
>> t = 5  # number of traits
>>
>> set.seed(1001)
>> runif(3)
>>
>> XL <- matrix(rnorm((l+1)*n),ncol=(l+1)) # first column should be ones
>> (intercept)
>> for(i in 1:n*l){ if(sample(1:10,1) > 5) XL[i]=0/0}
>>
>> colnames(XL) <- c("intercept", paste("cov",1:l,sep=""))
>> rownames(XL) <- paste("ind",1:n,sep="")
>> XL_db <- matrix2databel(XL,filename="XL",type="FLOAT")
>>
>>
>>
>> ---------------------------------------------------------------------------------------------
>> The error that I get is this:
>>
>> file contains data of unknown type 5
>> Error in !.Call("assignDoubleMatrix", x at data, newi, newj,
>> as.double(value),  :
>>   invalid argument type
>>
>> I cannot create any more databel flaot files....
>>
>>
>> A Frank
>>
>>
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 
> 
> 
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140729/b4a049de/attachment.sig>

From lennart at karssen.org  Wed Jul 30 16:54:59 2014
From: lennart at karssen.org (L.C. Karssen)
Date: Wed, 30 Jul 2014 16:54:59 +0200
Subject: [GenABEL-dev] [Genabel-commits] r1773 - pkg/DatABEL
In-Reply-To: <53D75689.5010005@karssen.org>
References: <20140728100451.63272187620@r-forge.r-project.org>
 <53D62182.1070306@karssen.org> <53D6FC47.5030107@mail.ru>
 <53D75689.5010005@karssen.org>
Message-ID: <53D90743.4050609@karssen.org>

Hi Maksim,

I just noticed that CRAN lists DatABEL v0.9-4.1, whereas you set the
version number in the DESCRIPTION file to 0.9-5 in one of your later
commits.

When I installed v0.9-4.1 and loaded it in R I did get the old (March
12) package date, which you fixed in r1779 and r1780.

Maybe you submitted an older version of the tar.gz file to CRAN, or did
you submit multiple versions and is 0.9-5 not processed by CRAN yet?


Best regards,

Lennart.


On 29-07-14 10:08, L.C. Karssen wrote:
> Hi Maksim,
> 
> On 29-07-14 03:43, Maksim Struchalin wrote:
>> Hi Lennart,
>>
>> I changed the numbering. Looks like the forth digit is for internal use
>> during 'R CMD check'.
> 
> Aha. I didn't know that.
> 
>>
>> There were no 'NOTE's relating documentation so it should be ok I guess.
> 
> Great! Let's see what CRAN has to say.
> 
> 
> Thanks for uploading the fix so quickly,
> 
> Lennart.
> 
>>
>> best,
>> Maksim
>>
>>
>> On 28/07/2014 17:10, L.C. Karssen wrote:
>>> Hi Maksim,
>>>
>>> On 28-07-14 12:04, noreply at r-forge.r-project.org wrote:
>>>> Author: maksim
>>>> Date: 2014-07-28 12:04:50 +0200 (Mon, 28 Jul 2014)
>>>> New Revision: 1773
>>>>
>>>> Modified:
>>>>    pkg/DatABEL/DESCRIPTION
>>>> Log:
>>>> Prepare for submission to CRAN
>>>>
>>>> Modified: pkg/DatABEL/DESCRIPTION
>>>> ===================================================================
>>>> --- pkg/DatABEL/DESCRIPTION	2014-07-28 09:41:46 UTC (rev 1772)
>>>> +++ pkg/DatABEL/DESCRIPTION	2014-07-28 10:04:50 UTC (rev 1773)
>>>> @@ -1,9 +1,9 @@
>>>>  Package: DatABEL
>>>>  Type: Package
>>>>  Title: file-based access to large matrices stored on HDD in binary format
>>>> -Version: 0.9-4
>>>> -Date: 2013-03-12
>>>> -Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar
>>>> +Version: 0.9-4.1
>>> I'm not sure if CRAN allows this type of numbering (with a 4th digit). I
>>> seem to remember it was not allowed.
>>> My suggestion would be to go for 0.9-5.
>>>
>>>
>>> Also, I hope my messing around with the documentation (which I still
>>> haven't finished :-(, unfortunately) will not result in big problems
>>> when running the CRAN checks.
>>>
>>> Let me know if you need help.
>>>
>>>
>>> Best,
>>>
>>> Lennart.
>>>
>>>> +Date: 2014-07-28
>>>> +Author: Yurii Aulchenko, Stepan Yakovenko, Erik Roos, Marcel Kempenaar, Maksim Struchalin
>>>>  Maintainer: Yurii Aulchenko <yurii at bionet.nsc.ru>
>>>>  Depends:
>>>>      R (>= 2.4.0),
>>>>
>>>> _______________________________________________
>>>> Genabel-commits mailing list
>>>> Genabel-commits at lists.r-forge.r-project.org
>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-commits
>>>>
>>>
>>>
>>> _______________________________________________
>>> genabel-devel mailing list
>>> genabel-devel at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>
>>
>>
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>
> 
> 
> 
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 

-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands

lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20140730/cc4abfce/attachment.sig>