[GenABEL-dev] ISNAN problem in filevector found by Jenkins

Yury Aulchenko yurii.aulchenko at gmail.com
Mon Nov 25 12:26:55 CET 2013 

Tried this on Mac OS X, see below

On Nov 23, 2013, at 15:12 PM, Maksim Struchalin <m.v.struchalin at mail.ru> wrote:

> Hi Lennart,
> 
> https://app.box.com/s/iy41ug5qg4sbylul9oyn
> 
> This is an example demonstrating how a GenABEL function "impute2databel" calls a function "iteratorDA" from DatABEL. Here, GenABEL is compiled without flv's and iterator's code (I deleted it from src).
> 
> Could you run the test?:
> 0) Dowload the file test_GenABEL_iterator.tar.gz from https://app.box.com/s/iy41ug5qg4sbylul9oyn
> 1) decompress test_GenABEL_iterator.tar.gz
> 2) cd test_GenABEL_iterator
> 3) R CMD INSTALL DatABEL_0.9-4.tar.gz
> 4) R CMD INSTALL GenABEL_1.7-7.tar.gz

getting many warnings at step (4)

> 5) run test.R

getting 

> x <- impute2databel(geno="TEST10x15.geno", sample="impute.sample5", out="TEST10x15_T.geno", makeprob=FALSE, old=TRUE)
Loading required package: DatABEL
DatABEL v.0.9-4 (March 12, 2013) loaded

Options in effect: 
	 --infile    = TEST10x15.geno
	 --outfile   = ./tmp333314
	 --skiprows  = OFF
	 --skipcols  = 5
	 --cnrow     = OFF
	 --rncol     = ON, using column 2 of 'TEST10x15.geno'
	 --transpose = ON
	 --Rmatrix   = OFF
	 --nanString = NA
Number of lines in source file is 10
Number of words in source file is 20
skiprows = 0
cnrow = 0
skipcols = 5
rncol = 2
Rmatrix = 0
numWords = 20
Creating file with numRows = 10
Creating file with numColumns = 15
text2fvf finished.
File 'TEST10x15_T.geno.dose' already exists.
ERROR in Rstuff:failed in ini_empty_FileMatrix_RError in !result : invalid argument type
Calls: impute2databel -> apply2dfo -> make_empty_fvf
In addition: Warning message:
In uninames(.Object at data) :
  uninames: some column names are not unique; use set_dimnames/get_dimnames for non-unique row/col names
Execution halted


> 
> It works on my Ubuntu. If it works on your Ubuntu, win and mac, then we can delete from GenABEL the simlinks to flv and databel.
> 
> best,
> Максим
> 
> 
> On 22/11/2013 21:34, L.C. Karssen wrote:
>> Hi максим,
>> 
>> 
>> On 11/19/2013 03:17 PM, Maksim Struchalin wrote:
>>> Hi Lennart,
>>> 
>>> I see you are improving your Russian :-).
>> Getting to know the Russian alphabet is step one :-).
>> 
>>> I understand your arguments. I think we can combine our two approaches.
>>> 1) We make a so/dll from filevector and let it use by
>>> ProbABEL/OmicABEL/Another_not_R_softABEL.
>>> 2) For GenABEL and other R packages, we make a DatABEL.
>>> 
>>> The code of filevector is the same both for 1) and 2). 
>> But that doesn't solve the problem of having symlinks to the fvlib
>> directory in our SVN tree... Which means that any update to filevector
>> can make the depending package (DatABEL) become uncompilable.
>> 
>> In the mean time I've set the first steps towards 'libfilevector' in
>> SVN, see commits 1415 and 1416. This works (at least for ProbABEL), but
>> more polishing is needed.
>> 
>> 
>>> We only add
>>> preprocessor commands (#ifdef and so on) to surround R specific code
>>> (ISNAN() and std::isnan). In this case, compiler choose itself weather
>>> it buids the lib for R or for OS.
>>> 
>>> If we will want to use only approach 1) for GenABEL in the future, we
>>> can quickly swith to it later.
>> True, for now this will work.
>> 
>> 
>> Best,
>> 
>> Lennart.
>> 
>>> best,
>>> Maksim
>>> 
>>> 
>>> 
>>> On 19/11/2013 16:10, L.C. Karssen wrote:
>>>> Hi максим,
>>>> 
>>>> (trying a Russian keyboard layout, no idea if this works...).
>>>> 
>>>> On 19-11-13 09:44, Maksim Struchalin wrote:
>>>>> It seems that your solution is workable but I see little difference with
>>>>> what it is now. Now the filevector code is incorporated in each
>>>>> packages. 
>>>> This is what I would like to change, indeed. Code that is reused by so
>>>> many packages should not be copied/symlinked into the code tree of those
>>>> packages. By symlinking it as we have now, there is no proper way of
>>>> specifying a version number of the filevector code. Which, in turn means
>>>> that if something changes in the filevector code all other packages need
>>>> to be changed immediately (just like what happened with your latest
>>>> change). If the filevector code have been a proper library we could have
>>>> simply said that ProbABEL still depends on the old filevector version
>>>> and take more time to make sure the two play nice together.
>>>> 
>>>> Moreover, with the filevector code in a separate library the whole
>>>> isnan() issue would not be a problem. We could simply use std::isnan(),
>>>> because CRAN wouldn't need to compile the .so/.dll, so no need of ISNAN().
>>>> When code is put in a library the internals don't matter as long as the
>>>> interface (function names + arguments) to the outside doesn't change.
>>>> 
>>>>> You propose to follow the same way but pack filelvector code
>>>>> in one file (dll or so) and distribute 9 packages form GenABEL with the
>>>>> same library.
>>>> Indeed. The problem with incorporating it all in DatABEL is that non-R
>>>> packages like ProbABEL and OmicABEL depend on the stuff in the fvlib
>>>> directory as well. Filevector is central to (almost) all packages in the
>>>> GenABEL suite, which is why I proposed to make a library out of it. And,
>>>> as noted above, this way packages can depend on different version of the
>>>> library.
>>>> 
>>>> We can of course discuss whether we want to distribute this .so/.dll as
>>>> a separate (operating system) package or withing the R packages. To me
>>>> the first option is the 'correct' one, but I see that this may impose on
>>>> the user (except on Windows and maybe MacOS, where the .so/.dll is
>>>> included in the R package).
>>>> 
>>>> 
>>>>> Last time I proposed to move filevector in DatABEL. All other packages
>>>>> (GenA and so on) will load DatAB in R and use filevector fucntions from
>>>>> DatA. When DatABEL is loaded through library(DatABEL), the file
>>>>> DatABEL.so is loaded as well.
>>>> I think this is what should be done with the DAlib directory (another
>>>> symlinked dir).
>>>> 
>>>>>  Thus, you do not need to ask users to
>>>>> install additional lib because it is in DatABEL already. I think this is
>>>>> a workable approach that will allow us to delete the filevector code (or
>>>>> filevector so/dll) from all the packages.
>>>>> 
>>>>> 
>>>>> This is some quote from the R manual how to register functions to make
>>>>> it available from DatAB to GenAB:
>>>>> 
>>>>> 
>>>>>       _______________________________________________
>>>>> 
>>>>> 
>>>>>       5.4 Registering native routines
>>>>> 
>>>>> By ‘native’ routine, we mean an entry point in compiled code.
>>>>> 
>>>>> In calls to |.C|, |.Call|, |.Fortran| and |.External|, R must locate the
>>>>> specified native routine by looking in the appropriate shared
>>>>> object/DLL. By default, R uses the operating system-specific dynamic
>>>>> loader to lookup the symbol in all loaded DLLs and elsewhere.
>>>>> Alternatively, the author of the DLL can explicitly register routines
>>>>> with R and use a single, platform-independent mechanism for finding the
>>>>> routines in the DLL. One can use this registration mechanism to provide
>>>>> additional information about a routine, including the number and type of
>>>>> the arguments, and also make it available to R programmers under a
>>>>> different name. In the future, registration may be used to implement a
>>>>> form of “secure” or limited native access.
>>>>> 
>>>>> _____________________________________________________
>>>>> 
>>>> Hmm, I will have to think about this. This seems to be about how R finds
>>>> out in which DLL a function is found (and maybe where the DLL is found
>>>> in the filesystem). I think this is separate from the point below, but
>>>> I'm not sure.
>>>> 
>>>>> Your argument was from "5.8 Linking to other packages: It is not in
>>>>> general possible to link a DLL in package *packA* to a DLL provided by
>>>>> package *packB". *I do not quite understand what they mean under 'link'.
>>>>> May be the mean link a library during intsalltion?
>>>> Yes, as far as I understand, they mean linking to a library during
>>>> installation/compilation.
>>>> 
>>>> 
>>>> Best,
>>>> 
>>>> Lennart.
>>>>> best,
>>>>> Maksim
>>>>> 
>>>>> 
>>>>> 
>>>>> On 19/11/2013 15:14, L.C. Karssen wrote:
>>>>>> Hi Maksim,
>>>>>> 
>>>>>> Good question... The idea is to generate a .dll file for Windows, but
>>>>>> I'm not sure what would be the best way to distribute that. It would be
>>>>>> interesting to see how other packages do that. For example, the XML
>>>>>> package depends on libxml2:
>>>>>> http://cran.r-project.org/web/packages/XML/index.html and the Rcurl
>>>>>> package depends on libcurl:
>>>>>> http://cran.r-project.org/web/packages/RCurl/index.html
>>>>>> 
>>>>>> In the XML package .zip file for Windows there is a directory libs/x64
>>>>>> and a directory libs/i386. Both contain XML.dll, so I think that for
>>>>>> Linux you simply specify a dependency on a library, whereas for Windows
>>>>>> the actual .dll is in the package (which is quite logical because
>>>>>> Windows lacks the package repositories that most Linux distros have).
>>>>>> It seems that for MacOS the .tgz file also contains a lib directory with
>>>>>> the .so file.
>>>>>> 
>>>>>> 
>>>>>> Best regards,
>>>>>> 
>>>>>> Lennart.
>>>>>> 
>>>>>> On 19-11-13 08:56, Maksim Struchalin wrote:
>>>>>>> Hi Lennart,
>>>>>>> 
>>>>>>> How the users under win will install such a library?
>>>>>>> 
>>>>>>> best,
>>>>>>> Maksim
>>>>>>> 
>>>>>>> On 19/11/2013 14:46, L.C. Karssen wrote:
>>>>>>>> Dear all,
>>>>>>>> 
>>>>>>>> The Jenkins setup already shows its value: After Maksim changed the call
>>>>>>>> from std::isnan() to ISNAN() in fvlib's CastUtils.cpp an automatic build
>>>>>>>> of ProbABEL was triggered and it failed (because ISNAN() is an R function).
>>>>>>>> 
>>>>>>>> I guess this is one more reason to try to convert fvlib into a real
>>>>>>>> (shared) library.
>>>>>>>> Does anyone have another workable solution?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Lennart.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> genabel-devel mailing list
>>>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>>> _______________________________________________
>>>>>>> genabel-devel mailing list
>>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>>> 
>>>>>> _______________________________________________
>>>>>> genabel-devel mailing list
>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>> 
>>>>> _______________________________________________
>>>>> genabel-devel mailing list
>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> genabel-devel mailing list
>>>> genabel-devel at lists.r-forge.r-project.org
>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>> 
>>> 
>>> _______________________________________________
>>> genabel-devel mailing list
>>> genabel-devel at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>> 
>> 
>> 
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
> 
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20131125/460979f9/attachment-0001.html>

More information about the genabel-devel mailing list