[GenABEL-dev] ISNAN problem in filevector found by Jenkins
L.C. Karssen
lennart at karssen.org
Mon Nov 25 11:01:10 CET 2013
Hi Maksim,
I've run the tests you described below on my Ubuntu installation and it
works. I only changed the name of the output files so that I could
compare them with the ones you created. The .fvd files are the same, but
the .fvi files are not. Maybe the .fvi file contains a date/time?
$ md5sum *dose.fv?
5de29707c2b2a09aa7e07569f35a565d TEST10x15_T.geno.dose.fvd
91c2b36a71bdfabe5a568d1c1c90a308 TEST10x15_T.geno.dose.fvi
5de29707c2b2a09aa7e07569f35a565d TEST10x15_T.geno.lck.dose.fvd
e811a8c5184bd2c7cb64b1b3bdde7827 TEST10x15_T.geno.lck.dose.fvi
Let me know if there is something else I can test.
Best,
Lennart.
On 11/23/2013 03:12 PM, Maksim Struchalin wrote:
> Hi Lennart,
>
> https://app.box.com/s/iy41ug5qg4sbylul9oyn
>
> This is an example demonstrating how a GenABEL function "impute2databel"
> calls a function "iteratorDA" from DatABEL. Here, GenABEL is compiled
> without flv's and iterator's code (I deleted it from src).
>
> Could you run the test?:
> 0) Dowload the file test_GenABEL_iterator.tar.gz from
> https://app.box.com/s/iy41ug5qg4sbylul9oyn
> 1) decompress test_GenABEL_iterator.tar.gz
> 2) cd test_GenABEL_iterator
> 3) R CMD INSTALL DatABEL_0.9-4.tar.gz
> 4) R CMD INSTALL GenABEL_1.7-7.tar.gz
> 5) run test.R
>
> It works on my Ubuntu. If it works on your Ubuntu, win and mac, then we
> can delete from GenABEL the simlinks to flv and databel.
>
> best,
> Максим
>
>
> On 22/11/2013 21:34, L.C. Karssen wrote:
>> Hi максим,
>>
>>
>> On 11/19/2013 03:17 PM, Maksim Struchalin wrote:
>>> Hi Lennart,
>>>
>>> I see you are improving your Russian :-).
>> Getting to know the Russian alphabet is step one :-).
>>
>>> I understand your arguments. I think we can combine our two approaches.
>>> 1) We make a so/dll from filevector and let it use by
>>> ProbABEL/OmicABEL/Another_not_R_softABEL.
>>> 2) For GenABEL and other R packages, we make a DatABEL.
>>>
>>> The code of filevector is the same both for 1) and 2).
>> But that doesn't solve the problem of having symlinks to the fvlib
>> directory in our SVN tree... Which means that any update to filevector
>> can make the depending package (DatABEL) become uncompilable.
>>
>> In the mean time I've set the first steps towards 'libfilevector' in
>> SVN, see commits 1415 and 1416. This works (at least for ProbABEL), but
>> more polishing is needed.
>>
>>
>>> We only add
>>> preprocessor commands (#ifdef and so on) to surround R specific code
>>> (ISNAN() and std::isnan). In this case, compiler choose itself weather
>>> it buids the lib for R or for OS.
>>>
>>> If we will want to use only approach 1) for GenABEL in the future, we
>>> can quickly swith to it later.
>> True, for now this will work.
>>
>>
>> Best,
>>
>> Lennart.
>>
>>> best,
>>> Maksim
>>>
>>>
>>>
>>> On 19/11/2013 16:10, L.C. Karssen wrote:
>>>> Hi максим,
>>>>
>>>> (trying a Russian keyboard layout, no idea if this works...).
>>>>
>>>> On 19-11-13 09:44, Maksim Struchalin wrote:
>>>>> It seems that your solution is workable but I see little difference with
>>>>> what it is now. Now the filevector code is incorporated in each
>>>>> packages.
>>>> This is what I would like to change, indeed. Code that is reused by so
>>>> many packages should not be copied/symlinked into the code tree of those
>>>> packages. By symlinking it as we have now, there is no proper way of
>>>> specifying a version number of the filevector code. Which, in turn means
>>>> that if something changes in the filevector code all other packages need
>>>> to be changed immediately (just like what happened with your latest
>>>> change). If the filevector code have been a proper library we could have
>>>> simply said that ProbABEL still depends on the old filevector version
>>>> and take more time to make sure the two play nice together.
>>>>
>>>> Moreover, with the filevector code in a separate library the whole
>>>> isnan() issue would not be a problem. We could simply use std::isnan(),
>>>> because CRAN wouldn't need to compile the .so/.dll, so no need of ISNAN().
>>>> When code is put in a library the internals don't matter as long as the
>>>> interface (function names + arguments) to the outside doesn't change.
>>>>
>>>>> You propose to follow the same way but pack filelvector code
>>>>> in one file (dll or so) and distribute 9 packages form GenABEL with the
>>>>> same library.
>>>> Indeed. The problem with incorporating it all in DatABEL is that non-R
>>>> packages like ProbABEL and OmicABEL depend on the stuff in the fvlib
>>>> directory as well. Filevector is central to (almost) all packages in the
>>>> GenABEL suite, which is why I proposed to make a library out of it. And,
>>>> as noted above, this way packages can depend on different version of the
>>>> library.
>>>>
>>>> We can of course discuss whether we want to distribute this .so/.dll as
>>>> a separate (operating system) package or withing the R packages. To me
>>>> the first option is the 'correct' one, but I see that this may impose on
>>>> the user (except on Windows and maybe MacOS, where the .so/.dll is
>>>> included in the R package).
>>>>
>>>>
>>>>> Last time I proposed to move filevector in DatABEL. All other packages
>>>>> (GenA and so on) will load DatAB in R and use filevector fucntions from
>>>>> DatA. When DatABEL is loaded through library(DatABEL), the file
>>>>> DatABEL.so is loaded as well.
>>>> I think this is what should be done with the DAlib directory (another
>>>> symlinked dir).
>>>>
>>>>> Thus, you do not need to ask users to
>>>>> install additional lib because it is in DatABEL already. I think this is
>>>>> a workable approach that will allow us to delete the filevector code (or
>>>>> filevector so/dll) from all the packages.
>>>>>
>>>>>
>>>>> This is some quote from the R manual how to register functions to make
>>>>> it available from DatAB to GenAB:
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>>
>>>>> 5.4 Registering native routines
>>>>>
>>>>> By ‘native’ routine, we mean an entry point in compiled code.
>>>>>
>>>>> In calls to |.C|, |.Call|, |.Fortran| and |.External|, R must locate the
>>>>> specified native routine by looking in the appropriate shared
>>>>> object/DLL. By default, R uses the operating system-specific dynamic
>>>>> loader to lookup the symbol in all loaded DLLs and elsewhere.
>>>>> Alternatively, the author of the DLL can explicitly register routines
>>>>> with R and use a single, platform-independent mechanism for finding the
>>>>> routines in the DLL. One can use this registration mechanism to provide
>>>>> additional information about a routine, including the number and type of
>>>>> the arguments, and also make it available to R programmers under a
>>>>> different name. In the future, registration may be used to implement a
>>>>> form of “secure” or limited native access.
>>>>>
>>>>> _____________________________________________________
>>>>>
>>>> Hmm, I will have to think about this. This seems to be about how R finds
>>>> out in which DLL a function is found (and maybe where the DLL is found
>>>> in the filesystem). I think this is separate from the point below, but
>>>> I'm not sure.
>>>>
>>>>> Your argument was from "5.8 Linking to other packages: It is not in
>>>>> general possible to link a DLL in package *packA* to a DLL provided by
>>>>> package *packB". *I do not quite understand what they mean under 'link'.
>>>>> May be the mean link a library during intsalltion?
>>>> Yes, as far as I understand, they mean linking to a library during
>>>> installation/compilation.
>>>>
>>>>
>>>> Best,
>>>>
>>>> Lennart.
>>>>> best,
>>>>> Maksim
>>>>>
>>>>>
>>>>>
>>>>> On 19/11/2013 15:14, L.C. Karssen wrote:
>>>>>> Hi Maksim,
>>>>>>
>>>>>> Good question... The idea is to generate a .dll file for Windows, but
>>>>>> I'm not sure what would be the best way to distribute that. It would be
>>>>>> interesting to see how other packages do that. For example, the XML
>>>>>> package depends on libxml2:
>>>>>> http://cran.r-project.org/web/packages/XML/index.html and the Rcurl
>>>>>> package depends on libcurl:
>>>>>> http://cran.r-project.org/web/packages/RCurl/index.html
>>>>>>
>>>>>> In the XML package .zip file for Windows there is a directory libs/x64
>>>>>> and a directory libs/i386. Both contain XML.dll, so I think that for
>>>>>> Linux you simply specify a dependency on a library, whereas for Windows
>>>>>> the actual .dll is in the package (which is quite logical because
>>>>>> Windows lacks the package repositories that most Linux distros have).
>>>>>> It seems that for MacOS the .tgz file also contains a lib directory with
>>>>>> the .so file.
>>>>>>
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Lennart.
>>>>>>
>>>>>> On 19-11-13 08:56, Maksim Struchalin wrote:
>>>>>>> Hi Lennart,
>>>>>>>
>>>>>>> How the users under win will install such a library?
>>>>>>>
>>>>>>> best,
>>>>>>> Maksim
>>>>>>>
>>>>>>> On 19/11/2013 14:46, L.C. Karssen wrote:
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> The Jenkins setup already shows its value: After Maksim changed the call
>>>>>>>> from std::isnan() to ISNAN() in fvlib's CastUtils.cpp an automatic build
>>>>>>>> of ProbABEL was triggered and it failed (because ISNAN() is an R function).
>>>>>>>>
>>>>>>>> I guess this is one more reason to try to convert fvlib into a real
>>>>>>>> (shared) library.
>>>>>>>> Does anyone have another workable solution?
>>>>>>>>
>>>>>>>>
>>>>>>>> Lennart.
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> genabel-devel mailing list
>>>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>>> _______________________________________________
>>>>>>> genabel-devel mailing list
>>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>>>
>>>>>> _______________________________________________
>>>>>> genabel-devel mailing list
>>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>
>>>>> _______________________________________________
>>>>> genabel-devel mailing list
>>>>> genabel-devel at lists.r-forge.r-project.org
>>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>>>
>>>>
>>>> _______________________________________________
>>>> genabel-devel mailing list
>>>> genabel-devel at lists.r-forge.r-project.org
>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>
>>>
>>> _______________________________________________
>>> genabel-devel mailing list
>>> genabel-devel at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>>>
>>
>>
>> _______________________________________________
>> genabel-devel mailing list
>> genabel-devel at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>
>
>
> _______________________________________________
> genabel-devel mailing list
> genabel-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/genabel-devel
>
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
Utrecht
The Netherlands
lennart at karssen.org
http://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 230 bytes
Desc: OpenPGP digital signature
URL: <http://lists.r-forge.r-project.org/pipermail/genabel-devel/attachments/20131125/22153d9a/attachment.sig>
More information about the genabel-devel
mailing list