[NMF-user] Fwd: matrix size limit?

[Renaud Gaujoux]

---------- Forwarded message ----------
From: Renaud Gaujoux <renaud at cbio.uct.ac.za>
Date: 26 May 2014 10:24
Subject: Re: [NMF-user] matrix size limit?
To: Thomas Tibbitts <Thomas.Tibbitts at infi.com>


Are you sure you are reading the data correctly? The names seem strange.


On 23 May 2014 22:28, Thomas Tibbitts <Thomas.Tibbitts at infi.com> wrote:
> Hi Renaud,
>
> Thank you for your quick response.
>
> This test returns TRUE:
> min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0
> [1] TRUE
>
> Then I received the following output with the next two commands:
> s <- rowSums(x, na.rm = TRUE)
> str(which(is.na(s) | s==0))
>  Named int [1:1691] 1 2 3 4 5 6 7 8 9 10 ...
>  - attr(*, "names")= chr [1:1691]
"ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48"|
__truncated__
"ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__
"ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0"|
__truncated__
"ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__ ...
>
> After the header row, the first few lines of the data in my file are:
>
ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48,0.48,0,0,0,0,0.47,0.53,0,0,0,0,0,0,0,0,0,0,0,0,0,0.56,0,0,0.5,0.48,0,0,0,0,0,0,0,0,0.48,0.46,0.45,0.47,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>
ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.5,0,0,0,0,0,0,0,0,0,0,0.54,0.48,0,0,0,0,0,0,0,0
>
ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>
ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>
> So the error message did not help. Do you have any other ideas?
> I am not sure how to turn on logging in R to get a complete log.
>
> Thank you again for your help!
> Best regards,
> Tom
>
> -----Original Message-----
> From: getoxxx at gmail.com [mailto:getoxxx at gmail.com] On Behalf Of Renaud
Gaujoux
> Sent: Monday, May 19, 2014 4:58 PM
> To: Thomas Tibbitts
> Cc: nmf-user at lists.r-forge.r-project.org
> Subject: Re: [NMF-user] matrix size limit?
>
> Hi Thomas,
>
> this is not really a large matrix and should not pose a problem, for
example on a toy dataset:
>
>> x <- rmatrix(4000, 100)
>> res <- nmf(x, 3)
>> runtime(res)
>    user  system elapsed
>  19.888   0.004  19.899
>> res <- nmf(x, 3, nrun = 3)
>> runtime(res)
>    user  system elapsed
>  33.262   0.004  33.447
>>
>
> Do you see warnings as well? Sending a complete log would help.
> The error you see is thrown if the following test is TRUE:
>
> min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0
>
> Can you please directly run this test on your data? It should return
TRUE. Then look at:
>
> s <- rowSums(x, na.rm = TRUE)
> str(which( is.na(s) | s == 0 ))
>
> which will give you a hint on which row(s) are detected as problematic.
By the way, is your dataset nonnegative?
>
> Bests,
> Renaud
>
> On 19 May 2014 18:37, Thomas Tibbitts <Thomas.Tibbitts at infi.com> wrote:
>> NMF-users,
>>
>>
>>
>> I can run NMF with a matrix of 1646 rows x 100 columns without error.
>>
>>
>>
>> However when I combine data sets into a larger matrix of 3495 rows x
>> 100 columns I keep getting the error:
>>
>> Error:  NMF::nmf  – Input matrix x contains at least one null or
>> NA-filled row.
>>
>>
>>
>> There are no nulls or NA-filled rows in the larger matrix, I wrote it
>> out to disk to make sure.
>>
>>
>>
>> So I ‘m thinking I hit an matrix size limitation.
>>
>> How can I fix this problem?
>>
>>
>>
>> Thank you.
>>
>> Tom
>>
>> This message and any attachments may contain confidential information
>> and/or privileged material. If you received this message in error,
>> please contact the sender and delete this message and any attachments
>> immediately. Thank you.
>>
>> _______________________________________________
>> nmf-user mailing list
>> nmf-user at lists.r-forge.r-project.org
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/nmf-user
>>
>>
>
>
>
> --
> Renaud Gaujoux, PhD
> Systems Immunology - Technion, Haifa, Israel
> This message and any attachments may contain confidential information
and/or privileged material. If you received this message in error, please
contact the sender and delete this message and any attachments immediately.
Thank you.



--
Renaud Gaujoux, PhD
Systems Immunology - Technion, Haifa, Israel



-- 
Renaud Gaujoux, PhD
Systems Immunology - Technion, Haifa, Israel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/nmf-user/attachments/20140612/463b1d47/attachment.html>