[NMF-user] Fwd: matrix size limit? [SOLVED]

[Renaud Gaujoux]

---------- Forwarded message ----------
From: Thomas Tibbitts <Thomas.Tibbitts at infi.com>
Date: 11 June 2014 21:16
Subject: Re: [NMF-user] matrix size limit?
To: Renaud Gaujoux <renaud at cbio.uct.ac.za>


The problem was rows of all zeros in the input matrix.
Thank you for your help.
Best regards,
Tom

Sent from my iPhone

> On May 26, 2014, at 4:25 AM, "Renaud Gaujoux" <renaud at cbio.uct.ac.za>
wrote:
>
> Are you sure you are reading the data correctly? The names seem strange.
>
>
>> On 23 May 2014 22:28, Thomas Tibbitts <Thomas.Tibbitts at infi.com> wrote:
>> Hi Renaud,
>>
>> Thank you for your quick response.
>>
>> This test returns TRUE:
>> min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0
>> [1] TRUE
>>
>> Then I received the following output with the next two commands:
>> s <- rowSums(x, na.rm = TRUE)
>> str(which(is.na(s) | s==0))
>> Named int [1:1691] 1 2 3 4 5 6 7 8 9 10 ...
>> - attr(*, "names")= chr [1:1691]
"ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48"|
__truncated__
"ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__
"ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0"|
__truncated__
"ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__ ...
>>
>> After the header row, the first few lines of the data in my file are:
>>
ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48,0.48,0,0,0,0,0.47,0.53,0,0,0,0,0,0,0,0,0,0,0,0,0,0.56,0,0,0.5,0.48,0,0,0,0,0,0,0,0,0.48,0.46,0.45,0.47,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>>
ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.5,0,0,0,0,0,0,0,0,0,0,0.54,0.48,0,0,0,0,0,0,0,0
>>
ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>>
ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>>
>> So the error message did not help. Do you have any other ideas?
>> I am not sure how to turn on logging in R to get a complete log.
>>
>> Thank you again for your help!
>> Best regards,
>> Tom
>>
>> -----Original Message-----
>> From: getoxxx at gmail.com [mailto:getoxxx at gmail.com] On Behalf Of Renaud
Gaujoux
>> Sent: Monday, May 19, 2014 4:58 PM
>> To: Thomas Tibbitts
>> Cc: nmf-user at lists.r-forge.r-project.org
>> Subject: Re: [NMF-user] matrix size limit?
>>
>> Hi Thomas,
>>
>> this is not really a large matrix and should not pose a problem, for
example on a toy dataset:
>>
>>> x <- rmatrix(4000, 100)
>>> res <- nmf(x, 3)
>>> runtime(res)
>>   user  system elapsed
>> 19.888   0.004  19.899
>>> res <- nmf(x, 3, nrun = 3)
>>> runtime(res)
>>   user  system elapsed
>> 33.262   0.004  33.447
>>
>> Do you see warnings as well? Sending a complete log would help.
>> The error you see is thrown if the following test is TRUE:
>>
>> min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0
>>
>> Can you please directly run this test on your data? It should return
TRUE. Then look at:
>>
>> s <- rowSums(x, na.rm = TRUE)
>> str(which( is.na(s) | s == 0 ))
>>
>> which will give you a hint on which row(s) are detected as problematic.
By the way, is your dataset nonnegative?
>>
>> Bests,
>> Renaud
>>
>>> On 19 May 2014 18:37, Thomas Tibbitts <Thomas.Tibbitts at infi.com> wrote:
>>> NMF-users,
>>>
>>>
>>>
>>> I can run NMF with a matrix of 1646 rows x 100 columns without error.
>>>
>>>
>>>
>>> However when I combine data sets into a larger matrix of 3495 rows x
>>> 100 columns I keep getting the error:
>>>
>>> Error:  NMF::nmf  – Input matrix x contains at least one null or
>>> NA-filled row.
>>>
>>>
>>>
>>> There are no nulls or NA-filled rows in the larger matrix, I wrote it
>>> out to disk to make sure.
>>>
>>>
>>>
>>> So I ‘m thinking I hit an matrix size limitation.
>>>
>>> How can I fix this problem?
>>>
>>>
>>>
>>> Thank you.
>>>
>>> Tom
>>>
>>> This message and any attachments may contain confidential information
>>> and/or privileged material. If you received this message in error,
>>> please contact the sender and delete this message and any attachments
>>> immediately. Thank you.
>>>
>>> _______________________________________________
>>> nmf-user mailing list
>>> nmf-user at lists.r-forge.r-project.org
>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/nmf-user
>>
>>
>>
>> --
>> Renaud Gaujoux, PhD
>> Systems Immunology - Technion, Haifa, Israel
>> This message and any attachments may contain confidential information
and/or privileged material. If you received this message in error, please
contact the sender and delete this message and any attachments immediately.
Thank you.
>
>
>
> --
> Renaud Gaujoux, PhD
> Systems Immunology - Technion, Haifa, Israel
This message and any attachments may contain confidential information
and/or privileged material. If you received this message in error, please
contact the sender and delete this message and any attachments immediately.
Thank you.




-- 
Renaud Gaujoux, PhD
Systems Immunology - Technion, Haifa, Israel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/nmf-user/attachments/20140612/580ee906/attachment-0001.html>