[NMF-user] Fwd: matrix size limit?

[Renaud Gaujoux]

---------- Forwarded message ----------
From: Thomas Tibbitts <Thomas.Tibbitts at infi.com>
Date: 23 May 2014 22:28
Subject: RE: [NMF-user] matrix size limit?
To: Renaud Gaujoux <renaud at cbio.uct.ac.za>


Hi Renaud,

Thank you for your quick response.

This test returns TRUE:
min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0
[1] TRUE

Then I received the following output with the next two commands:
s <- rowSums(x, na.rm = TRUE)
str(which(is.na(s) | s==0))
 Named int [1:1691] 1 2 3 4 5 6 7 8 9 10 ...
 - attr(*, "names")= chr [1:1691]
"ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48"|
__truncated__
"ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__
"ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0"|
__truncated__
"ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"|
__truncated__ ...

After the header row, the first few lines of the data in my file are:
ABCA13_11152_A2223P,0,0,0,0,0,0,0.46,0.5,0.53,0.51,0,0,0,0,0,0,0,0,0.48,0.45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.46,0,0,0,0,0.48,0.48,0,0,0,0,0.47,0.53,0,0,0,0,0,0,0,0,0,0,0,0,0,0.56,0,0,0.5,0.48,0,0,0,0,0,0,0,0,0.48,0.46,0.45,0.47,0,0,0,0,0,0,0,0,0,0,0,0,0,0
ABCA13_11157_K2306T,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.5,0,0,0,0,0,0,0,0,0,0,0.54,0.48,0,0,0,0,0,0,0,0
ABCA13_11161_S2537A,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.48,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
ABCA13_11163_D2630G,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.47,0.48,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

So the error message did not help. Do you have any other ideas?
I am not sure how to turn on logging in R to get a complete log.

Thank you again for your help!
Best regards,
Tom

-----Original Message-----
From: getoxxx at gmail.com [mailto:getoxxx at gmail.com] On Behalf Of Renaud
Gaujoux
Sent: Monday, May 19, 2014 4:58 PM
To: Thomas Tibbitts
Cc: nmf-user at lists.r-forge.r-project.org
Subject: Re: [NMF-user] matrix size limit?

Hi Thomas,

this is not really a large matrix and should not pose a problem, for
example on a toy dataset:

> x <- rmatrix(4000, 100)
> res <- nmf(x, 3)
> runtime(res)
   user  system elapsed
 19.888   0.004  19.899
> res <- nmf(x, 3, nrun = 3)
> runtime(res)
   user  system elapsed
 33.262   0.004  33.447
>

Do you see warnings as well? Sending a complete log would help.
The error you see is thrown if the following test is TRUE:

min(rowSums(x, na.rm = TRUE), na.rm = TRUE) == 0

Can you please directly run this test on your data? It should return TRUE.
Then look at:

s <- rowSums(x, na.rm = TRUE)
str(which( is.na(s) | s == 0 ))

which will give you a hint on which row(s) are detected as problematic. By
the way, is your dataset nonnegative?

Bests,
Renaud

On 19 May 2014 18:37, Thomas Tibbitts <Thomas.Tibbitts at infi.com> wrote:
> NMF-users,
>
>
>
> I can run NMF with a matrix of 1646 rows x 100 columns without error.
>
>
>
> However when I combine data sets into a larger matrix of 3495 rows x
> 100 columns I keep getting the error:
>
> Error:  NMF::nmf  – Input matrix x contains at least one null or
> NA-filled row.
>
>
>
> There are no nulls or NA-filled rows in the larger matrix, I wrote it
> out to disk to make sure.
>
>
>
> So I ‘m thinking I hit an matrix size limitation.
>
> How can I fix this problem?
>
>
>
> Thank you.
>
> Tom
>
> This message and any attachments may contain confidential information
> and/or privileged material. If you received this message in error,
> please contact the sender and delete this message and any attachments
> immediately. Thank you.
>
> _______________________________________________
> nmf-user mailing list
> nmf-user at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/nmf-user
>
>



--
Renaud Gaujoux, PhD
Systems Immunology - Technion, Haifa, Israel
This message and any attachments may contain confidential information
and/or privileged material. If you received this message in error, please
contact the sender and delete this message and any attachments immediately.
Thank you.



-- 
Renaud Gaujoux, PhD
Systems Immunology - Technion, Haifa, Israel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/nmf-user/attachments/20140612/21acf1cd/attachment.html>