[datatable-help] Error in coercing matrices within j expressions

[Nathaniel Graham]

I'm currently using a (moderately) complex function, call
if f(), as a j expression to analyze my data.  The data itself
is about 1.2M rows, which I analyze by group.
A group may have as few as one row or as many as 10K.
The output from the function is a two-column data.table
where the rows are interesting (for my work) pairs of
observations--I have no idea how many pairs will be
interesting until the function runs, but in abstract it could
be every unique combination (so as many as 50M rows
of output for one call to f()).  It is common, and not an
error, for groups to have no meaningful pairs to return.

I've been using the following line to create the output for
f():

indices <- data.table(i = integer(), j = integer())

I then append to 'indices' any useful pairs using:

indices <- rbind(indices, list(idx[i], idx[j]))

This works, but is very, very slow, in part because I'm
using rbind().  I want to switch to using the built-in matrix,
because rbind() should be much faster for them.  Using
the following line to create the matrix:

indices <- matrix(nrow = 0, ncol = 2, dimnames = list(c(NULL),c("i","j")))

results in the following error:

Logical error. Type of column should have been checked by now

Note that the values returned are always integers.  Results are
coerced via:

data.table(indices)

before returning from f().  If I don't explicitly coerce, I get the
following error:

j doesn't evaluate to the same number of columns for each group

If someone could tell me what I'm doing wrong, or some other
equivalent way to noticeably speed up the whole process, I'd
be very grateful.


-------
Nathaniel Graham
npgraham1 at gmail.com
npgraham1 at uky.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20130917/790f2f87/attachment.html>