[datatable-help] Using data.table code in an interactive desktop application written in C# or Java

[Nicolas Paris]

Hi,

When manipulating huge datasets, the bottleneck is certainly not the
viz tool (shiny, c#, html5 etc).
This won't be data-manipulation part (you won't find faster than
data.table with c# manipulation since most part of data.table is C based).
The slow part of your tool is probably the data-loading phase.

Open-CPU allows to cache data when the opencpu services is starting.
This means each user querying your dataset won't have to wait until the
dataset is loaded into open-cpu memory. see
http://stackoverflow.com/questions/12468574/are-data-objects-persistent-in-opencpus-r-session
Rserver and other stuff are quite old comparing to open-cpu. And I would
go with them.

Keep separate your view (html5, c#), backend (opencpu/R) and database
(opencpu/data.table). And one day you can moove to more flexible
architecture such backend (java/c#) and  database (distributed such hive) if
your dataset cannot feet into RAM memory.

But the opencpu way allows you to fast deploy something compatible with
your 10K lines of data.table/R code

Le 16 avril 2017 à 13:22, Don Brady écrivait :
> Thanks you very much for the comments.  Those are some good ways to get he data
> back and forth.  In fact, I think we can regard those mechanical issues as
> taken care of.  I can see a number of ways to do that.
> 
> However, once I get the columnar data (what was data.tables in R) into C# and
> stored in memory, what representation of it should I use in C# that would be
> most like data.table?   And how would I manipulate it then and there in C#, but
> in data.table like ways, without involving R at that point.  Does anyone know
> what .Net way would be most like data.table?  Would it be LINQ Objects, for
> example, or some in-memory data package?   I do not want to do SQL etc. which
> is so painful compared to data.table.  I know this is not a data.table question
> as such, but people here are the only people who can understand it because they
> know what data.table is.  I want to be able to add one column to another etc.
> as I can do in data.table.  (Then maybe eventually send it back to R as a
> data.table which is just mechanics again).
> 
> If I am too far off topic let me know and I apologize.
> 
> Thanks,
> 
> Don

> _______________________________________________
> datatable-help mailing list
> datatable-help at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help