<div>I don't do anything that mixes R and python so I can't comment on using the two interchangeably.</div>
<div> </div>
<div>There are some api's that I've played around with that seem okay, but they probably buckle under a reasonable amount of preasure.<br><br></div>
<div class="gmail_quote">On Tue, Dec 7, 2010 at 10:47 AM, Santosh Srinivas <span dir="ltr"><<a href="mailto:santosh.srinivas@gmail.com">santosh.srinivas@gmail.com</a>></span> wrote:<br>
<blockquote style="BORDER-LEFT: #ccc 1px solid; MARGIN: 0px 0px 0px 0.8ex; PADDING-LEFT: 1ex" class="gmail_quote">First ... thanks for data.table .... even I (with average R skills) am<br>able to do some cool stuff on large datasets and FAST!.<br>
<br>I've heard that integrability with R is quite decent?<br>Python is next on my list but I hope my experience will be complementary<br>
<div>
<div></div>
<div class="h5"><br><br>On Tue, Dec 7, 2010 at 10:10 PM, Rob Forler <<a href="mailto:rforler@uchicago.edu">rforler@uchicago.edu</a>> wrote:<br>> It's not a hard language to pick up, but definitely one of the challenges<br>
> with replacing R with python is many of the stat and data packages that<br>> exist in R.<br>> Python has strong scientific and data packages, but I not at the same level.<br>><br>> On the other hand python is much stronger from a language point of view<br>
> (good oo etc), and has many more operational tools.<br>><br>> -Rob<br>><br>> On Tue, Dec 7, 2010 at 8:30 AM, Matthew Dowle <<a href="mailto:mdowle@mdowle.plus.com">mdowle@mdowle.plus.com</a>><br>> wrote:<br>
>><br>>> Interesting. I don't know python but it's been on the radar a few times.<br>>><br>>> "Rob Forler" <<a href="mailto:rforler@uchicago.edu">rforler@uchicago.edu</a>> wrote in message<br>
>> news:AANLkTikCZL=WhE_UoWiUvoQ+TLXj8dgrJ6iG4GhdOSdv@mail.gmail.com...<br>>> I'm coding in python now. The group I'm in now has a similar tool (closed<br>>> source) to data.table but in python and is based on numpy.<br>
>><br>>> The api isn't as beautiful as data.table's, but has similar functionality.<br>>><br>>> -Rob<br>>><br>>> On Tue, Dec 7, 2010 at 7:54 AM, Matthew Dowle <<a href="mailto:mdowle@mdowle.plus.com">mdowle@mdowle.plus.com</a>><br>
>> wrote:<br>>>><br>>>> Thanks Rob. That begs an obvious question then ... what are you coding in<br>>>> now ?<br>>>><br>>>> "Rob Forler" <<a href="mailto:rforler@uchicago.edu">rforler@uchicago.edu</a>> wrote in message<br>
>>> news:AANLkTikSsoy8F6aKuQYXW5GWeMLiE+Y07D_3X+K7AeX=@mail.gmail.com...<br>>>> I can attest that I used data.table very extensively for several months<br>>>> on large datasets (financial). I was replacing a fair of poorly coded<br>
>>> data/frame, sql, plyr, apply code, and was able to match the previous<br>>>> numbers and do a significant amount of new analysis because of the ease of<br>>>> using data.tables.<br>>>><br>
>>> If I was still coding in R on a regular basis you can gauarauntee I'd use<br>>>> data.table every day.<br>>>><br>>>> Thanks,<br>>>> Rob<br>>>><br>>>> On Tue, Dec 7, 2010 at 7:30 AM, Matthew Dowle <<a href="mailto:mdowle@mdowle.plus.com">mdowle@mdowle.plus.com</a>><br>
>>> wrote:<br>>>>><br>>>>> Just to clarify also about the date of first release - March 2010? Any<br>>>>> chance Mel you looked at the CRAN archive page and read off the last<br>
>>>> row?<br>>>>> Oldest is first not last on that page :<br>>>>><br>>>>> <a href="http://cran.r-project.org/src/contrib/Archive/data.table/" target="_blank">http://cran.r-project.org/src/contrib/Archive/data.table/</a><br>
>>>><br>>>>> v1.0 was released April 2006 but that was removed from CRAN happily<br>>>>> because<br>>>>> base quickly (within weeks) included features that removed the need for<br>
>>>> data.table. It was re-released in Aug 2008 with new functionality so<br>>>>> that's<br>>>>> the relevant release date for your purpose.<br>>>>><br>>>>> Feel free to post the puzzling results. You've done well to use it for 2<br>
>>>> weeks without posting, so you can probably tilt towards using this list<br>>>>> more<br>>>>> (on a new thread please). If we can get you over those hurdles first<br>>>>> then<br>
>>>> reconsider if the 'robustness' question still stands.<br>>>>><br>>>>> Other info which you may have not have found yet ...<br>>>>><br>>>>> Crantastic has 5 detailed user reviews of data.table. It does state that<br>
>>>> v1.1 was released over 2 years ago, too, so leads me to guess you may<br>>>>> have<br>>>>> missed the link to crantastic on the data.table homepage.<br>>>>><br>>>>> There are some oddities in the ranking formula but if you look at<br>
>>>> <a href="http://crantastic.org/popcon" target="_blank">http://crantastic.org/popcon</a> and realise that the batch near the bottom<br>>>>> starting with reshape, ggplot2 and plyr should be at the top (seems like<br>
>>>> a<br>>>>> bug, I'll let them know) then data.table appears to be around the 8th<br>>>>> most<br>>>>> popular CRAN package with average score 4.7/5 and 10 users, compared to<br>
>>>> ggplot2's 39 users. So crantastic itself is not popular since everyone<br>>>>> knows that ggplot2 has many more than 39 users, and some very popular<br>>>>> and<br>>>>> stable packages don't have any votes at all. Even so perhaps this small<br>
>>>> amount of data may be useful in your assessment generally. "data.table"<br>>>>> is<br>>>>> not the easiest to google for.<br>>>>><br>>>>> The NEWS file (link on the homepage) says that v1.2 was released in Aug<br>
>>>> 2008, too, at the bottom, along with what changed in each release since<br>>>>> then.<br>>>>><br>>>>> Matthew<br>>>>><br>>>>> "Tom Short" <<a href="mailto:tshort.rlists@gmail.com">tshort.rlists@gmail.com</a>> wrote in message<br>
>>>> news:AANLkTik=0j5da9j8_zVaW4DZhygKg6oqRP1Pg+JG3TFg@mail.gmail.com...<br>>>>> > On Mon, Dec 6, 2010 at 10:54 PM, mbacou <<a href="mailto:mel@mbacou.com">mel@mbacou.com</a>> wrote:<br>>>>> >> My question is: is data.table ready for production? Would you rely on<br>
>>>> >> it<br>>>>> >> for<br>>>>> >> sensitive publications?<br>>>>> ><br>>>>> > If you have tight time deadlines, you may want to go with what you<br>
>>>> > have experience with, especially if it involves complicated queries or<br>>>>> > manipulations. If you've already tried the data.table features you'll<br>>>>> > need for "production", then using data.table may help you get things<br>
>>>> > done faster.<br>>>>> ><br>>>>> > Data.table has been robust for me on 6-GB datasets on a machine with<br>>>>> > 24 GB of ram. With data.table, as with most tools, user error is more<br>
>>>> > likely than a tool bug, so you need to test/check your data and your<br>>>>> > results.<br>>>>> ><br>>>>> > - Tom<br>>>>><br>>>>><br>
>>>><br>>>>> _______________________________________________<br>>>>> datatable-help mailing list<br>>>>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
>>>><br>>>>> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>
>>><br>>>> ________________________________<br>>>><br>>>> _______________________________________________<br>>>> datatable-help mailing list<br>>>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
>>><br>>>> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>>>><br>
>>> _______________________________________________<br>>>> datatable-help mailing list<br>>>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
>>><br>>>> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>>>><br>
>><br>>> ________________________________<br>>><br>>> _______________________________________________<br>>> datatable-help mailing list<br>>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
>><br>>> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>>><br>>> _______________________________________________<br>
>> datatable-help mailing list<br>>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>>><br>>> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>
>><br>><br>><br>> _______________________________________________<br>> datatable-help mailing list<br>> <a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br>
> <a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>><br>><br>_______________________________________________<br>
datatable-help mailing list<br><a href="mailto:datatable-help@lists.r-forge.r-project.org">datatable-help@lists.r-forge.r-project.org</a><br><a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help</a><br>
</div></div></blockquote></div><br>