[Rprotobuf-yada] google protocol buffers

[Romain François]

[repost on the mailing list]

On 10/27/2009 03:03 PM, Romain François wrote:
> On 10/27/2009 02:20 PM, Dirk Eddelbuettel wrote:
>>
>> On 27 October 2009 at 14:06, Romain François wrote:
>> | I should add the disclaimer that I also am a pretty busy person, but
>> | that sounds nice, so I'll make some time.
>>
>> Same here. I really should be working on other things :-/ But this was
>> fun
>> so far.
>>
>> |> That was my idea too, but I know a _lot less_ about object system and
>> |> reflection than you.
>> |
>> | That's because I live in a java world and you live in a C++ world. java
>> | makes reflection really really easy.
>>
>> Quite so.
>>
>> | I discover new bits of reflection as I add things to rJava (a theme
>> park
>> | I have been visiting quite often since simon gave me the keys).
>> | Reflection (and a few other tricks) makes this code possible in R:
>> |
>> | .jinit()
>> | attach( javaImport( c("java.util","java.awt") ) )
>> | v<- new( Vector )
>> | v$add( "foo" )
>> | v$add( new( Point, 10L, 10L ) )
>> | lapply( v, function(o) o$getClass()$getSimpleName() )
>>
>> I don;t think we can do that in C++.
>>
>> |> As far as I know, 'standard C++' has nothing for reflection. C#
>> adds some of
>> |> the reflection stuff. The proto file has the added information, so
>> we either
>> |> need to parse it (yuck!) or rely on the generated stubs -- but
>> where to we
>> |> then get the glue from?
>> |
>> | I don't know much about it, since I've never used it, but Qt also has
>> | some reflection capabilities, but I think this only works for
>> subclasses
>> | of QObject. (this might be what Deepayan, ... are using)
>>
>> Yes but Qt uses the Moc layer and thereby creeates something on top of
>> C++.
>> You can't take that 'out of Qt' and into pure C++.
>>
>> | Last year, I spent quite some time on the internals of the bison parser
>> | grammar for R (for the sake of my syntax highlighter, which I still
>> need
>> | to release) and I also have worked on another for parsing bibtex files.
>> | In both situations, I started from an existent grammar file, but I
>> | should now be able to create one from scratch. I think it is easy now
>> | because I have not yet read the language description document
>>
>> :-)
>>
>> That may work, and would be cool, but I think it would violate the
>> 'spirit'
>> of ProtoBuf as you take it out of protoc. But one could start it as a
>> proof
>> of concept, and then maybe add it into a 'forked' or 'enhanced' protoc.
>> Generating R code directly would be __awesome__
>
> One last, and then I do some actual work. I think we'll have to look
> into the python implementation of the compiler (the thing that takes a
> proto file and makes the python code from)

... and we might not even need to. See this thread.
http://groups.google.com/group/protobuf/browse_thread/thread/d0f6d8ad88d2d100#

Basically, you can use the --descriptor_set_out= flag in protoc to 
create a FileDescriptor message that contains (as a protocol buffer) the 
description of the content of the file.

I've created a mailing list on the r-forge project, which should 
activate itself in a few days, so that we can structure discussions and 
leave a trace for the future generations ...


> I'm not much of a python freak, but the code does not seem too
> difficult. I'm looking at files
>
> src/google/protobuf/compiler/python/python_generator.*
>
> and I can sort of read the python file that has been generated.
>
> There are even hooks to add languages to the command line interface, so
> that we'll have :
>
> protoc file.proto --r-out=tmp
>
> ...
>
>> All that said, they are awesome for several things and the
>> serialization and
>> fast transmission that brought them to RHIPE is also very promising. I am
>> still not quite sure why Saptarshi hasn't used R serialization but I
>> am sure
>> there are good reasons somewhere.
>>
>> Dirk
>>
>
>


-- 
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://tr.im/BcPw : celebrating R commit #50000
|- http://tr.im/ztCu : RGG #158:161: examples of package IDPmisc
`- http://tr.im/yw8E : New R package : sos