<div dir="ltr">I'm glad it works, thanks for the feedback... and thanks for your patience while I get things like this ironed out.<div><br></div><div style>Ross</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Fri, Oct 11, 2013 at 7:37 AM, Peter Carl <span dir="ltr"><<a href="mailto:peter@braverock.com" target="_blank">peter@braverock.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Works great - thanks again...<br>
<div class="HOEnZb"><div class="h5"><br>
pcc<br>
--<br>
Peter Carl<br>
<a href="http://www.braverock.com/peter" target="_blank">http://www.braverock.com/peter</a><br>
<br>
> Thanks for doing this - I'll test it today.<br>
><br>
> To your point on comparing different methods - I completely agree. That<br>
> said, comparing the outputs of the different methods can be useful for<br>
> understanding methodological differences and ferreting out issues like<br>
> this one.<br>
><br>
> The heuristic methods are less accurate and possibly less precise than the<br>
> closed form methods (it isn't clear to me how much I care about this in<br>
> the face of estimation error), not to mention slower and more compute<br>
> intensive (but I know I care about those issues), and we'll always prefer<br>
> the latter when they are applicable. The heuristic methods, of course,<br>
> have their place in visualization and developing intuition about the more<br>
> 'abstract' methods, as well as in more complex objective sets.<br>
><br>
> I actually identified this issue comparing the two ROI-based methods,<br>
> noting that one of the dimensions was "off". Although I'm guessing that<br>
> this will take care of the issue, I'm not entirely sure. I'm glad you<br>
> told me that none of the arguments were being passed - it gives me some<br>
> comfort that the mETL measure was probably being calc'd at 95% instead of<br>
> 91.7% in the ROI optimization - this change should impact the portfolio<br>
> selected significantly. So hopefully this will work.<br>
><br>
> pcc<br>
> --<br>
> Peter Carl<br>
> <a href="http://www.braverock.com/peter" target="_blank">http://www.braverock.com/peter</a><br>
><br>
>> Peter,<br>
>><br>
>> In commit r3216 I made revisions so that the ROI solvers recognize clean<br>
>> and any other arguments in the $arguments slot in each objective. I am<br>
>> really glad you brought this up because I realized that nothing in the<br>
>> $arguments slot was being recognized for optimize_method="ROI"... this<br>
>> was<br>
>> something I overlooked. This is corrected and cleaned returns can now be<br>
>> used with optimize_method="ROI" for the QP/LP problems. I also added<br>
>> "StdDev" as a valid objective name, but still use "var" to calculate the<br>
>> variance-covariance matrix used in the objective function.<br>
>><br>
>> Regarding your example above with the min SD portfolio having a lower<br>
>> mETL<br>
>> than the min mETL. I'm not 100% sure on this, but it could depend on<br>
>> if/how<br>
>> you call constrained_objective to calculate the objective measures. For<br>
>> example, the results of the minimum ETL portfolio with ROI will be<br>
>> different than the minimum mETL portfolio with RP. The LP formulation to<br>
>> minimize ETL uses empirical returns data whereas the optimization with<br>
>> random portfolios uses modified ETL by default which will likely result<br>
>> in<br>
>> different optimal weights. It is a case of comparing "apples to<br>
>> oranges".<br>
>><br>
>> Attached is a quick script demonstrating this. Relevant outputs below.<br>
>><br>
>>> # Minimum ETL using ROI<br>
>>> optROI <- optimize.portfolio(R=ret, portfolio=tmp1,<br>
>> optimize_method="ROI", trace=TRUE)<br>
>> [1] 0.0833<br>
>>> extractObjectiveMeasures(optROI)<br>
>> $ES<br>
>> [1] 0.01718848<br>
>><br>
>>> # Use constrained_objective to calculate mETL with the optimal weights<br>
>> from optROI<br>
>>> constrained_objective(w=optROI$weights, R=ret, portfolio=tmp1,<br>
>> trace=TRUE)$objective_measures<br>
>> $ES<br>
>> [,1]<br>
>> ES 0.01761023<br>
>> attr(,"names")<br>
>> [1] "ES"<br>
>><br>
>>> # minimum mETL with random portfolios<br>
>>> optRP <- optimize.portfolio(R=ret, portfolio=tmp1,<br>
>> optimize_method="random", search_size=2000, trace=TRUE)<br>
>>> extractObjectiveMeasures(optRP)<br>
>> $ES<br>
>> [,1]<br>
>> ES 0.01740961<br>
>> attr(,"names")<br>
>> [1] "ES"<br>
>><br>
>> Ross<br>
>><br>
>><br>
>><br>
>> On Wed, Oct 9, 2013 at 3:20 PM, Ross Bennett<br>
>> <<a href="mailto:rossbennett34@gmail.com">rossbennett34@gmail.com</a>>wrote:<br>
>><br>
>>> Peter,<br>
>>><br>
>>> Thanks for the detailed description of what is going on. I will be able<br>
>>> to<br>
>>> dig into this deeper tomorrow afternoon, but here are some initial<br>
>>> comments.<br>
>>><br>
>>> See comments in-line.<br>
>>><br>
>>><br>
>>><br>
>>> On Wed, Oct 9, 2013 at 1:38 PM, Peter Carl <<a href="mailto:peter@braverock.com">peter@braverock.com</a>> wrote:<br>
>>><br>
>>>> Ross,<br>
>>>><br>
>>>> I'm running into the following:<br>
>>>><br>
>>>> > buoys.portfmeas<br>
>>>> Mean StdDev mETL<br>
>>>> MinSD 0.005435103 0.009286484 0.01357282<br>
>>>> MinmETL 0.005846839 0.011196709 0.01371471<br>
>>>><br>
>>>> Note that the minimum standard deviation portfolio has a lower mETL<br>
>>>> than<br>
>>>> the Min mETL portfolio. Both were calculated with ROI.<br>
>>>><br>
>>>> MinSD.portf <- add.objective(portfolio=init.portf,<br>
>>>> type="risk", # the kind of objective this is<br>
>>>> name="var", # name of the function<br>
>>>> arguments=list(clean=clean)<br>
>>>> )<br>
>>>><br>
>>>> > MinSD.ROI<-optimize.portfolio(R=R,<br>
>>>> + portfolio=MinSD.portf,<br>
>>>> + optimize_method='ROI'<br>
>>>> + )<br>
>>>> Warning message:<br>
>>>> In constrained_objective(w = weights, R = R, portfolio, trace = TRUE,<br>
>>>> :<br>
>>>> some arguments stored for var do not match<br>
>>>><br>
>>>> That warning is true, "var" doesn't take that list of arguments,<br>
>>>> including<br>
>>>> "clean". But when I use name="StdDev" in the portfolio construction,<br>
>>>> I<br>
>>>> get the following error:<br>
>>>><br>
>>>> Error in optimize.portfolio(R = R, portfolio = MinSD.portf,<br>
>>>> optimize_method = "ROI") :<br>
>>>> ROI only solves mean, var, or sample ETL/ES/CVaR type business<br>
>>>> objectives, choose a different optimize_method.<br>
>>>> In addition: Warning message:<br>
>>>> In <a href="http://is.na" target="_blank">is.na</a>(le) : <a href="http://is.na" target="_blank">is.na</a>() applied to non-(list or vector) of type 'NULL'<br>
>>>><br>
>>>> So the first issue is to map "StdDev" into the list of acceptable<br>
>>>> functions for that solver. It should then accept the "clean" argument<br>
>>>> and<br>
>>>> that should fix part of the issue.<br>
>>>><br>
>>><br>
>>> Got it, I'll add functionality for StdDev to be accepted by the ROI<br>
>>> solvers. I'll have to add StdDev, but still use "var" to calculate the<br>
>>> variance-covariance matrix used in the objective function. I will also<br>
>>> have<br>
>>> to detect the "clean" argument so that the cleaned returns are used to<br>
>>> calculate the variance-covariance matrix.<br>
>>><br>
>>><br>
>>>><br>
>>>> The second issue is that the Min mETL portfolio being identified isn't<br>
>>>> being plotted as the minimum mETL portfolio being identified in the<br>
>>>> cloud<br>
>>>> of random portfolios. This is the same issue I saw earlier that<br>
>>>> turned<br>
>>>> out to be related to "clean" not being applied consistently across<br>
>>>> objectives.<br>
>>>><br>
>>>> In this case, it should be:<br>
>>>> p=1-1/12<br>
>>>> clean="boudt"<br>
>>>> MinmETL.portf <- add.objective(portfolio=init.portf,<br>
>>>> type="risk", # the kind of objective this is<br>
>>>> name="ES", # the function to minimize<br>
>>>> arguments=list(p=p, clean=clean)<br>
>>>> )<br>
>>>><br>
>>>> > MinmETL.ROI<-optimize.portfolio(R=R,<br>
>>>> + portfolio=MinmETL.portf,<br>
>>>> + optimize_method='ROI',<br>
>>>> + trace=TRUE, verbose=TRUE<br>
>>>> + )<br>
>>>> Warning message:<br>
>>>> In <a href="http://is.na" target="_blank">is.na</a>(le) : <a href="http://is.na" target="_blank">is.na</a>() applied to non-(list or vector) of type 'NULL'<br>
>>>><br>
>>>> I'm not sure if the warning is related. But I'm wondering if "clean"<br>
>>>> is<br>
>>>> getting passed into "ES" with ROI. If it isn't, then the optimizer is<br>
>>>> likely selecting the wrong portfolio. Could you take a look at this<br>
>>>> when<br>
>>>> you have a moment?<br>
>>>><br>
>>><br>
>>> It could be related, in general, the "clean" argument is not passed to<br>
>>> ROI. I should be able to make some changes so that the cleaned returns<br>
>>> are<br>
>>> used for the ROI solvers. I'll take a closer look and report back with<br>
>>> my<br>
>>> findings.<br>
>>><br>
>>><br>
>>>><br>
>>>> Thanks,<br>
>>>><br>
>>>> pcc<br>
>>>> --<br>
>>>> Peter Carl<br>
>>>> <a href="http://www.braverock.com/peter" target="_blank">http://www.braverock.com/peter</a><br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> GSoC-PortA mailing list<br>
>>>> <a href="mailto:GSoC-PortA@lists.r-forge.r-project.org">GSoC-PortA@lists.r-forge.r-project.org</a><br>
>>>> <a href="http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta" target="_blank">http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta</a><br>
>>>><br>
>>><br>
>>><br>
>> _______________________________________________<br>
>> GSoC-PortA mailing list<br>
>> <a href="mailto:GSoC-PortA@lists.r-forge.r-project.org">GSoC-PortA@lists.r-forge.r-project.org</a><br>
>> <a href="http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta" target="_blank">http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta</a><br>
>><br>
><br>
> _______________________________________________<br>
> GSoC-PortA mailing list<br>
> <a href="mailto:GSoC-PortA@lists.r-forge.r-project.org">GSoC-PortA@lists.r-forge.r-project.org</a><br>
> <a href="http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta" target="_blank">http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta</a><br>
><br>
<br>
<br>
_______________________________________________<br>
GSoC-PortA mailing list<br>
<a href="mailto:GSoC-PortA@lists.r-forge.r-project.org">GSoC-PortA@lists.r-forge.r-project.org</a><br>
<a href="http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta" target="_blank">http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta</a><br>
</div></div></blockquote></div><br></div>