From brian at braverock.com  Mon Jul  1 21:53:57 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Mon, 01 Jul 2013 14:53:57 -0500
Subject: [GSoC-PortA] print and summary methods
Message-ID: <51D1DE55.5020303@braverock.com>

Ross,

I was just running through your testing scripts.

I notice this in the testing_portfolio_specification.R output:

$constraints
$constraints[[1]]
An object containing 5 constraints.
Some constraints are of type nonlinear.

$constraints[[2]]
An object containing 5 constraints.
Some constraints are of type nonlinear.

There are actually only two constraints slots.  Each of the constraint 
objects has 5 slots in the list, but these are the normal constraints 
slots, not 5 separate constraint objects.

I guess I would expect something more like:

#this is the output of str

$constraints[[1]]
   ..$ :List of 5
   .. ..$ type   : chr "weight_sum"
   .. ..$ enabled: logi TRUE
   .. ..$ min_sum: num 0.99
   .. ..$ max_sum: num 1.01
   .. ..$ call   : language add.constraint(portfolio = pspec, type = 
"weight_sum", enabled = TRUE,      min_sum = 0.99, max_sum = 1.01, 
indexnum = 1)
   .. ..- attr(*, "class")= chr [1:2] "weight_sum_constraint" "constraint"
$constraints[[2]]
   ..$ :List of 5
   .. ..$ type   : chr "box"
   .. ..$ enabled: logi FALSE
   .. ..$ min    : Named num [1:5] 0.1 0.05 0.1 0.15 0.2
   .. .. ..- attr(*, "names")= chr [1:5] "Convertible Arbitrage" "CTA 
Global" "Distressed Securities" "Emerging Markets" ...
   .. ..$ max    : Named num [1:5] 0.4 0.4 0.5 0.45 0.6
   .. .. ..- attr(*, "names")= chr [1:5] "Convertible Arbitrage" "CTA 
Global" "Distressed Securities" "Emerging Markets" ...
   .. ..$ call   : language add.constraint(portfolio = pspec, type = 
"box", min = c(0.1, 0.05, 0.1,      0.15, 0.2), max = c(0.4, 0.4, 0.5, 
0.45, 0.6), indexnum = 2)
   .. ..- attr(*, "class")= chr [1:2] "box_constraint" "constraint"


## or better yet

$constraints[[1]]
type: 'weight_sum'
enabled: TRUE
min_sum: 0.99
max_sum: 1.01
$constraints[[2]]
type: 'box'
enabled: TRUE
min: 0.1 0.05 0.1 0.15 0.2
max: 0.4 0.4 0.5 0.45 0.6
# these are named, so the named vector would be even better, of course

And similar print methods for objectives?

Thoughts?

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Mon Jul  1 22:22:28 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Mon, 1 Jul 2013 15:22:28 -0500
Subject: [GSoC-PortA] print and summary methods
In-Reply-To: <51D1DE55.5020303@braverock.com>
References: <51D1DE55.5020303@braverock.com>
Message-ID: <CAEESW=WN+JHcWtb=1jMT030ZtbyR9RDpGiCAimqJZ9sNJoqKUQ@mail.gmail.com>

Brian,

That is odd you are getting that for an output, I haven't done any work on
the print or summary methods. What you have outlined below for a print or
summary method makes perfect sense.

I just ran testing_portfolio_**specification.R and here is what I get.
> print(pspec$constraints)
[[1]]
$type
[1] "weight_sum"

$enabled
[1] TRUE

$min_sum
[1] 0.99

$max_sum
[1] 1.01

$call
add.constraint(portfolio = pspec, type = "weight_sum", enabled = TRUE,
    min_sum = 0.99, max_sum = 1.01, indexnum = 1)

attr(,"class")
[1] "weight_sum_constraint" "constraint"

[[2]]
$type
[1] "box"

$enabled
[1] FALSE

$min
Convertible Arbitrage            CTA Global Distressed Securities
                 0.10                  0.05                  0.10
     Emerging Markets Equity Market Neutral
                 0.15                  0.20

$max
Convertible Arbitrage            CTA Global Distressed Securities
                 0.40                  0.40                  0.50
     Emerging Markets Equity Market Neutral
                 0.45                  0.60

$call
add.constraint(portfolio = pspec, type = "box", min = c(0.1,
    0.05, 0.1, 0.15, 0.2), max = c(0.4, 0.4, 0.5, 0.45, 0.6),
    indexnum = 2)

attr(,"class")
[1] "box_constraint" "constraint"

Any idea why we get such different outputs?

Ross

On Mon, Jul 1, 2013 at 2:53 PM, Brian G. Peterson <brian at braverock.com>wrote:

> Ross,
>
> I was just running through your testing scripts.
>
> I notice this in the testing_portfolio_**specification.R output:
>
> $constraints
> $constraints[[1]]
> An object containing 5 constraints.
> Some constraints are of type nonlinear.
>
> $constraints[[2]]
> An object containing 5 constraints.
> Some constraints are of type nonlinear.
>
> There are actually only two constraints slots.  Each of the constraint
> objects has 5 slots in the list, but these are the normal constraints
> slots, not 5 separate constraint objects.
>
> I guess I would expect something more like:
>
> #this is the output of str
>
> $constraints[[1]]
>   ..$ :List of 5
>   .. ..$ type   : chr "weight_sum"
>   .. ..$ enabled: logi TRUE
>   .. ..$ min_sum: num 0.99
>   .. ..$ max_sum: num 1.01
>   .. ..$ call   : language add.constraint(portfolio = pspec, type =
> "weight_sum", enabled = TRUE,      min_sum = 0.99, max_sum = 1.01, indexnum
> = 1)
>   .. ..- attr(*, "class")= chr [1:2] "weight_sum_constraint" "constraint"
> $constraints[[2]]
>   ..$ :List of 5
>   .. ..$ type   : chr "box"
>   .. ..$ enabled: logi FALSE
>   .. ..$ min    : Named num [1:5] 0.1 0.05 0.1 0.15 0.2
>   .. .. ..- attr(*, "names")= chr [1:5] "Convertible Arbitrage" "CTA
> Global" "Distressed Securities" "Emerging Markets" ...
>   .. ..$ max    : Named num [1:5] 0.4 0.4 0.5 0.45 0.6
>   .. .. ..- attr(*, "names")= chr [1:5] "Convertible Arbitrage" "CTA
> Global" "Distressed Securities" "Emerging Markets" ...
>   .. ..$ call   : language add.constraint(portfolio = pspec, type = "box",
> min = c(0.1, 0.05, 0.1,      0.15, 0.2), max = c(0.4, 0.4, 0.5, 0.45, 0.6),
> indexnum = 2)
>   .. ..- attr(*, "class")= chr [1:2] "box_constraint" "constraint"
>
>
> ## or better yet
>
> $constraints[[1]]
> type: 'weight_sum'
> enabled: TRUE
> min_sum: 0.99
> max_sum: 1.01
> $constraints[[2]]
> type: 'box'
> enabled: TRUE
> min: 0.1 0.05 0.1 0.15 0.2
> max: 0.4 0.4 0.5 0.45 0.6
> # these are named, so the named vector would be even better, of course
>
> And similar print methods for objectives?
>
> Thoughts?
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130701/a46e619e/attachment.html>

From brian at braverock.com  Mon Jul  1 22:24:41 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Mon, 01 Jul 2013 15:24:41 -0500
Subject: [GSoC-PortA] other testing scripts
Message-ID: <51D1E589.9090904@braverock.com>

I just finished running through the other testing scripts from the sandbox.

I was able to run all of them except for 'testing_ROI_Martin.R' for 
which I don't have the data.

My other tests of objectives and constraints also seem to make sense.

Looks good so far.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From brian at braverock.com  Mon Jul  1 22:26:41 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Mon, 01 Jul 2013 15:26:41 -0500
Subject: [GSoC-PortA] print and summary methods
In-Reply-To: <CAEESW=WN+JHcWtb=1jMT030ZtbyR9RDpGiCAimqJZ9sNJoqKUQ@mail.gmail.com>
References: <51D1DE55.5020303@braverock.com>
 <CAEESW=WN+JHcWtb=1jMT030ZtbyR9RDpGiCAimqJZ9sNJoqKUQ@mail.gmail.com>
Message-ID: <51D1E601.2040304@braverock.com>

On 07/01/2013 03:22 PM, Ross Bennett wrote:
> Any idea why we get such different outputs?
>
> Ross
>

I'm guessing difference in R version.  Your output is what I would have 
expected, but perhaps being overwritten by either the R version or 
something else I might have loaded.

I guess there's nothing to worry about now then, and we'll come back to 
print and/or summary methods later.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Mon Jul  1 22:46:47 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Mon, 1 Jul 2013 15:46:47 -0500
Subject: [GSoC-PortA] other testing scripts
In-Reply-To: <51D1E589.9090904@braverock.com>
References: <51D1E589.9090904@braverock.com>
Message-ID: <CAEESW=Xc7PN-+9WB8jUBVCw1xsU-beoJKs7=8odc83XvS_phyA@mail.gmail.com>

Brian,

Thanks for the feedback. The data file was used by Doug in a portfolio
optimization course to test the results of his slides/material with
PortfolioAnalytics.

Doug, does it make sense (or possible depending on the data vendor) to
include the crsp.short.Rdata file with the PortfolioAnalytics package?

Ross

On Mon, Jul 1, 2013 at 3:24 PM, Brian G. Peterson <brian at braverock.com>wrote:

> I just finished running through the other testing scripts from the sandbox.
>
> I was able to run all of them except for 'testing_ROI_Martin.R' for which
> I don't have the data.
>
> My other tests of objectives and constraints also seem to make sense.
>
> Looks good so far.
>
> Regards,
>
> Brian
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130701/5041efab/attachment.html>

From martinrd at comcast.net  Tue Jul  2 01:05:47 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 1 Jul 2013 16:05:47 -0700
Subject: [GSoC-PortA] other testing scripts
In-Reply-To: <51D1E589.9090904@braverock.com>
References: <51D1E589.9090904@braverock.com>
Message-ID: <003001ce76af$8640a160$92c1e420$@comcast.net>

 

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Monday, July 01, 2013 1:25 PM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: [GSoC-PortA] other testing scripts

 

I just finished running through the other testing scripts from the sandbox.

 

I was able to run all of them except for 'testing_ROI_Martin.R' for which I
don't have the data.

[Doug] Ross, I that your script, or was it there from last year?

 

My other tests of objectives and constraints also seem to make sense.

 

Looks good so far.

 

Regards,

 

Brian

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130701/9c623268/attachment-0001.html>

From rossbennett34 at gmail.com  Tue Jul  2 01:14:41 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Mon, 1 Jul 2013 18:14:41 -0500
Subject: [GSoC-PortA] other testing scripts
In-Reply-To: <003001ce76af$8640a160$92c1e420$@comcast.net>
References: <51D1E589.9090904@braverock.com>
 <003001ce76af$8640a160$92c1e420$@comcast.net>
Message-ID: <CAEESW=U3-gWRv8GWj5jzcusWC-XJZzUOFVXemz5hk8MpT4LzFg@mail.gmail.com>

Doug,

It is a script that I added. I replicated your examples from 1. theory
review weights constrained mvo v5.pdf using PortfolioAnalytics.

Ross

On Mon, Jul 1, 2013 at 6:05 PM, Doug Martin <martinrd at comcast.net> wrote:

> ** **
>
> ** **
>
> ** **
>
> ** **
>
> -----Original Message-----
> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:
> gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G.
> Peterson
> Sent: Monday, July 01, 2013 1:25 PM
> To: gsoc-porta at r-forge.wu-wien.ac.at
> Subject: [GSoC-PortA] other testing scripts
>
> ** **
>
> I just finished running through the other testing scripts from the sandbox.
> ****
>
> ** **
>
> I was able to run all of them except for 'testing_ROI_Martin.R' for which
> I don't have the data.****
>
> *[Doug] Ross, I that your script, or was it there from last year?*****
>
> ** **
>
> My other tests of objectives and constraints also seem to make sense.****
>
> ** **
>
> Looks good so far.****
>
> ** **
>
> Regards,****
>
> ** **
>
> Brian****
>
> ** **
>
> --****
>
> Brian G. Peterson****
>
> http://braverock.com/brian/****
>
> Ph: 773-459-4973****
>
> IM: bgpbraverock****
>
> _______________________________________________****
>
> GSoC-PortA mailing list****
>
> GSoC-PortA at lists.r-forge.r-project.org****
>
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta****
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130701/a0d0117c/attachment.html>

From martinrd at comcast.net  Tue Jul  2 01:35:38 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 1 Jul 2013 16:35:38 -0700
Subject: [GSoC-PortA] other testing scripts
In-Reply-To: <CAEESW=U3-gWRv8GWj5jzcusWC-XJZzUOFVXemz5hk8MpT4LzFg@mail.gmail.com>
References: <51D1E589.9090904@braverock.com>
 <003001ce76af$8640a160$92c1e420$@comcast.net>
 <CAEESW=U3-gWRv8GWj5jzcusWC-XJZzUOFVXemz5hk8MpT4LzFg@mail.gmail.com>
Message-ID: <004c01ce76b3$b17fa840$147ef8c0$@comcast.net>

Ah, perfect.  I'll take a look.  Doug

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Monday, July 01, 2013 4:15 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] other testing scripts

 

Doug,

 

It is a script that I added. I replicated your examples from 1. theory
review weights constrained mvo v5.pdf using PortfolioAnalytics.

 

Ross

On Mon, Jul 1, 2013 at 6:05 PM, Doug Martin <martinrd at comcast.net> wrote:

 

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Monday, July 01, 2013 1:25 PM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: [GSoC-PortA] other testing scripts

 

I just finished running through the other testing scripts from the sandbox.

 

I was able to run all of them except for 'testing_ROI_Martin.R' for which I
don't have the data.

[Doug] Ross, I that your script, or was it there from last year?

 

My other tests of objectives and constraints also seem to make sense.

 

Looks good so far.

 

Regards,

 

Brian

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130701/f15f32f9/attachment.html>

From rossbennett34 at gmail.com  Wed Jul  3 05:13:13 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Tue, 2 Jul 2013 22:13:13 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <001801ce75b2$280c8260$78258720$@comcast.net>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
Message-ID: <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>

All,

I added an rp_constraint() function that uses logic from
randomize_portfolio() to transform a weights vector element by element to
satisfy (if possible) leverage *and* box constraints.

Here is a summary of where I am at with the mapping function.

   - Box Constraints (min/max)
      - rp_transform() takes a weight vector that violates either
      min_sum/max_sum leverage constraints *or* min/max box constraints and
      returns and weighting vector that satisfies leverage *and* box
constraints.
      - txfrm_box_constraint() takes a weight vector that violates min/max
      box constraints and will set any weight that violates min or max
to its min
      or max respectively. This is too simplistic and does not take
into account
      min_sum/max_sum leverage constraints.
      - I think rp_transform() is the better option here... thoughts?


   - Leverage Constraints (min_sum/max_sum)
      - rp_transform() takes a weight vector that violates either
      min_sum/max_sum leverage constraints *or* min/max box constraints and
      returns and weighting vector that satisfies leverage *and* box
constraints.
      - txfrm_weight_sum_constraint() takes a weight vector that violates
      min_sum/max_sum leverage constraints and normalizes the entire weights
      vector to satisfy leverage constraints. This is too simplistic
and does not
      take into account min/max box constraints.
      - I think rp_transform() is the better option here... thoughts?


   - Group Constraints (groups, cLO, cUP)
      - txfrm_group_constraint() loops through the groups and checks if cLO
      or cUP is violated. If cLO or cUP is violated the weights of the given
      group are normalized to equal cLO or cUP, whichever is violated.
This will
      likely change the sum of the weights vector and violate
min_sum/max_sum so
      we will have to "re-transform".
      - I think txfrm_group_constraint() is a good first step because it
      gets us close to satisfying the group constraints.
      - I'm working on incorporating the group constraints into
      rp_transform().
      - I'm not seeing how to use the eval(parse(text=formula), data) code to
      evaluate group constraints. Do you have a simple example?


   - Diversification Constraint
      - I'm having a hard time coming up with a straightforward solution to
      transform the vector of weights to meet the diversification
constraint. One
      idea I was working on was to generate N random portfolios and select the
      portfolio with the closest diversification value.
      - Note that I define diversification as: diversification = 1 -
      sum(weights^2)
      - Would it be better to just handle this *like* an objective and
      penalize in constrained_objective()?


   - Turnover Constraint
      - I'm having a hard time coming up with a straightforward solution to
      transform the vector of weights to meet the turnover constraint.
One idea I
      was working on was to generate N random portfolios and select
the portfolio
      with the closest turnover value.
      - Would it be better to just handle this *like* an objective and
      penalize in constrained_objective()?


   - Position Limit Constraint
      - txfrm_position_limit_constraint() sets the nassets - max_pos
      minimum weights equal to 0.
      - An issue is that for any min_i > 0, this will violate the min box
      constraint and be penalized later. Would it make sense to change
min_i to 0
      for asset_i that is set equal to 0?

One last thing is that I have the mapping function in a loop to get each
constraint object. The weights vector will be transformed in the order that
the constraints were added. In order to honor the hierarchy of how we
transform the weights vector, this should not be in a loop so we control
the order of transformation. Is that correct?

I look forward to your feedback and comments.

Thanks,
Ross

On Sun, Jun 30, 2013 at 11:51 AM, Doug Martin <martinrd at comcast.net> wrote:

> ** **
>
> ** **
>
> ** **
>
> -----Original Message-----
> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:
> gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G.
> Peterson
> Sent: Saturday, June 29, 2013 6:45 AM
> To: PortfolioAnalytics
> Subject: [GSoC-PortA] mapping function
>
> ** **
>
> Based on side conversations with Ross and Peter, I thought I should talk a
> little bit about next steps related to the mapping function.****
>
> ** **
>
> Apologies for the long email, I want to be complete, and I hope that some
> of this can make its way to the documentation.****
>
> ** **
>
> The purpose of the mapping function is to transform a weights vector that
> does not meet all the constraints into a weights vector that does meet the
> constraints, if one exists, hopefully with a minimum of transformation.***
> *
>
> ** **
>
> In the random portfolios code, we've used a couple of techniques pioneered
> by Pat Burns.  The philosophical idea is that your optimum portfolio is
> most likely to exist at the edges of the feasible space.****
>
> ** **
>
> At the first R/Finance conference, Pat used the analogy of a mountain
> lake, where the lake represents the feasible space.  With a combination of
> lots of different constraints, the shore of the lake will not be smooth or
> regular.  The lake (the feasible space) may not take up a large percentage
> of the terrain.****
>
> ** **
>
> If we randomly place a rock anywhere in the terrain, some of them will
> land in the lake, inside the feasible space, but most will land outside, on
> the slopes of the mountains that surround the lake.  The goal should be to
> nudge these towards the shores of the lake (our feasible space).****
>
> ** **
>
> Having exhausted the analogy, let's talk details.****
>
> ** **
>
> A slightly more rigorous treatment of the problem is given here:****
>
> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224****
>
> ** **
>
> *[Doug] This is the 2010 paper, which I just read much of.  Very nice
> paper.  I find Burns papers pretty useless except for pointing out nice
> statistical aspect and promoting PortfolioProbe.   For example in the paper
> you sent, he does not say clearly what he is doing in generating the random
> portfolios and what he means by out-of-sample.  As you once mentioned, I
> guess you got most details through direct conversation with him.*
>
> * *
>
> *Then I found the new Shaw 2011 paper at
> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. *
>
> *It has some nice additional material, e.g., Section 4.3 on lower bounds
> and Section 4.4 on bounded shorting, among other.  I still need to finish
> reading this version.*
>
> * *
>
> *In general if you accept random portfolios that violate the constraints,
> what is a good way to: (a) Accept only those within a certain distance,
> with appropriately defined metric,  of the feasible regions (you don?t want
> to consider all infeasible solutions ? I guess this the reason for min.sum,
> max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to
> take a solution nearest to a vertex?*
>
> * *
>
> * *
>
> * *
>
> It is possible that can use this method directly for random portfolios
> (and that we could add the ectra constraint types to DEoptim).  If so, much
> of the rest of what I'll write here is irrelevant.  I strongly suspect that
> there will be some constraint types that will still need to be 'adjusted'
> via a mapping method like the one laid out below, since a stochastic solver
> will hand us a vector that needs to be transformed at least in part to move
> into the feasible space.  It's alsom not entirely clear to me that the
> methods presented in the paper can satisfy all our constraint types.****
>
> ** **
>
> ** **
>
> I think our first step should be to test each constraint type, in some
> sort of hierarchy, starting with box constraints (almost all solvers
> support box constraints, of course), since some of the other
> transformations will violate the box constraints, and we'll need to
> transform back again.****
>
> ** **
>
> Each constraint can be evaluated as a logical expression against the
> weights vector.  You can see code for doing something similar with time
> series data in the sigFormula function in quantstrat. It takes advantage of
> some base R functionality that can treat an R object (in this case the
> weights vector) as an environment or 'frame'. This allows the columns of
> the data to be addressed without any major manipulation, simply by column
> name (asset name in the weights vector, possibly after adding names back
> in).****
>
> ** **
>
> The code looks something like this:****
>
> eval(parse(text=formula), data)****
>
> ** **
>
> So, 'data' is our weights vector, and 'formula' is an expression that can
> be evaluated as a formula by R.  Evaluating this formula will give us TRUE
> or FALSE to denote whether the weights vector is in compliance or in
> violation of that constraint.  Then, we'll need to transform the weight
> vector, if possible, to comply with that constraint.****
>
> ** **
>
> Specific Cases:****
>
> I've implemented this transformation for box constraints in the random
> portfolios code.  We don't need the evaluation I'll describe next for box
> constraints, because each single weight is handled separately.****
>
> ** **
>
> min_sum and max_sum leverage constraints can be evaluated without using
> the formula, since the formula is simple, and can be expressed in simple R
> code.  The transformation can be accomplished by transforming the entire
> vector.  There's code to do this in both the random portfolios code and in
> constrained_objective.  It is probably preferable to do the transformation
> one weight at a time, as I do in the random portfolios code, to end closer
> to the edges of the feasible space, while continuing to take the box
> constraints into account.****
>
> ** **
>
> linear (in)equality constraints and group constraints can be evaluated
> generically via the formula method I've described above.  Then individual
> weights can be transformed taking the value of the constraint****
>
> (<,>,=) into account (along with the box constraints and leverage
> constraints).****
>
> ** **
>
> and so on...****
>
> ** **
>
> Challenges:****
>
> - recovering the transformed vector from a optimization solver that
> doesn't directly support a mapping function.  I've got some tricks for this
> using environments that we can revisit after we get the basic methodology
> working.****
>
> ** **
>
> -allowing for progressively relaxing constraints when the constraints are
> simply too restrictive.  Perhaps Doug has some documentation on this as
> he's done it in the past, or perhaps we can simply deal with it in the
> penalty part of constrained_objective()****
>
> ** **
>
> Hopefully this was helpful.****
>
> ** **
>
> Regards,****
>
> ** **
>
> Brian****
>
> ** **
>
> --****
>
> Brian G. Peterson****
>
> http://braverock.com/brian/****
>
> Ph: 773-459-4973****
>
> IM: bgpbraverock****
>
> _______________________________________________****
>
> GSoC-PortA mailing list****
>
> GSoC-PortA at lists.r-forge.r-project.org****
>
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta****
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130702/bb5f6d88/attachment.html>

From martinrd at comcast.net  Wed Jul  3 06:57:06 2013
From: martinrd at comcast.net (Doug Martin)
Date: Tue, 2 Jul 2013 21:57:06 -0700
Subject: [GSoC-PortA] Turnover Constraints Example
Message-ID: <005401ce77a9$c4e6c700$4eb45500$@comcast.net>

Please see slides 7-10 of the attached slide deck, which are obtained with
the code in the zip file.

I note that using toc = 0 gives an inconsistent constraints message, but
this could be easily fixed in the

efficient frontier code to just return the portfolio with the initial
weights.  Ross you have already run this code

for a homework problem.

 

Next I will get an issue straightened out in the proportional costs code and
efficient frontier calculation

and send examples working properly.

 

Doug

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Monday, July 01, 2013 1:47 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] other testing scripts

 

Brian,

 

Thanks for the feedback. The data file was used by Doug in a portfolio
optimization course to test the results of his slides/material with
PortfolioAnalytics. 

 

Doug, does it make sense (or possible depending on the data vendor) to
include the crsp.short.Rdata file with the PortfolioAnalytics package?

 

Ross

On Mon, Jul 1, 2013 at 3:24 PM, Brian G. Peterson <brian at braverock.com>
wrote:

I just finished running through the other testing scripts from the sandbox.

I was able to run all of them except for 'testing_ROI_Martin.R' for which I
don't have the data.

My other tests of objectives and constraints also seem to make sense.

Looks good so far.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130702/c02464d3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 5.  transaction costs and MIP v2.ppt
Type: application/vnd.ms-powerpoint
Size: 420352 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130702/c02464d3/attachment-0001.ppt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: advanced onstraints.zip
Type: application/x-zip-compressed
Size: 42771 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130702/c02464d3/attachment-0001.bin>

From rossbennett34 at gmail.com  Sat Jul  6 18:34:09 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Sat, 6 Jul 2013 11:34:09 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
Message-ID: <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>

All,

I have a few thoughts on the hierarchy and how to transform weights that
violate constraints and would appreciate any input or feedback.

The process outlined below is what I am thinking of following for the
constraint mapping fn_map() function.

   - Step 1: Test weights vector for violation of min_sum or max_sum. If
   violated, transform the weights vector with rp_transform() taking into
   account both leverage and box constraints. Another option is to normalize
   the weights by transforming the entire vector like what is done
   constrained_objective()... is one way preferable over the other?
   - Step 2: Test weights vector for violation of min or max. If violated,
   transform the weights vector with rp_transform() taking into account both
   leverage and box constraints. If we can't generate a feasible portfolio,
   this is because min or max is too restrictive.  Try relaxing min or max.
   e.g., if min is violated, we could simply relax min by doing something like
   min <- min - 0.05 and try this N times to generate a feasible portfolio. Or
   we could randomly select an element of min and decrease it instead of
   modifying the entire vector.
   - Step 3: Test weights vector for violation of groups, cLO, or cUP. If
   violated, transform the weights vector with rp_transform() taking into
   account leverage, box, and group constraints. If we can't generate a
   feasible portfolio, try relaxing cLO or cUP. e.g., if cLO is violated, we
   could simply relax cLO by doing something like cLO <- cLO - 0.05 and try
   this N times to generate a feasible portfolio. Or we could randomly select
   an element of min and decrease it instead of modifying the entire vector.
   -
   - Step 4: Test weights vector for violation of max_pos. If violated,
   transform the weights vector with rp_transform() taking into account
   leverage, box, group, and position limit constraints.

Please advise if this is consistent with how you see the hierarchy in the
constraint mapping function working?

Thanks,
Ross

On Tue, Jul 2, 2013 at 10:13 PM, Ross Bennett <rossbennett34 at gmail.com>wrote:

> All,
>
> I added an rp_constraint() function that uses logic from
> randomize_portfolio() to transform a weights vector element by element to
> satisfy (if possible) leverage *and* box constraints.
>
> Here is a summary of where I am at with the mapping function.
>
>    - Box Constraints (min/max)
>       - rp_transform() takes a weight vector that violates either
>       min_sum/max_sum leverage constraints *or* min/max box constraints and
>       returns and weighting vector that satisfies leverage *and* box constraints.
>       - txfrm_box_constraint() takes a weight vector that violates
>       min/max box constraints and will set any weight that violates min or max to
>       its min or max respectively. This is too simplistic and does not take into
>       account min_sum/max_sum leverage constraints.
>       - I think rp_transform() is the better option here... thoughts?
>
>
>    - Leverage Constraints (min_sum/max_sum)
>       - rp_transform() takes a weight vector that violates either
>       min_sum/max_sum leverage constraints *or* min/max box constraints and
>       returns and weighting vector that satisfies leverage *and* box constraints.
>       - txfrm_weight_sum_constraint() takes a weight vector that violates
>       min_sum/max_sum leverage constraints and normalizes the entire weights
>       vector to satisfy leverage constraints. This is too simplistic and does not
>       take into account min/max box constraints.
>       - I think rp_transform() is the better option here... thoughts?
>
>
>    - Group Constraints (groups, cLO, cUP)
>       - txfrm_group_constraint() loops through the groups and checks if
>       cLO or cUP is violated. If cLO or cUP is violated the weights of the given
>       group are normalized to equal cLO or cUP, whichever is violated. This will
>       likely change the sum of the weights vector and violate min_sum/max_sum so
>       we will have to "re-transform".
>       - I think txfrm_group_constraint() is a good first step because it
>       gets us close to satisfying the group constraints.
>       - I'm working on incorporating the group constraints into
>       rp_transform().
>       - I'm not seeing how to use the eval(parse(text=formula), data) code to
>       evaluate group constraints. Do you have a simple example?
>
>
>    - Diversification Constraint
>       - I'm having a hard time coming up with a straightforward solution
>       to transform the vector of weights to meet the diversification constraint.
>       One idea I was working on was to generate N random portfolios and select
>       the portfolio with the closest diversification value.
>       - Note that I define diversification as: diversification = 1 -
>       sum(weights^2)
>       - Would it be better to just handle this *like* an objective and
>       penalize in constrained_objective()?
>
>
>    - Turnover Constraint
>       - I'm having a hard time coming up with a straightforward solution
>       to transform the vector of weights to meet the turnover constraint. One
>       idea I was working on was to generate N random portfolios and select the
>       portfolio with the closest turnover value.
>       - Would it be better to just handle this *like* an objective and
>       penalize in constrained_objective()?
>
>
>    - Position Limit Constraint
>       - txfrm_position_limit_constraint() sets the nassets - max_pos
>       minimum weights equal to 0.
>       - An issue is that for any min_i > 0, this will violate the min box
>       constraint and be penalized later. Would it make sense to change min_i to 0
>       for asset_i that is set equal to 0?
>
> One last thing is that I have the mapping function in a loop to get each
> constraint object. The weights vector will be transformed in the order that
> the constraints were added. In order to honor the hierarchy of how we
> transform the weights vector, this should not be in a loop so we control
> the order of transformation. Is that correct?
>
> I look forward to your feedback and comments.
>
> Thanks,
> Ross
>
> On Sun, Jun 30, 2013 at 11:51 AM, Doug Martin <martinrd at comcast.net>wrote:
>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> -----Original Message-----
>> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:
>> gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G.
>> Peterson
>> Sent: Saturday, June 29, 2013 6:45 AM
>> To: PortfolioAnalytics
>> Subject: [GSoC-PortA] mapping function
>>
>> ** **
>>
>> Based on side conversations with Ross and Peter, I thought I should talk
>> a little bit about next steps related to the mapping function.****
>>
>> ** **
>>
>> Apologies for the long email, I want to be complete, and I hope that some
>> of this can make its way to the documentation.****
>>
>> ** **
>>
>> The purpose of the mapping function is to transform a weights vector that
>> does not meet all the constraints into a weights vector that does meet the
>> constraints, if one exists, hopefully with a minimum of transformation.**
>> **
>>
>> ** **
>>
>> In the random portfolios code, we've used a couple of techniques
>> pioneered by Pat Burns.  The philosophical idea is that your optimum
>> portfolio is most likely to exist at the edges of the feasible space.****
>>
>> ** **
>>
>> At the first R/Finance conference, Pat used the analogy of a mountain
>> lake, where the lake represents the feasible space.  With a combination of
>> lots of different constraints, the shore of the lake will not be smooth or
>> regular.  The lake (the feasible space) may not take up a large percentage
>> of the terrain.****
>>
>> ** **
>>
>> If we randomly place a rock anywhere in the terrain, some of them will
>> land in the lake, inside the feasible space, but most will land outside, on
>> the slopes of the mountains that surround the lake.  The goal should be to
>> nudge these towards the shores of the lake (our feasible space).****
>>
>> ** **
>>
>> Having exhausted the analogy, let's talk details.****
>>
>> ** **
>>
>> A slightly more rigorous treatment of the problem is given here:****
>>
>> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224****
>>
>> ** **
>>
>> *[Doug] This is the 2010 paper, which I just read much of.  Very nice
>> paper.  I find Burns papers pretty useless except for pointing out nice
>> statistical aspect and promoting PortfolioProbe.   For example in the paper
>> you sent, he does not say clearly what he is doing in generating the random
>> portfolios and what he means by out-of-sample.  As you once mentioned, I
>> guess you got most details through direct conversation with him.*
>>
>> * *
>>
>> *Then I found the new Shaw 2011 paper at
>> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. *
>>
>> *It has some nice additional material, e.g., Section 4.3 on lower bounds
>> and Section 4.4 on bounded shorting, among other.  I still need to finish
>> reading this version.*
>>
>> * *
>>
>> *In general if you accept random portfolios that violate the
>> constraints, what is a good way to: (a) Accept only those within a certain
>> distance, with appropriately defined metric,  of the feasible regions (you
>> don?t want to consider all infeasible solutions ? I guess this the reason
>> for min.sum, max.sum, etc.?), and (b) Assuming an appropriate metric, does
>> one want to take a solution nearest to a vertex?*
>>
>> * *
>>
>> * *
>>
>> * *
>>
>> It is possible that can use this method directly for random portfolios
>> (and that we could add the ectra constraint types to DEoptim).  If so, much
>> of the rest of what I'll write here is irrelevant.  I strongly suspect that
>> there will be some constraint types that will still need to be 'adjusted'
>> via a mapping method like the one laid out below, since a stochastic solver
>> will hand us a vector that needs to be transformed at least in part to move
>> into the feasible space.  It's alsom not entirely clear to me that the
>> methods presented in the paper can satisfy all our constraint types.****
>>
>> ** **
>>
>> ** **
>>
>> I think our first step should be to test each constraint type, in some
>> sort of hierarchy, starting with box constraints (almost all solvers
>> support box constraints, of course), since some of the other
>> transformations will violate the box constraints, and we'll need to
>> transform back again.****
>>
>> ** **
>>
>> Each constraint can be evaluated as a logical expression against the
>> weights vector.  You can see code for doing something similar with time
>> series data in the sigFormula function in quantstrat. It takes advantage of
>> some base R functionality that can treat an R object (in this case the
>> weights vector) as an environment or 'frame'. This allows the columns of
>> the data to be addressed without any major manipulation, simply by column
>> name (asset name in the weights vector, possibly after adding names back
>> in).****
>>
>> ** **
>>
>> The code looks something like this:****
>>
>> eval(parse(text=formula), data)****
>>
>> ** **
>>
>> So, 'data' is our weights vector, and 'formula' is an expression that can
>> be evaluated as a formula by R.  Evaluating this formula will give us TRUE
>> or FALSE to denote whether the weights vector is in compliance or in
>> violation of that constraint.  Then, we'll need to transform the weight
>> vector, if possible, to comply with that constraint.****
>>
>> ** **
>>
>> Specific Cases:****
>>
>> I've implemented this transformation for box constraints in the random
>> portfolios code.  We don't need the evaluation I'll describe next for box
>> constraints, because each single weight is handled separately.****
>>
>> ** **
>>
>> min_sum and max_sum leverage constraints can be evaluated without using
>> the formula, since the formula is simple, and can be expressed in simple R
>> code.  The transformation can be accomplished by transforming the entire
>> vector.  There's code to do this in both the random portfolios code and in
>> constrained_objective.  It is probably preferable to do the transformation
>> one weight at a time, as I do in the random portfolios code, to end closer
>> to the edges of the feasible space, while continuing to take the box
>> constraints into account.****
>>
>> ** **
>>
>> linear (in)equality constraints and group constraints can be evaluated
>> generically via the formula method I've described above.  Then individual
>> weights can be transformed taking the value of the constraint****
>>
>> (<,>,=) into account (along with the box constraints and leverage
>> constraints).****
>>
>> ** **
>>
>> and so on...****
>>
>> ** **
>>
>> Challenges:****
>>
>> - recovering the transformed vector from a optimization solver that
>> doesn't directly support a mapping function.  I've got some tricks for this
>> using environments that we can revisit after we get the basic methodology
>> working.****
>>
>> ** **
>>
>> -allowing for progressively relaxing constraints when the constraints are
>> simply too restrictive.  Perhaps Doug has some documentation on this as
>> he's done it in the past, or perhaps we can simply deal with it in the
>> penalty part of constrained_objective()****
>>
>> ** **
>>
>> Hopefully this was helpful.****
>>
>> ** **
>>
>> Regards,****
>>
>> ** **
>>
>> Brian****
>>
>> ** **
>>
>> --****
>>
>> Brian G. Peterson****
>>
>> http://braverock.com/brian/****
>>
>> Ph: 773-459-4973****
>>
>> IM: bgpbraverock****
>>
>> _______________________________________________****
>>
>> GSoC-PortA mailing list****
>>
>> GSoC-PortA at lists.r-forge.r-project.org****
>>
>> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta***
>> *
>>
>> _______________________________________________
>> GSoC-PortA mailing list
>> GSoC-PortA at lists.r-forge.r-project.org
>> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130706/86176afe/attachment-0001.html>

From martinrd at comcast.net  Sat Jul  6 19:23:52 2013
From: martinrd at comcast.net (Doug Martin)
Date: Sat, 6 Jul 2013 10:23:52 -0700
Subject: [GSoC-PortA] other testing scripts
In-Reply-To: <CAEESW=Xc7PN-+9WB8jUBVCw1xsU-beoJKs7=8odc83XvS_phyA@mail.gmail.com>
References: <51D1E589.9090904@braverock.com>
 <CAEESW=Xc7PN-+9WB8jUBVCw1xsU-beoJKs7=8odc83XvS_phyA@mail.gmail.com>
Message-ID: <001501ce7a6d$9f59a7c0$de0cf740$@comcast.net>

 

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Monday, July 01, 2013 1:47 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] other testing scripts

 

Brian,

 

Thanks for the feedback. The data file was used by Doug in a portfolio
optimization course to test the results of his slides/material with
PortfolioAnalytics. 

 

Doug, does it make sense (or possible depending on the data vendor) to
include the crsp.short.Rdata file with the PortfolioAnalytics package?

[Doug] It would make sense but I want to renew an agreement with CRSP for an
expanded data set for the 2nd edition of MPO, and would include making the
data accessible with PortfolioAnalytics.  This will take some time.

 

Ross

On Mon, Jul 1, 2013 at 3:24 PM, Brian G. Peterson <brian at braverock.com>
wrote:

I just finished running through the other testing scripts from the sandbox.

I was able to run all of them except for 'testing_ROI_Martin.R' for which I
don't have the data.

My other tests of objectives and constraints also seem to make sense.

Looks good so far.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130706/1d495440/attachment.html>

From martinrd at comcast.net  Sat Jul  6 20:31:39 2013
From: martinrd at comcast.net (Doug Martin)
Date: Sat, 6 Jul 2013 11:31:39 -0700
Subject: [GSoC-PortA] mapping function
In-Reply-To: <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
Message-ID: <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>

Brian,

 

I don't really understand what is going on, e.g., why do you need min_sum
and max_sum?  I get the impression that the random portfolio weights are
being generated without a full-investment constraint, but with a range of
near-full-investment constraints.  Why is that needed?    If I understand
Shaw (2010) correctly there exists a good method of randomly sampling from
the simplex of positive numbers, i.e., long-only and full-investment, in a
manner that promises to work well in higher dimensions by generating several
sets of random samples ranging from one that is most concentrated at 1/n and
ones that are more concentrated near edges and vertices.  In Shaw (2011)
Section 4.3 he describes a modification for handling lower bounds, and at
first glance it appears that it can be modified for upper bounds, hence box
constraints since the long-only part is already covered.  Caveat:  I have
only quickly skimmed the first parts of the Shaw papers so maybe I am
missing something.

 

Doug

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Saturday, July 06, 2013 9:34 AM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] mapping function

 

All,

 

I have a few thoughts on the hierarchy and how to transform weights that
violate constraints and would appreciate any input or feedback.

 

The process outlined below is what I am thinking of following for the
constraint mapping fn_map() function.

*	Step 1: Test weights vector for violation of min_sum or max_sum. If
violated, transform the weights vector with rp_transform() taking into
account both leverage and box constraints. Another option is to normalize
the weights by transforming the entire vector like what is done
constrained_objective()... is one way preferable over the other?
*	Step 2: Test weights vector for violation of min or max. If
violated, transform the weights vector with rp_transform() taking into
account both leverage and box constraints. If we can't generate a feasible
portfolio, this is because min or max is too restrictive.  Try relaxing min
or max. e.g., if min is violated, we could simply relax min by doing
something like min <- min - 0.05 and try this N times to generate a feasible
portfolio. Or we could randomly select an element of min and decrease it
instead of modifying the entire vector.
*	Step 3: Test weights vector for violation of groups, cLO, or cUP. If
violated, transform the weights vector with rp_transform() taking into
account leverage, box, and group constraints. If we can't generate a
feasible portfolio, try relaxing cLO or cUP. e.g., if cLO is violated, we
could simply relax cLO by doing something like cLO <- cLO - 0.05 and try
this N times to generate a feasible portfolio. Or we could randomly select
an element of min and decrease it instead of modifying the entire vector.
*	 
*	Step 4: Test weights vector for violation of max_pos. If violated,
transform the weights vector with rp_transform() taking into account
leverage, box, group, and position limit constraints.

Please advise if this is consistent with how you see the hierarchy in the
constraint mapping function working?

 

Thanks,

Ross

 

On Tue, Jul 2, 2013 at 10:13 PM, Ross Bennett <rossbennett34 at gmail.com>
wrote:

All,

 

I added an rp_constraint() function that uses logic from
randomize_portfolio() to transform a weights vector element by element to
satisfy (if possible) leverage *and* box constraints.

 

Here is a summary of where I am at with the mapping function.

*	Box Constraints (min/max)

*	rp_transform() takes a weight vector that violates either
min_sum/max_sum leverage constraints *or* min/max box constraints and
returns and weighting vector that satisfies leverage *and* box constraints.
*	txfrm_box_constraint() takes a weight vector that violates min/max
box constraints and will set any weight that violates min or max to its min
or max respectively. This is too simplistic and does not take into account
min_sum/max_sum leverage constraints.
*	I think rp_transform() is the better option here... thoughts?

*	Leverage Constraints (min_sum/max_sum)

*	rp_transform() takes a weight vector that violates either
min_sum/max_sum leverage constraints *or* min/max box constraints and
returns and weighting vector that satisfies leverage *and* box constraints.
*	txfrm_weight_sum_constraint() takes a weight vector that violates
min_sum/max_sum leverage constraints and normalizes the entire weights
vector to satisfy leverage constraints. This is too simplistic and does not
take into account min/max box constraints.
*	I think rp_transform() is the better option here... thoughts?

*	Group Constraints (groups, cLO, cUP) 

*	txfrm_group_constraint() loops through the groups and checks if cLO
or cUP is violated. If cLO or cUP is violated the weights of the given group
are normalized to equal cLO or cUP, whichever is violated. This will likely
change the sum of the weights vector and violate min_sum/max_sum so we will
have to "re-transform".
*	I think txfrm_group_constraint() is a good first step because it
gets us close to satisfying the group constraints.
*	I'm working on incorporating the group constraints into
rp_transform().
*	I'm not seeing how to use the eval(parse(text=formula), data) code
to evaluate group constraints. Do you have a simple example?

*	Diversification Constraint

*	I'm having a hard time coming up with a straightforward solution to
transform the vector of weights to meet the diversification constraint. One
idea I was working on was to generate N random portfolios and select the
portfolio with the closest diversification value.
*	Note that I define diversification as: diversification = 1 -
sum(weights^2)
*	Would it be better to just handle this *like* an objective and
penalize in constrained_objective()?

*	Turnover Constraint

*	I'm having a hard time coming up with a straightforward solution to
transform the vector of weights to meet the turnover constraint. One idea I
was working on was to generate N random portfolios and select the portfolio
with the closest turnover value.
*	Would it be better to just handle this *like* an objective and
penalize in constrained_objective()?

*	Position Limit Constraint

*	txfrm_position_limit_constraint() sets the nassets - max_pos minimum
weights equal to 0.
*	An issue is that for any min_i > 0, this will violate the min box
constraint and be penalized later. Would it make sense to change min_i to 0
for asset_i that is set equal to 0?

One last thing is that I have the mapping function in a loop to get each
constraint object. The weights vector will be transformed in the order that
the constraints were added. In order to honor the hierarchy of how we
transform the weights vector, this should not be in a loop so we control the
order of transformation. Is that correct?

 

I look forward to your feedback and comments.

 

Thanks,

Ross

 

On Sun, Jun 30, 2013 at 11:51 AM, Doug Martin <martinrd at comcast.net> wrote:

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson

Sent: Saturday, June 29, 2013 6:45 AM
To: PortfolioAnalytics
Subject: [GSoC-PortA] mapping function

 

Based on side conversations with Ross and Peter, I thought I should talk a
little bit about next steps related to the mapping function.

 

Apologies for the long email, I want to be complete, and I hope that some of
this can make its way to the documentation.

 

The purpose of the mapping function is to transform a weights vector that
does not meet all the constraints into a weights vector that does meet the
constraints, if one exists, hopefully with a minimum of transformation.

 

In the random portfolios code, we've used a couple of techniques pioneered
by Pat Burns.  The philosophical idea is that your optimum portfolio is most
likely to exist at the edges of the feasible space.

 

At the first R/Finance conference, Pat used the analogy of a mountain lake,
where the lake represents the feasible space.  With a combination of lots of
different constraints, the shore of the lake will not be smooth or regular.
The lake (the feasible space) may not take up a large percentage of the
terrain.

 

If we randomly place a rock anywhere in the terrain, some of them will land
in the lake, inside the feasible space, but most will land outside, on the
slopes of the mountains that surround the lake.  The goal should be to nudge
these towards the shores of the lake (our feasible space).

 

Having exhausted the analogy, let's talk details.

 

A slightly more rigorous treatment of the problem is given here:

 <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224>
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224

 

[Doug] This is the 2010 paper, which I just read much of.  Very nice paper.
I find Burns papers pretty useless except for pointing out nice statistical
aspect and promoting PortfolioProbe.   For example in the paper you sent, he
does not say clearly what he is doing in generating the random portfolios
and what he means by out-of-sample.  As you once mentioned, I guess you got
most details through direct conversation with him.

 

Then I found the new Shaw 2011 paper at
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. 

It has some nice additional material, e.g., Section 4.3 on lower bounds and
Section 4.4 on bounded shorting, among other.  I still need to finish
reading this version.

 

In general if you accept random portfolios that violate the constraints,
what is a good way to: (a) Accept only those within a certain distance, with
appropriately defined metric,  of the feasible regions (you don't want to
consider all infeasible solutions - I guess this the reason for min.sum,
max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to
take a solution nearest to a vertex?

 

 

 

It is possible that can use this method directly for random portfolios (and
that we could add the ectra constraint types to DEoptim).  If so, much of
the rest of what I'll write here is irrelevant.  I strongly suspect that
there will be some constraint types that will still need to be 'adjusted'
via a mapping method like the one laid out below, since a stochastic solver
will hand us a vector that needs to be transformed at least in part to move
into the feasible space.  It's alsom not entirely clear to me that the
methods presented in the paper can satisfy all our constraint types.

 

 

I think our first step should be to test each constraint type, in some sort
of hierarchy, starting with box constraints (almost all solvers support box
constraints, of course), since some of the other transformations will
violate the box constraints, and we'll need to transform back again.

 

Each constraint can be evaluated as a logical expression against the weights
vector.  You can see code for doing something similar with time series data
in the sigFormula function in quantstrat. It takes advantage of some base R
functionality that can treat an R object (in this case the weights vector)
as an environment or 'frame'. This allows the columns of the data to be
addressed without any major manipulation, simply by column name (asset name
in the weights vector, possibly after adding names back in).

 

The code looks something like this:

eval(parse(text=formula), data)

 

So, 'data' is our weights vector, and 'formula' is an expression that can be
evaluated as a formula by R.  Evaluating this formula will give us TRUE or
FALSE to denote whether the weights vector is in compliance or in violation
of that constraint.  Then, we'll need to transform the weight vector, if
possible, to comply with that constraint.

 

Specific Cases:

I've implemented this transformation for box constraints in the random
portfolios code.  We don't need the evaluation I'll describe next for box
constraints, because each single weight is handled separately.

 

min_sum and max_sum leverage constraints can be evaluated without using the
formula, since the formula is simple, and can be expressed in simple R code.
The transformation can be accomplished by transforming the entire vector.
There's code to do this in both the random portfolios code and in
constrained_objective.  It is probably preferable to do the transformation
one weight at a time, as I do in the random portfolios code, to end closer
to the edges of the feasible space, while continuing to take the box
constraints into account.

 

linear (in)equality constraints and group constraints can be evaluated
generically via the formula method I've described above.  Then individual
weights can be transformed taking the value of the constraint

(<,>,=) into account (along with the box constraints and leverage
constraints).

 

and so on...

 

Challenges:

- recovering the transformed vector from a optimization solver that doesn't
directly support a mapping function.  I've got some tricks for this using
environments that we can revisit after we get the basic methodology working.

 

-allowing for progressively relaxing constraints when the constraints are
simply too restrictive.  Perhaps Doug has some documentation on this as he's
done it in the past, or perhaps we can simply deal with it in the penalty
part of constrained_objective()

 

Hopefully this was helpful.

 

Regards,

 

Brian

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130706/dcb7906a/attachment-0001.html>

From rossbennett34 at gmail.com  Sat Jul  6 22:27:29 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Sat, 6 Jul 2013 15:27:29 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
Message-ID: <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>

Doug,

I'm sure Brian will provide more detail, but hopefully this will provide
some initial clarification about random portfolios and the logic I am using
in rp_transform().

The random portfolio weights are being generated with a user specified
leverage constraint, min_sum and max_sum. One could generate a dollar
neutral portfolio by specifying min_sum=-0.01 and max_sum=1.01 or a
leveraged portfolio with min_sum=1.49 and max_sum=1.51. I think this gives
more flexibility than the method proposed by Shaw.

I need to spend some more time understanding the Shaw (2011) paper, but it
is unclear to me if Shaw's method will work if we want to specify
constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and max=c(0.85,
0.92, 0.73, 0.75, 0.82) or if it only works for box constraints where all
the values of min and max are equal.

Thanks,
Ross

On Sat, Jul 6, 2013 at 1:31 PM, Doug Martin <martinrd at comcast.net> wrote:

> Brian,****
>
> ** **
>
> I don?t really understand what is going on, e.g., why do you need min_sum
> and max_sum?  I get the impression that the random portfolio weights are
> being generated without a full-investment constraint, but with a range of
> near-full-investment constraints.  Why is that needed?    If I understand
> Shaw (2010) correctly there exists a good method of randomly sampling from
> the simplex of positive numbers, i.e., long-only and full-investment, in a
> manner that promises to work well in higher dimensions by generating
> several sets of random samples ranging from one that is most concentrated
> at 1/n and ones that are more concentrated near edges and vertices.  In
> Shaw (2011) Section 4.3 he describes a modification for handling lower
> bounds, and at first glance it appears that it can be modified for upper
> bounds, hence box constraints since the long-only part is already covered.
> Caveat:  I have only quickly skimmed the first parts of the Shaw papers so
> maybe I am missing something.****
>
> ** **
>
> Doug****
>
> ** **
>
> ** **
>
> *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:
> gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross
> Bennett
> *Sent:* Saturday, July 06, 2013 9:34 AM
> *To:* PortfolioAnalytics
>
> *Subject:* Re: [GSoC-PortA] mapping function****
>
> ** **
>
> All,****
>
> ** **
>
> I have a few thoughts on the hierarchy and how to transform weights that
> violate constraints and would appreciate any input or feedback.****
>
> ** **
>
> The process outlined below is what I am thinking of following for the
> constraint mapping fn_map() function.****
>
>    - Step 1: Test weights vector for violation of min_sum or max_sum. If
>    violated, transform the weights vector with rp_transform() taking into
>    account both leverage and box constraints. Another option is to normalize
>    the weights by transforming the entire vector like what is done
>    constrained_objective()... is one way preferable over the other?****
>    - Step 2: Test weights vector for violation of min or max. If
>    violated, transform the weights vector with rp_transform() taking into
>    account both leverage and box constraints. If we can't generate a feasible
>    portfolio, this is because min or max is too restrictive.  Try relaxing min
>    or max. e.g., if min is violated, we could simply relax min by doing
>    something like min <- min - 0.05 and try this N times to generate a
>    feasible portfolio. Or we could randomly select an element of min and
>    decrease it instead of modifying the entire vector.****
>    - Step 3: Test weights vector for violation of groups, cLO, or cUP. If
>    violated, transform the weights vector with rp_transform() taking into
>    account leverage, box, and group constraints. If we can't generate a
>    feasible portfolio, try relaxing cLO or cUP. e.g., if cLO is violated, we
>    could simply relax cLO by doing something like cLO <- cLO - 0.05 and try
>    this N times to generate a feasible portfolio. Or we could randomly select
>    an element of min and decrease it instead of modifying the entire vector.
>    ****
>    - ** **
>    - Step 4: Test weights vector for violation of max_pos. If violated,
>    transform the weights vector with rp_transform() taking into account
>    leverage, box, group, and position limit constraints.****
>
> Please advise if this is consistent with how you see the hierarchy in the
> constraint mapping function working?****
>
> ** **
>
> Thanks,****
>
> Ross****
>
> ** **
>
> On Tue, Jul 2, 2013 at 10:13 PM, Ross Bennett <rossbennett34 at gmail.com>
> wrote:****
>
> All,****
>
> ** **
>
> I added an rp_constraint() function that uses logic from
> randomize_portfolio() to transform a weights vector element by element to
> satisfy (if possible) leverage *and* box constraints.****
>
> ** **
>
> Here is a summary of where I am at with the mapping function.****
>
>    - Box Constraints (min/max)****
>
>
>    - rp_transform() takes a weight vector that violates either
>       min_sum/max_sum leverage constraints *or* min/max box constraints and
>       returns and weighting vector that satisfies leverage *and* box constraints.
>       ****
>       - txfrm_box_constraint() takes a weight vector that violates
>       min/max box constraints and will set any weight that violates min or max to
>       its min or max respectively. This is too simplistic and does not take into
>       account min_sum/max_sum leverage constraints.****
>       - I think rp_transform() is the better option here... thoughts?****
>
>
>    - Leverage Constraints (min_sum/max_sum)****
>
>
>    - rp_transform() takes a weight vector that violates either
>       min_sum/max_sum leverage constraints *or* min/max box constraints and
>       returns and weighting vector that satisfies leverage *and* box constraints.
>       ****
>       - txfrm_weight_sum_constraint() takes a weight vector that violates
>       min_sum/max_sum leverage constraints and normalizes the entire weights
>       vector to satisfy leverage constraints. This is too simplistic and does not
>       take into account min/max box constraints.****
>       - I think rp_transform() is the better option here... thoughts?****
>
>
>    - Group Constraints (groups, cLO, cUP) ****
>
>
>    - txfrm_group_constraint() loops through the groups and checks if cLO
>       or cUP is violated. If cLO or cUP is violated the weights of the given
>       group are normalized to equal cLO or cUP, whichever is violated. This will
>       likely change the sum of the weights vector and violate min_sum/max_sum so
>       we will have to "re-transform".****
>       - I think txfrm_group_constraint() is a good first step because it
>       gets us close to satisfying the group constraints.****
>       - I'm working on incorporating the group constraints into
>       rp_transform().****
>       - I'm not seeing how to use the eval(parse(text=formula), data) code to
>       evaluate group constraints. Do you have a simple example?****
>
>
>    - Diversification Constraint****
>
>
>    - I'm having a hard time coming up with a straightforward solution to
>       transform the vector of weights to meet the diversification constraint. One
>       idea I was working on was to generate N random portfolios and select the
>       portfolio with the closest diversification value.****
>       - Note that I define diversification as: diversification = 1 -
>       sum(weights^2)****
>       - Would it be better to just handle this *like* an objective and
>       penalize in constrained_objective()?****
>
>
>    - Turnover Constraint****
>
>
>    - I'm having a hard time coming up with a straightforward solution to
>       transform the vector of weights to meet the turnover constraint. One idea I
>       was working on was to generate N random portfolios and select the portfolio
>       with the closest turnover value.****
>       - Would it be better to just handle this *like* an objective and
>       penalize in constrained_objective()?****
>
>
>    - Position Limit Constraint****
>
>
>    - txfrm_position_limit_constraint() sets the nassets - max_pos minimum
>       weights equal to 0.****
>       - An issue is that for any min_i > 0, this will violate the min box
>       constraint and be penalized later. Would it make sense to change min_i to 0
>       for asset_i that is set equal to 0?****
>
> One last thing is that I have the mapping function in a loop to get each
> constraint object. The weights vector will be transformed in the order that
> the constraints were added. In order to honor the hierarchy of how we
> transform the weights vector, this should not be in a loop so we control
> the order of transformation. Is that correct?****
>
> ** **
>
> I look forward to your feedback and comments.****
>
> ** **
>
> Thanks,****
>
> Ross****
>
> ** **
>
> On Sun, Jun 30, 2013 at 11:51 AM, Doug Martin <martinrd at comcast.net>
> wrote:****
>
>  ****
>
>  ****
>
>  ****
>
> -----Original Message-----
> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:
> gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G.
> Peterson****
>
> Sent: Saturday, June 29, 2013 6:45 AM
> To: PortfolioAnalytics
> Subject: [GSoC-PortA] mapping function****
>
>  ****
>
> Based on side conversations with Ross and Peter, I thought I should talk a
> little bit about next steps related to the mapping function.****
>
>  ****
>
> Apologies for the long email, I want to be complete, and I hope that some
> of this can make its way to the documentation.****
>
>  ****
>
> The purpose of the mapping function is to transform a weights vector that
> does not meet all the constraints into a weights vector that does meet the
> constraints, if one exists, hopefully with a minimum of transformation.***
> *
>
>  ****
>
> In the random portfolios code, we've used a couple of techniques pioneered
> by Pat Burns.  The philosophical idea is that your optimum portfolio is
> most likely to exist at the edges of the feasible space.****
>
>  ****
>
> At the first R/Finance conference, Pat used the analogy of a mountain
> lake, where the lake represents the feasible space.  With a combination of
> lots of different constraints, the shore of the lake will not be smooth or
> regular.  The lake (the feasible space) may not take up a large percentage
> of the terrain.****
>
>  ****
>
> If we randomly place a rock anywhere in the terrain, some of them will
> land in the lake, inside the feasible space, but most will land outside, on
> the slopes of the mountains that surround the lake.  The goal should be to
> nudge these towards the shores of the lake (our feasible space).****
>
>  ****
>
> Having exhausted the analogy, let's talk details.****
>
>  ****
>
> A slightly more rigorous treatment of the problem is given here:****
>
> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224****
>
>  ****
>
> *[Doug] This is the 2010 paper, which I just read much of.  Very nice
> paper.  I find Burns papers pretty useless except for pointing out nice
> statistical aspect and promoting PortfolioProbe.   For example in the paper
> you sent, he does not say clearly what he is doing in generating the random
> portfolios and what he means by out-of-sample.  As you once mentioned, I
> guess you got most details through direct conversation with him.*****
>
> * *****
>
> *Then I found the new Shaw 2011 paper at
> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. *****
>
> *It has some nice additional material, e.g., Section 4.3 on lower bounds
> and Section 4.4 on bounded shorting, among other.  I still need to finish
> reading this version.*****
>
> * *****
>
> *In general if you accept random portfolios that violate the constraints,
> what is a good way to: (a) Accept only those within a certain distance,
> with appropriately defined metric,  of the feasible regions (you don?t want
> to consider all infeasible solutions ? I guess this the reason for min.sum,
> max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to
> take a solution nearest to a vertex?*****
>
> * *****
>
> * *****
>
> * *****
>
> It is possible that can use this method directly for random portfolios
> (and that we could add the ectra constraint types to DEoptim).  If so, much
> of the rest of what I'll write here is irrelevant.  I strongly suspect that
> there will be some constraint types that will still need to be 'adjusted'
> via a mapping method like the one laid out below, since a stochastic solver
> will hand us a vector that needs to be transformed at least in part to move
> into the feasible space.  It's alsom not entirely clear to me that the
> methods presented in the paper can satisfy all our constraint types.****
>
>  ****
>
>  ****
>
> I think our first step should be to test each constraint type, in some
> sort of hierarchy, starting with box constraints (almost all solvers
> support box constraints, of course), since some of the other
> transformations will violate the box constraints, and we'll need to
> transform back again.****
>
>  ****
>
> Each constraint can be evaluated as a logical expression against the
> weights vector.  You can see code for doing something similar with time
> series data in the sigFormula function in quantstrat. It takes advantage of
> some base R functionality that can treat an R object (in this case the
> weights vector) as an environment or 'frame'. This allows the columns of
> the data to be addressed without any major manipulation, simply by column
> name (asset name in the weights vector, possibly after adding names back
> in).****
>
>  ****
>
> The code looks something like this:****
>
> eval(parse(text=formula), data)****
>
>  ****
>
> So, 'data' is our weights vector, and 'formula' is an expression that can
> be evaluated as a formula by R.  Evaluating this formula will give us TRUE
> or FALSE to denote whether the weights vector is in compliance or in
> violation of that constraint.  Then, we'll need to transform the weight
> vector, if possible, to comply with that constraint.****
>
>  ****
>
> Specific Cases:****
>
> I've implemented this transformation for box constraints in the random
> portfolios code.  We don't need the evaluation I'll describe next for box
> constraints, because each single weight is handled separately.****
>
>  ****
>
> min_sum and max_sum leverage constraints can be evaluated without using
> the formula, since the formula is simple, and can be expressed in simple R
> code.  The transformation can be accomplished by transforming the entire
> vector.  There's code to do this in both the random portfolios code and in
> constrained_objective.  It is probably preferable to do the transformation
> one weight at a time, as I do in the random portfolios code, to end closer
> to the edges of the feasible space, while continuing to take the box
> constraints into account.****
>
>  ****
>
> linear (in)equality constraints and group constraints can be evaluated
> generically via the formula method I've described above.  Then individual
> weights can be transformed taking the value of the constraint****
>
> (<,>,=) into account (along with the box constraints and leverage
> constraints).****
>
>  ****
>
> and so on...****
>
>  ****
>
> Challenges:****
>
> - recovering the transformed vector from a optimization solver that
> doesn't directly support a mapping function.  I've got some tricks for this
> using environments that we can revisit after we get the basic methodology
> working.****
>
>  ****
>
> -allowing for progressively relaxing constraints when the constraints are
> simply too restrictive.  Perhaps Doug has some documentation on this as
> he's done it in the past, or perhaps we can simply deal with it in the
> penalty part of constrained_objective()****
>
>  ****
>
> Hopefully this was helpful.****
>
>  ****
>
> Regards,****
>
>  ****
>
> Brian****
>
>  ****
>
> --****
>
> Brian G. Peterson****
>
> http://braverock.com/brian/****
>
> Ph: 773-459-4973****
>
> IM: bgpbraverock****
>
> _______________________________________________****
>
> GSoC-PortA mailing list****
>
> GSoC-PortA at lists.r-forge.r-project.org****
>
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta****
>
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta****
>
> ** **
>
> ** **
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130706/8adc0592/attachment-0001.html>

From martinrd at comcast.net  Sun Jul  7 07:04:38 2013
From: martinrd at comcast.net (Doug Martin)
Date: Sat, 6 Jul 2013 22:04:38 -0700
Subject: [GSoC-PortA] mapping function
In-Reply-To: <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
Message-ID: <005101ce7acf$88791a60$996b4f20$@comcast.net>

 

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Saturday, July 06, 2013 1:27 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] mapping function

 

Doug,

 

I'm sure Brian will provide more detail, but hopefully this will provide
some initial clarification about random portfolios and the logic I am using
in rp_transform().

 

The random portfolio weights are being generated with a user specified
leverage constraint, min_sum and max_sum. One could generate a dollar
neutral portfolio by specifying min_sum=-0.01 and max_sum=1.01

[Doug] I guess you meant min_sum = -.01 and max_sum = + .01 for the dollar
neutral case.

or a leveraged portfolio with min_sum=1.49 and max_sum=1.51. I think this
gives more flexibility than the method proposed by Shaw.

[Doug]  Brian and Peter:  I have looked over your nice R-finance 2010 and
2012 presentations and will eventually play around with the R scripts for
the latter.   I also looked again at slides 43-51 in Burns 2009.   And I'm
still trying to figure out what algorithm(s) are actually used in the random
portfolio method in PortfolioAnalytics.   In both 2010 and 2012  you say
>From a portfolio seed, generate random permutations

of weights that meet your constraints on each asset.   However on slide 43
Burns seems to reject (at least that specific form of) random permutations,
and indeed just using random permutations of a fixed seed is not appealing
except maybe for permutation tests.   On slide 44 Burns proposes a sensible
general idea which is apparently what you are using.  But there is no
specificity of the random search method.   I could if necessary figure it
out from the "random" choice code in optimize.portfolio, but it would be a
lot more pleasant if you could point me to, or provide a precise description
of how the algorithm currently works in PortfolioAnalytics.  Then I would
have a solid starting point.

 

A related question:  is the reason for using the min_sum and max_sum window
for obtaining a reasonable number of random weight vectors that
approximately satisfy the desired equality constraint, and  similarly for
other constraints.  I'm guessing that this is not necessary to various
extents with the Shaw method.  For example it seems like the dollar neutral
set of random weights can be obtained by translating the simplex by along
the high-D 45 degree line to the origin and the leverage set of random
weights by translating in the opposite direction.  Of course you would still
have the problem of generating a huge number of random weight vectors to
minimize the constraints violation objective. 

 

I need to spend some more time understanding the Shaw (2011) paper, but it
is unclear to me if Shaw's method will work if we want to specify
constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and max=c(0.85,
0.92, 0.73, 0.75, 0.82) or if it only works for box constraints where all
the values of min and max are equal.

[Doug] I'm not sure it even works for the box constraints with non-zero
lower bound - it looks like you can generate random weight vectors with
either a general set of lower bounds on the simplex or a general set of
upper bounds on the simplex but not both.  I'll read that part again
tomorrow to be sure.

 

P.S. Peter are you on this group list (I guess you are but added you in "To"
just in case you are not).

 

Thanks,

Ross

On Sat, Jul 6, 2013 at 1:31 PM, Doug Martin <martinrd at comcast.net> wrote:

Brian,

 

I don't really understand what is going on, e.g., why do you need min_sum
and max_sum?  I get the impression that the random portfolio weights are
being generated without a full-investment constraint, but with a range of
near-full-investment constraints.  Why is that needed?    If I understand
Shaw (2010) correctly there exists a good method of randomly sampling from
the simplex of positive numbers, i.e., long-only and full-investment, in a
manner that promises to work well in higher dimensions by generating several
sets of random samples ranging from one that is most concentrated at 1/n and
ones that are more concentrated near edges and vertices.  In Shaw (2011)
Section 4.3 he describes a modification for handling lower bounds, and at
first glance it appears that it can be modified for upper bounds, hence box
constraints since the long-only part is already covered.  Caveat:  I have
only quickly skimmed the first parts of the Shaw papers so maybe I am
missing something.

 

Doug

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Saturday, July 06, 2013 9:34 AM
To: PortfolioAnalytics


Subject: Re: [GSoC-PortA] mapping function

 

All,

 

I have a few thoughts on the hierarchy and how to transform weights that
violate constraints and would appreciate any input or feedback.

 

The process outlined below is what I am thinking of following for the
constraint mapping fn_map() function.

*	Step 1: Test weights vector for violation of min_sum or max_sum. If
violated, transform the weights vector with rp_transform() taking into
account both leverage and box constraints. Another option is to normalize
the weights by transforming the entire vector like what is done
constrained_objective()... is one way preferable over the other?
*	Step 2: Test weights vector for violation of min or max. If
violated, transform the weights vector with rp_transform() taking into
account both leverage and box constraints. If we can't generate a feasible
portfolio, this is because min or max is too restrictive.  Try relaxing min
or max. e.g., if min is violated, we could simply relax min by doing
something like min <- min - 0.05 and try this N times to generate a feasible
portfolio. Or we could randomly select an element of min and decrease it
instead of modifying the entire vector.
*	Step 3: Test weights vector for violation of groups, cLO, or cUP. If
violated, transform the weights vector with rp_transform() taking into
account leverage, box, and group constraints. If we can't generate a
feasible portfolio, try relaxing cLO or cUP. e.g., if cLO is violated, we
could simply relax cLO by doing something like cLO <- cLO - 0.05 and try
this N times to generate a feasible portfolio. Or we could randomly select
an element of min and decrease it instead of modifying the entire vector.
*	 
*	Step 4: Test weights vector for violation of max_pos. If violated,
transform the weights vector with rp_transform() taking into account
leverage, box, group, and position limit constraints.

Please advise if this is consistent with how you see the hierarchy in the
constraint mapping function working?

 

Thanks,

Ross

 

On Tue, Jul 2, 2013 at 10:13 PM, Ross Bennett <rossbennett34 at gmail.com>
wrote:

All,

 

I added an rp_constraint() function that uses logic from
randomize_portfolio() to transform a weights vector element by element to
satisfy (if possible) leverage *and* box constraints.

 

Here is a summary of where I am at with the mapping function.

*	Box Constraints (min/max)

*	rp_transform() takes a weight vector that violates either
min_sum/max_sum leverage constraints *or* min/max box constraints and
returns and weighting vector that satisfies leverage *and* box constraints.
*	txfrm_box_constraint() takes a weight vector that violates min/max
box constraints and will set any weight that violates min or max to its min
or max respectively. This is too simplistic and does not take into account
min_sum/max_sum leverage constraints.
*	I think rp_transform() is the better option here... thoughts?

*	Leverage Constraints (min_sum/max_sum)

*	rp_transform() takes a weight vector that violates either
min_sum/max_sum leverage constraints *or* min/max box constraints and
returns and weighting vector that satisfies leverage *and* box constraints.
*	txfrm_weight_sum_constraint() takes a weight vector that violates
min_sum/max_sum leverage constraints and normalizes the entire weights
vector to satisfy leverage constraints. This is too simplistic and does not
take into account min/max box constraints.
*	I think rp_transform() is the better option here... thoughts?

*	Group Constraints (groups, cLO, cUP) 

*	txfrm_group_constraint() loops through the groups and checks if cLO
or cUP is violated. If cLO or cUP is violated the weights of the given group
are normalized to equal cLO or cUP, whichever is violated. This will likely
change the sum of the weights vector and violate min_sum/max_sum so we will
have to "re-transform".
*	I think txfrm_group_constraint() is a good first step because it
gets us close to satisfying the group constraints.
*	I'm working on incorporating the group constraints into
rp_transform().
*	I'm not seeing how to use the eval(parse(text=formula), data) code
to evaluate group constraints. Do you have a simple example?

*	Diversification Constraint

*	I'm having a hard time coming up with a straightforward solution to
transform the vector of weights to meet the diversification constraint. One
idea I was working on was to generate N random portfolios and select the
portfolio with the closest diversification value.
*	Note that I define diversification as: diversification = 1 -
sum(weights^2)
*	Would it be better to just handle this *like* an objective and
penalize in constrained_objective()?

*	Turnover Constraint

*	I'm having a hard time coming up with a straightforward solution to
transform the vector of weights to meet the turnover constraint. One idea I
was working on was to generate N random portfolios and select the portfolio
with the closest turnover value.
*	Would it be better to just handle this *like* an objective and
penalize in constrained_objective()?

*	Position Limit Constraint

*	txfrm_position_limit_constraint() sets the nassets - max_pos minimum
weights equal to 0.
*	An issue is that for any min_i > 0, this will violate the min box
constraint and be penalized later. Would it make sense to change min_i to 0
for asset_i that is set equal to 0?

One last thing is that I have the mapping function in a loop to get each
constraint object. The weights vector will be transformed in the order that
the constraints were added. In order to honor the hierarchy of how we
transform the weights vector, this should not be in a loop so we control the
order of transformation. Is that correct?

 

I look forward to your feedback and comments.

 

Thanks,

Ross

 

On Sun, Jun 30, 2013 at 11:51 AM, Doug Martin <martinrd at comcast.net> wrote:

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson

Sent: Saturday, June 29, 2013 6:45 AM
To: PortfolioAnalytics
Subject: [GSoC-PortA] mapping function

 

Based on side conversations with Ross and Peter, I thought I should talk a
little bit about next steps related to the mapping function.

 

Apologies for the long email, I want to be complete, and I hope that some of
this can make its way to the documentation.

 

The purpose of the mapping function is to transform a weights vector that
does not meet all the constraints into a weights vector that does meet the
constraints, if one exists, hopefully with a minimum of transformation.

 

In the random portfolios code, we've used a couple of techniques pioneered
by Pat Burns.  The philosophical idea is that your optimum portfolio is most
likely to exist at the edges of the feasible space.

 

At the first R/Finance conference, Pat used the analogy of a mountain lake,
where the lake represents the feasible space.  With a combination of lots of
different constraints, the shore of the lake will not be smooth or regular.
The lake (the feasible space) may not take up a large percentage of the
terrain.

 

If we randomly place a rock anywhere in the terrain, some of them will land
in the lake, inside the feasible space, but most will land outside, on the
slopes of the mountains that surround the lake.  The goal should be to nudge
these towards the shores of the lake (our feasible space).

 

Having exhausted the analogy, let's talk details.

 

A slightly more rigorous treatment of the problem is given here:

 <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224>
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224

 

[Doug] This is the 2010 paper, which I just read much of.  Very nice paper.
I find Burns papers pretty useless except for pointing out nice statistical
aspect and promoting PortfolioProbe.   For example in the paper you sent, he
does not say clearly what he is doing in generating the random portfolios
and what he means by out-of-sample.  As you once mentioned, I guess you got
most details through direct conversation with him.

 

Then I found the new Shaw 2011 paper at
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. 

It has some nice additional material, e.g., Section 4.3 on lower bounds and
Section 4.4 on bounded shorting, among other.  I still need to finish
reading this version.

 

In general if you accept random portfolios that violate the constraints,
what is a good way to: (a) Accept only those within a certain distance, with
appropriately defined metric,  of the feasible regions (you don't want to
consider all infeasible solutions - I guess this the reason for min.sum,
max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to
take a solution nearest to a vertex?

 

 

 

It is possible that can use this method directly for random portfolios (and
that we could add the ectra constraint types to DEoptim).  If so, much of
the rest of what I'll write here is irrelevant.  I strongly suspect that
there will be some constraint types that will still need to be 'adjusted'
via a mapping method like the one laid out below, since a stochastic solver
will hand us a vector that needs to be transformed at least in part to move
into the feasible space.  It's alsom not entirely clear to me that the
methods presented in the paper can satisfy all our constraint types.

 

 

I think our first step should be to test each constraint type, in some sort
of hierarchy, starting with box constraints (almost all solvers support box
constraints, of course), since some of the other transformations will
violate the box constraints, and we'll need to transform back again.

 

Each constraint can be evaluated as a logical expression against the weights
vector.  You can see code for doing something similar with time series data
in the sigFormula function in quantstrat. It takes advantage of some base R
functionality that can treat an R object (in this case the weights vector)
as an environment or 'frame'. This allows the columns of the data to be
addressed without any major manipulation, simply by column name (asset name
in the weights vector, possibly after adding names back in).

 

The code looks something like this:

eval(parse(text=formula), data)

 

So, 'data' is our weights vector, and 'formula' is an expression that can be
evaluated as a formula by R.  Evaluating this formula will give us TRUE or
FALSE to denote whether the weights vector is in compliance or in violation
of that constraint.  Then, we'll need to transform the weight vector, if
possible, to comply with that constraint.

 

Specific Cases:

I've implemented this transformation for box constraints in the random
portfolios code.  We don't need the evaluation I'll describe next for box
constraints, because each single weight is handled separately.

 

min_sum and max_sum leverage constraints can be evaluated without using the
formula, since the formula is simple, and can be expressed in simple R code.
The transformation can be accomplished by transforming the entire vector.
There's code to do this in both the random portfolios code and in
constrained_objective.  It is probably preferable to do the transformation
one weight at a time, as I do in the random portfolios code, to end closer
to the edges of the feasible space, while continuing to take the box
constraints into account.

 

linear (in)equality constraints and group constraints can be evaluated
generically via the formula method I've described above.  Then individual
weights can be transformed taking the value of the constraint

(<,>,=) into account (along with the box constraints and leverage
constraints).

 

and so on...

 

Challenges:

- recovering the transformed vector from a optimization solver that doesn't
directly support a mapping function.  I've got some tricks for this using
environments that we can revisit after we get the basic methodology working.

 

-allowing for progressively relaxing constraints when the constraints are
simply too restrictive.  Perhaps Doug has some documentation on this as he's
done it in the past, or perhaps we can simply deal with it in the penalty
part of constrained_objective()

 

Hopefully this was helpful.

 

Regards,

 

Brian

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

 


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130706/b4229319/attachment-0001.html>

From brian at braverock.com  Mon Jul  8 13:59:34 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Mon, 08 Jul 2013 06:59:34 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
Message-ID: <51DAA9A6.4000506@braverock.com>

On 07/06/2013 03:27 PM, Ross Bennett wrote:
> I'm sure Brian will provide more detail, but hopefully this will provide
> some initial clarification about random portfolios and the logic I am
> using in rp_transform().
>
> The random portfolio weights are being generated with a user specified
> leverage constraint, min_sum and max_sum. One could generate a dollar
> neutral portfolio by specifying min_sum=-0.01 and max_sum=1.01 or a
> leveraged portfolio with min_sum=1.49 and max_sum=1.51. I think this
> gives more flexibility than the method proposed by Shaw.

> I need to spend some more time understanding the Shaw (2011) paper, but
> it is unclear to me if Shaw's method will work if we want to specify
> constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and max=c(0.85,
> 0.92, 0.73, 0.75, 0.82) or if it only works for box constraints where
> all the values of min and max are equal.

Shaw's approach is quite interesting, but I think it is also rather 
limited in the types of constraints he examines.

Also, the 'reject everything that doesn't meet the constraints' approach 
isn't practical, in my opinion.  Perhaps in C/C++ code, but even then, 
as the number of assets increases (all his examples are toy ones), the 
number of non-conforming portfolios will also increase.

Doug asked elsewhere on this thread for a concise description of how the 
random portfolios code worked before Ross started improving things (and 
all his changes looks like an improvement to me so far).

___________________________________________________________________

So, the original algorithm, developed by trial and error from my notes 
of a private conversation with Pat:

- generate a uniform random draw satisfying box constraints
   for all assets

- check to see if it is feasible (leverage constraints)
   + assuming it is not feasible,

     randomly choose one weight from the weights vector

     do another random draw for that weight that is larger or smaller 
than the original
     draw to move the portfolio closer to the leverage constraint

^
\--return to check to see if the portfolio is feasible

The Burns-style method 'walks' the portfolio closer to the edges of the 
feasible space, one weight at a time.

It's obviously a bit more complicated than this, but not much.  I think 
it would be good to add a complete outline of the modified algorithm to 
the documentation once we're happy with the approach.
______________________________________________________________________

Also elsewhere in this thread was a discussion of transforming the 
entire vector to meet the leverage constraint.  While this 'works', it 
has two drawbacks.  The first is that you may then violate the box 
constraints.  The second is that you will be further from the edges of 
the feasible space.  It may make sense to write a simple test script  on 
50,100,200,500 assets that either transforms one asset at a time, or 
*first* transforms the entire vector, and then transforms individual assets


Shaw's reliance on vertex combinations between pairs of assets doesn't 
seem as useful to me as the number of assets increases, but perhaps it's 
only because it has been 20 years since I stared at Mathematica code.

Overall, I'd like to implement the Shaw methods, but I'd like that to be 
secondary to getting something that works which has the flexibility to 
handle all the constraint types we've identified.

So far, I think the progress on fn_map has been phenomenal.  For Random 
Portfolios or DEoptim, this should give us what we need to have feasible 
portfolios with any sort of layered constraints.  We'll need to 
integrate the use of fn_map into constrained_objective for solvers that 
don't specify all the constraint types.  I'll try to spend some time 
outlining that later today.

Regards,

   - Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From martinrd at comcast.net  Mon Jul  8 19:51:33 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 8 Jul 2013 10:51:33 -0700
Subject: [GSoC-PortA] mapping function
In-Reply-To: <51DAA9A6.4000506@braverock.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
Message-ID: <004301ce7c03$d24c6090$76e521b0$@comcast.net>

Brian,

Thanks for your helpful comments below.  It is fortunate that we will be
meeting in a couple of weeks, and can discuss in more detail then.

Meanwhile, I am going to concentrate on the "standard" stuff that is needed
for the first few chapters of the 2nd edition.

Doug

P.S. When do you head to Alaska?


-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Monday, July 08, 2013 5:00 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] mapping function

On 07/06/2013 03:27 PM, Ross Bennett wrote:
> I'm sure Brian will provide more detail, but hopefully this will 
> provide some initial clarification about random portfolios and the 
> logic I am using in rp_transform().
>
> The random portfolio weights are being generated with a user specified 
> leverage constraint, min_sum and max_sum. One could generate a dollar 
> neutral portfolio by specifying min_sum=-0.01 and max_sum=1.01 or a 
> leveraged portfolio with min_sum=1.49 and max_sum=1.51. I think this 
> gives more flexibility than the method proposed by Shaw.

> I need to spend some more time understanding the Shaw (2011) paper, 
> but it is unclear to me if Shaw's method will work if we want to 
> specify constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and 
> max=c(0.85, 0.92, 0.73, 0.75, 0.82) or if it only works for box 
> constraints where all the values of min and max are equal.

Shaw's approach is quite interesting, but I think it is also rather limited
in the types of constraints he examines.

Also, the 'reject everything that doesn't meet the constraints' approach
isn't practical, in my opinion.  Perhaps in C/C++ code, but even then, as
the number of assets increases (all his examples are toy ones), the number
of non-conforming portfolios will also increase.

Doug asked elsewhere on this thread for a concise description of how the
random portfolios code worked before Ross started improving things (and all
his changes looks like an improvement to me so far).

___________________________________________________________________

So, the original algorithm, developed by trial and error from my notes of a
private conversation with Pat:

- generate a uniform random draw satisfying box constraints
   for all assets

- check to see if it is feasible (leverage constraints)
   + assuming it is not feasible,

     randomly choose one weight from the weights vector

     do another random draw for that weight that is larger or smaller than
the original
     draw to move the portfolio closer to the leverage constraint

^
\--return to check to see if the portfolio is feasible

The Burns-style method 'walks' the portfolio closer to the edges of the
feasible space, one weight at a time.

It's obviously a bit more complicated than this, but not much.  I think it
would be good to add a complete outline of the modified algorithm to the
documentation once we're happy with the approach.
______________________________________________________________________

Also elsewhere in this thread was a discussion of transforming the entire
vector to meet the leverage constraint.  While this 'works', it has two
drawbacks.  The first is that you may then violate the box constraints.  The
second is that you will be further from the edges of the feasible space.  It
may make sense to write a simple test script  on
50,100,200,500 assets that either transforms one asset at a time, or
*first* transforms the entire vector, and then transforms individual assets


Shaw's reliance on vertex combinations between pairs of assets doesn't seem
as useful to me as the number of assets increases, but perhaps it's only
because it has been 20 years since I stared at Mathematica code.

Overall, I'd like to implement the Shaw methods, but I'd like that to be
secondary to getting something that works which has the flexibility to
handle all the constraint types we've identified.

So far, I think the progress on fn_map has been phenomenal.  For Random
Portfolios or DEoptim, this should give us what we need to have feasible
portfolios with any sort of layered constraints.  We'll need to integrate
the use of fn_map into constrained_objective for solvers that don't specify
all the constraint types.  I'll try to spend some time outlining that later
today.

Regards,

   - Brian

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


From brian at braverock.com  Mon Jul  8 19:54:02 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Mon, 08 Jul 2013 12:54:02 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <004301ce7c03$d24c6090$76e521b0$@comcast.net>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
 <004301ce7c03$d24c6090$76e521b0$@comcast.net>
Message-ID: <51DAFCBA.1080409@braverock.com>

On 07/08/2013 12:51 PM, Doug Martin wrote:
> Brian,
>
> Thanks for your helpful comments below.  It is fortunate that we will be
> meeting in a couple of weeks, and can discuss in more detail then.
>
> Meanwhile, I am going to concentrate on the "standard" stuff that is needed
> for the first few chapters of the 2nd edition.

It would be helpful to have a list, especially of anything that differs 
from the first few chapters of the first edition.

> P.S. When do you head to Alaska?

I fly to seattle on Saturday, and Will be essentially out of contact, 
except for occasional moments in a wifi coffee shop, from Sun-Sun.  I 
arrive back in Seattle Sunday the 21st in the AM.

Regards,

Brian



> -----Original Message-----
> From: gsoc-porta-bounces at lists.r-forge.r-project.org
> [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
> G. Peterson
> Sent: Monday, July 08, 2013 5:00 AM
> To: gsoc-porta at r-forge.wu-wien.ac.at
> Subject: Re: [GSoC-PortA] mapping function
>
> On 07/06/2013 03:27 PM, Ross Bennett wrote:
>> I'm sure Brian will provide more detail, but hopefully this will
>> provide some initial clarification about random portfolios and the
>> logic I am using in rp_transform().
>>
>> The random portfolio weights are being generated with a user specified
>> leverage constraint, min_sum and max_sum. One could generate a dollar
>> neutral portfolio by specifying min_sum=-0.01 and max_sum=1.01 or a
>> leveraged portfolio with min_sum=1.49 and max_sum=1.51. I think this
>> gives more flexibility than the method proposed by Shaw.
>
>> I need to spend some more time understanding the Shaw (2011) paper,
>> but it is unclear to me if Shaw's method will work if we want to
>> specify constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and
>> max=c(0.85, 0.92, 0.73, 0.75, 0.82) or if it only works for box
>> constraints where all the values of min and max are equal.
>
> Shaw's approach is quite interesting, but I think it is also rather limited
> in the types of constraints he examines.
>
> Also, the 'reject everything that doesn't meet the constraints' approach
> isn't practical, in my opinion.  Perhaps in C/C++ code, but even then, as
> the number of assets increases (all his examples are toy ones), the number
> of non-conforming portfolios will also increase.
>
> Doug asked elsewhere on this thread for a concise description of how the
> random portfolios code worked before Ross started improving things (and all
> his changes looks like an improvement to me so far).
>
> ___________________________________________________________________
>
> So, the original algorithm, developed by trial and error from my notes of a
> private conversation with Pat:
>
> - generate a uniform random draw satisfying box constraints
>     for all assets
>
> - check to see if it is feasible (leverage constraints)
>     + assuming it is not feasible,
>
>       randomly choose one weight from the weights vector
>
>       do another random draw for that weight that is larger or smaller than
> the original
>       draw to move the portfolio closer to the leverage constraint
>
> ^
> \--return to check to see if the portfolio is feasible
>
> The Burns-style method 'walks' the portfolio closer to the edges of the
> feasible space, one weight at a time.
>
> It's obviously a bit more complicated than this, but not much.  I think it
> would be good to add a complete outline of the modified algorithm to the
> documentation once we're happy with the approach.
> ______________________________________________________________________
>
> Also elsewhere in this thread was a discussion of transforming the entire
> vector to meet the leverage constraint.  While this 'works', it has two
> drawbacks.  The first is that you may then violate the box constraints.  The
> second is that you will be further from the edges of the feasible space.  It
> may make sense to write a simple test script  on
> 50,100,200,500 assets that either transforms one asset at a time, or
> *first* transforms the entire vector, and then transforms individual assets
>
>
> Shaw's reliance on vertex combinations between pairs of assets doesn't seem
> as useful to me as the number of assets increases, but perhaps it's only
> because it has been 20 years since I stared at Mathematica code.
>
> Overall, I'd like to implement the Shaw methods, but I'd like that to be
> secondary to getting something that works which has the flexibility to
> handle all the constraint types we've identified.
>
> So far, I think the progress on fn_map has been phenomenal.  For Random
> Portfolios or DEoptim, this should give us what we need to have feasible
> portfolios with any sort of layered constraints.  We'll need to integrate
> the use of fn_map into constrained_objective for solvers that don't specify
> all the constraint types.  I'll try to spend some time outlining that later
> today.
>
> Regards,
>
>     - Brian
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From martinrd at comcast.net  Mon Jul  8 19:58:38 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 8 Jul 2013 10:58:38 -0700
Subject: [GSoC-PortA] mapping function
In-Reply-To: <51DAFCBA.1080409@braverock.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
 <004301ce7c03$d24c6090$76e521b0$@comcast.net>
 <51DAFCBA.1080409@braverock.com>
Message-ID: <005101ce7c04$d2988f50$77c9adf0$@comcast.net>

 

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Monday, July 08, 2013 10:54 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] mapping function

 

On 07/08/2013 12:51 PM, Doug Martin wrote:

> Brian,

> 

> Thanks for your helpful comments below.  It is fortunate that we will 

> be meeting in a couple of weeks, and can discuss in more detail then.

> 

> Meanwhile, I am going to concentrate on the "standard" stuff that is 

> needed for the first few chapters of the 2nd edition.

 

It would be helpful to have a list, especially of anything that differs from
the first few chapters of the first edition.

[Doug] The first few chapters will be quite different.  By Wed. I  will send
a TOC with overview comments. 

 

> P.S. When do you head to Alaska?

 

I fly to seattle on Saturday, and Will be essentially out of contact, except
for occasional moments in a wifi coffee shop, from Sun-Sun.  I arrive back
in Seattle Sunday the 21st in the AM.

 

Regards,

 

Brian

 

 

 

> -----Original Message-----

> From:  <mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>
gsoc-porta-bounces at lists.r-forge.r-project.org

> [ <mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>
mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of 

> Brian G. Peterson

> Sent: Monday, July 08, 2013 5:00 AM

> To:  <mailto:gsoc-porta at r-forge.wu-wien.ac.at>
gsoc-porta at r-forge.wu-wien.ac.at

> Subject: Re: [GSoC-PortA] mapping function

> 

> On 07/06/2013 03:27 PM, Ross Bennett wrote:

>> I'm sure Brian will provide more detail, but hopefully this will 

>> provide some initial clarification about random portfolios and the 

>> logic I am using in rp_transform().

>> 

>> The random portfolio weights are being generated with a user 

>> specified leverage constraint, min_sum and max_sum. One could 

>> generate a dollar neutral portfolio by specifying min_sum=-0.01 and 

>> max_sum=1.01 or a leveraged portfolio with min_sum=1.49 and 

>> max_sum=1.51. I think this gives more flexibility than the method
proposed by Shaw.

> 

>> I need to spend some more time understanding the Shaw (2011) paper, 

>> but it is unclear to me if Shaw's method will work if we want to 

>> specify constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and 

>> max=c(0.85, 0.92, 0.73, 0.75, 0.82) or if it only works for box 

>> constraints where all the values of min and max are equal.

> 

> Shaw's approach is quite interesting, but I think it is also rather 

> limited in the types of constraints he examines.

> 

> Also, the 'reject everything that doesn't meet the constraints' 

> approach isn't practical, in my opinion.  Perhaps in C/C++ code, but 

> even then, as the number of assets increases (all his examples are toy 

> ones), the number of non-conforming portfolios will also increase.

> 

> Doug asked elsewhere on this thread for a concise description of how 

> the random portfolios code worked before Ross started improving things 

> (and all his changes looks like an improvement to me so far).

> 

> ___________________________________________________________________

> 

> So, the original algorithm, developed by trial and error from my notes 

> of a private conversation with Pat:

> 

> - generate a uniform random draw satisfying box constraints

>     for all assets

> 

> - check to see if it is feasible (leverage constraints)

>     + assuming it is not feasible,

> 

>       randomly choose one weight from the weights vector

> 

>       do another random draw for that weight that is larger or smaller 

> than the original

>       draw to move the portfolio closer to the leverage constraint

> 

> ^

> \--return to check to see if the portfolio is feasible

> 

> The Burns-style method 'walks' the portfolio closer to the edges of 

> the feasible space, one weight at a time.

> 

> It's obviously a bit more complicated than this, but not much.  I 

> think it would be good to add a complete outline of the modified 

> algorithm to the documentation once we're happy with the approach.

> ______________________________________________________________________

> 

> Also elsewhere in this thread was a discussion of transforming the 

> entire vector to meet the leverage constraint.  While this 'works', it 

> has two drawbacks.  The first is that you may then violate the box 

> constraints.  The second is that you will be further from the edges of 

> the feasible space.  It may make sense to write a simple test script  

> on

> 50,100,200,500 assets that either transforms one asset at a time, or

> *first* transforms the entire vector, and then transforms individual 

> assets

> 

> 

> Shaw's reliance on vertex combinations between pairs of assets doesn't 

> seem as useful to me as the number of assets increases, but perhaps 

> it's only because it has been 20 years since I stared at Mathematica code.

> 

> Overall, I'd like to implement the Shaw methods, but I'd like that to 

> be secondary to getting something that works which has the flexibility 

> to handle all the constraint types we've identified.

> 

> So far, I think the progress on fn_map has been phenomenal.  For 

> Random Portfolios or DEoptim, this should give us what we need to have 

> feasible portfolios with any sort of layered constraints.  We'll need 

> to integrate the use of fn_map into constrained_objective for solvers 

> that don't specify all the constraint types.  I'll try to spend some 

> time outlining that later today.

> 

> Regards,

> 

>     - Brian

> 

> --

> Brian G. Peterson

>  <http://braverock.com/brian/> http://braverock.com/brian/

> Ph: 773-459-4973

> IM: bgpbraverock

> _______________________________________________

> GSoC-PortA mailing list

>  <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

>  <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

> 

> _______________________________________________

> GSoC-PortA mailing list

>  <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

>  <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

> 

 

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130708/53af5597/attachment-0001.html>

From martinrd at comcast.net  Mon Jul  8 19:58:38 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 8 Jul 2013 10:58:38 -0700
Subject: [GSoC-PortA] mapping function
In-Reply-To: <51DAFCBA.1080409@braverock.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
 <004301ce7c03$d24c6090$76e521b0$@comcast.net>
 <51DAFCBA.1080409@braverock.com>
Message-ID: <005101ce7c04$d2988f50$77c9adf0$@comcast.net>

 

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Monday, July 08, 2013 10:54 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] mapping function

 

On 07/08/2013 12:51 PM, Doug Martin wrote:

> Brian,

> 

> Thanks for your helpful comments below.  It is fortunate that we will 

> be meeting in a couple of weeks, and can discuss in more detail then.

> 

> Meanwhile, I am going to concentrate on the "standard" stuff that is 

> needed for the first few chapters of the 2nd edition.

 

It would be helpful to have a list, especially of anything that differs from
the first few chapters of the first edition.

[Doug] The first few chapters will be quite different.  By Wed. I  will send
a TOC with overview comments. 

 

> P.S. When do you head to Alaska?

 

I fly to seattle on Saturday, and Will be essentially out of contact, except
for occasional moments in a wifi coffee shop, from Sun-Sun.  I arrive back
in Seattle Sunday the 21st in the AM.

 

Regards,

 

Brian

 

 

 

> -----Original Message-----

> From:  <mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>
gsoc-porta-bounces at lists.r-forge.r-project.org

> [ <mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>
mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of 

> Brian G. Peterson

> Sent: Monday, July 08, 2013 5:00 AM

> To:  <mailto:gsoc-porta at r-forge.wu-wien.ac.at>
gsoc-porta at r-forge.wu-wien.ac.at

> Subject: Re: [GSoC-PortA] mapping function

> 

> On 07/06/2013 03:27 PM, Ross Bennett wrote:

>> I'm sure Brian will provide more detail, but hopefully this will 

>> provide some initial clarification about random portfolios and the 

>> logic I am using in rp_transform().

>> 

>> The random portfolio weights are being generated with a user 

>> specified leverage constraint, min_sum and max_sum. One could 

>> generate a dollar neutral portfolio by specifying min_sum=-0.01 and 

>> max_sum=1.01 or a leveraged portfolio with min_sum=1.49 and 

>> max_sum=1.51. I think this gives more flexibility than the method
proposed by Shaw.

> 

>> I need to spend some more time understanding the Shaw (2011) paper, 

>> but it is unclear to me if Shaw's method will work if we want to 

>> specify constraints such as min=c(0.05, 0.12, 0.15, 0.05, 0.08) and 

>> max=c(0.85, 0.92, 0.73, 0.75, 0.82) or if it only works for box 

>> constraints where all the values of min and max are equal.

> 

> Shaw's approach is quite interesting, but I think it is also rather 

> limited in the types of constraints he examines.

> 

> Also, the 'reject everything that doesn't meet the constraints' 

> approach isn't practical, in my opinion.  Perhaps in C/C++ code, but 

> even then, as the number of assets increases (all his examples are toy 

> ones), the number of non-conforming portfolios will also increase.

> 

> Doug asked elsewhere on this thread for a concise description of how 

> the random portfolios code worked before Ross started improving things 

> (and all his changes looks like an improvement to me so far).

> 

> ___________________________________________________________________

> 

> So, the original algorithm, developed by trial and error from my notes 

> of a private conversation with Pat:

> 

> - generate a uniform random draw satisfying box constraints

>     for all assets

> 

> - check to see if it is feasible (leverage constraints)

>     + assuming it is not feasible,

> 

>       randomly choose one weight from the weights vector

> 

>       do another random draw for that weight that is larger or smaller 

> than the original

>       draw to move the portfolio closer to the leverage constraint

> 

> ^

> \--return to check to see if the portfolio is feasible

> 

> The Burns-style method 'walks' the portfolio closer to the edges of 

> the feasible space, one weight at a time.

> 

> It's obviously a bit more complicated than this, but not much.  I 

> think it would be good to add a complete outline of the modified 

> algorithm to the documentation once we're happy with the approach.

> ______________________________________________________________________

> 

> Also elsewhere in this thread was a discussion of transforming the 

> entire vector to meet the leverage constraint.  While this 'works', it 

> has two drawbacks.  The first is that you may then violate the box 

> constraints.  The second is that you will be further from the edges of 

> the feasible space.  It may make sense to write a simple test script  

> on

> 50,100,200,500 assets that either transforms one asset at a time, or

> *first* transforms the entire vector, and then transforms individual 

> assets

> 

> 

> Shaw's reliance on vertex combinations between pairs of assets doesn't 

> seem as useful to me as the number of assets increases, but perhaps 

> it's only because it has been 20 years since I stared at Mathematica code.

> 

> Overall, I'd like to implement the Shaw methods, but I'd like that to 

> be secondary to getting something that works which has the flexibility 

> to handle all the constraint types we've identified.

> 

> So far, I think the progress on fn_map has been phenomenal.  For 

> Random Portfolios or DEoptim, this should give us what we need to have 

> feasible portfolios with any sort of layered constraints.  We'll need 

> to integrate the use of fn_map into constrained_objective for solvers 

> that don't specify all the constraint types.  I'll try to spend some 

> time outlining that later today.

> 

> Regards,

> 

>     - Brian

> 

> --

> Brian G. Peterson

>  <http://braverock.com/brian/> http://braverock.com/brian/

> Ph: 773-459-4973

> IM: bgpbraverock

> _______________________________________________

> GSoC-PortA mailing list

>  <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

>  <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

> 

> _______________________________________________

> GSoC-PortA mailing list

>  <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

>  <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

> 

 

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130708/53af5597/attachment-0002.html>

From rossbennett34 at gmail.com  Wed Jul 10 01:14:26 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Tue, 9 Jul 2013 18:14:26 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <51DAA9A6.4000506@braverock.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
Message-ID: <CAEESW=Uw6nZ347=N6mtkgs-geY-S-zUdmH25P=xr2tWT2rThOQ@mail.gmail.com>

On Mon, Jul 8, 2013 at 6:59 AM, Brian G. Peterson <brian at braverock.com>wrote:
>
> The Burns-style method 'walks' the portfolio closer to the edges of the
> feasible space, one weight at a time.
>
> It's obviously a bit more complicated than this, but not much.  I think it
> would be good to add a complete outline of the modified algorithm to the
> documentation once we're happy with the approach.
>

I started working on a mapping function vignette that will document the
process of the mapping function and outline the algorithm for rp_transform.


> Also elsewhere in this thread was a discussion of transforming the entire
> vector to meet the leverage constraint.  While this 'works', it has two
> drawbacks.  The first is that you may then violate the box constraints.
>  The second is that you will be further from the edges of the feasible
> space.  It may make sense to write a simple test script  on 50,100,200,500
> assets that either transforms one asset at a time, or *first* transforms
> the entire vector, and then transforms individual assets
>

I will write the simple test script per your comments to compare the two
methods. I agree with you that the major downside is that the box
constraints will likely be violated. it is also likely that group
constraint will be violated and the weights vector will just have to be
transformed again to satisfy the constraints.

Shaw's reliance on vertex combinations between pairs of assets doesn't seem
> as useful to me as the number of assets increases, but perhaps it's only
> because it has been 20 years since I stared at Mathematica code.
>
> Overall, I'd like to implement the Shaw methods, but I'd like that to be
> secondary to getting something that works which has the flexibility to
> handle all the constraint types we've identified.
>

Over the weekend I wrote a script to replicate the results in section 4.5
and 5.1 of the Shaw 2011 paper and I got results that were very close to
his. I put the script in the sandbox folder and named it rp_shaw.R. It
would be interesting to include the Shaw method with the caveat to the user
that it only works for limited constraints. I'll shelve this until later.


> So far, I think the progress on fn_map has been phenomenal.  For Random
> Portfolios or DEoptim, this should give us what we need to have feasible
> portfolios with any sort of layered constraints.  We'll need to integrate
> the use of fn_map into constrained_objective for solvers that don't specify
> all the constraint types.  I'll try to spend some time outlining that later
> today.
>

Looking forward to seeing your outline of integrating fn_map into
constrained_objective... it's interesting seeing the pieces come together.
Are you happy with where fn_map is at for now or are there any more changes
you would like to see?

Thanks,
Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130709/d0bd21c2/attachment.html>

From rossbennett34 at gmail.com  Wed Jul 10 04:17:49 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Tue, 9 Jul 2013 21:17:49 -0500
Subject: [GSoC-PortA] mapping function
In-Reply-To: <CAEESW=Uw6nZ347=N6mtkgs-geY-S-zUdmH25P=xr2tWT2rThOQ@mail.gmail.com>
References: <51CEE4E7.8010106@braverock.com>
 <001801ce75b2$280c8260$78258720$@comcast.net>
 <CAEESW=WmJftM+hSkM3EYv=D4ykRsQ3ZmEJxHexmyWnWZpK+i=A@mail.gmail.com>
 <CAEESW=VkdWE9idyaB8PVr6xcOnWVv0xwJ+Y+C4von7qETnivuQ@mail.gmail.com>
 <002f01ce7a77$17a095a0$46e1c0e0$@comcast.net>
 <CAEESW=Vy=vVmY7BbA8rG+PKV2JcC3LPbCBe9nRAeLkt51tHNpw@mail.gmail.com>
 <51DAA9A6.4000506@braverock.com>
 <CAEESW=Uw6nZ347=N6mtkgs-geY-S-zUdmH25P=xr2tWT2rThOQ@mail.gmail.com>
Message-ID: <CAEESW=XWDTey+P=0B8hUAyHrP=3jXmbb7chzwkbz9PYRAWoGqg@mail.gmail.com>

On Tue, Jul 9, 2013 at 6:14 PM, Ross Bennett <rossbennett34 at gmail.com>wrote:

>
> On Mon, Jul 8, 2013 at 6:59 AM, Brian G. Peterson <brian at braverock.com>wrote:
>
>> Also elsewhere in this thread was a discussion of transforming the entire
>> vector to meet the leverage constraint.  While this 'works', it has two
>> drawbacks.  The first is that you may then violate the box constraints.
>>  The second is that you will be further from the edges of the feasible
>> space.  It may make sense to write a simple test script  on 50,100,200,500
>> assets that either transforms one asset at a time, or *first* transforms
>> the entire vector, and then transforms individual assets
>>
>
> I will write the simple test script per your comments to compare the two
> methods. I agree with you that the major downside is that the box
> constraints will likely be violated. it is also likely that group
> constraint will be violated and the weights vector will just have to be
> transformed again to satisfy the constraints.
>

I added a script to the sandbox folder called
leverage_transformation_testing.R. I tested a handful of scenarios for
different box constraints and starting values for the sum of weights. What
I found is that the success rate is pretty sensitive to the box constraints
and starting value of the sum of portfolio weights. Success rate defined as
the percentage of portfolios that still meet box constraints after
transforming the whole vector to meet leverage constraints.

 - 95%+ success rate when the min vector of lower bounds for box
constraints is all 0s

 - For the case of 250 assets, if I set just 10 elements of the min vector
equal to 0.01    the success rate drops from 96% to 37%... very sensitive
to positive elements in the min vector.

I could do some more exhaustive testing, but I'm not sure how much we'll
gain. Even if we transform the entire vector first, some of the time we'll
have to transform again to satisfy box constraints. How often "some of the
time" is could vary from 4% to 63% of the time based on the testing I did.
I think the two drawbacks you listed above outweigh the time we'll save by
not having to transform individual elements some of the time to satisfy
leverage constraints.

Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130709/2f8c831e/attachment.html>

From brian at braverock.com  Thu Jul 11 19:13:33 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Thu, 11 Jul 2013 12:13:33 -0500
Subject: [GSoC-PortA] Incorporating New Portfolio Object
In-Reply-To: <CAEESW=V42SDUejdNJAOrsg5beip6Cvirow+gKBxbVaMbHQ_v_w@mail.gmail.com>
References: <CAEESW=V42SDUejdNJAOrsg5beip6Cvirow+gKBxbVaMbHQ_v_w@mail.gmail.com>
Message-ID: <51DEE7BD.1010407@braverock.com>

On 07/10/2013 07:54 PM, Ross Bennett wrote:
> I'd like to begin work on making changes to the code to accept the new
> portfolio object. My plan is to first make the changes to
> constrained_objective, optimize.portfolio, and random_portfolios to
> accept a portfolio object instead of a constraint object. I'll make "v2"
> versions of the functions so that I can test against the current
> versions. If our next step is to integrate the mapping function into
>   constrained_objective, I think it would be good to have a working
> version of constrained_objective that accepts a portfolio object because
> fn_map also accepts a portfolio object.
>
> Is this a good time to make these changes? Any thoughts/concerns?

As I already said in a private email, and as Ross has already started 
doing, I think we're in a good position to move forward to the _v2 steps 
that Ross has proposed.

I promised to talk about integrating fn_map into constrained_objective().

There are a handful of potential pitfalls from modifying the weights 
vector inside the objective function.

The largest of these is that the optimizer won't know what the 'real' 
weights (parameters) vector is.  A related pitfall is that we need to 
know what the 'right' (adjusted) weighs vector is after we adjust it.

The solution to this lies in the 'store_output' option.  If store_output 
is TRUE, constrained_objective() will store the (transformed/adjusted) 
weights vector.

That is just a list.  We'd need to add a slot to that list for the 
un-transformed weights vector, but that's about it.  (feel free to add 
anything else you think might be useful too).

on line 95 of the current code (constrained_objective.R r2538), on line 
95, starts a block for the fn parameters 'normalize'.  I think that 
integrating fn_map means replacing the body of the if block from line 95 
to the else clause on line 118 can be replaced with an appropriate call 
to fn_map.

Some additional wrinkles:

optimize.portfolio.rebalancing:
the environment for storing the extra output will exist on each of the 
workers, we'll need to write a custom .combine function for foreach to 
collect all the extra info and put it in the master process' 
environment, but we can cover that after it works for the main case.

DEoptim (in parallel mode):
DEoptim's parallel mode may also need a custom .combine function for its 
calls to foreach.

Other:
As a reminder, I'll have very limited access to email from Sun Jul 14th 
to Sun Jul 21st.  From the 21st-224th I'll be spending time with Doug 
and Guy, and expect to spend a lot of time talking and thinking about 
PortfolioAnalytics.

Regards,

Brian


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Thu Jul 11 21:27:58 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Thu, 11 Jul 2013 14:27:58 -0500
Subject: [GSoC-PortA] Incorporating New Portfolio Object
In-Reply-To: <51DEE7BD.1010407@braverock.com>
References: <CAEESW=V42SDUejdNJAOrsg5beip6Cvirow+gKBxbVaMbHQ_v_w@mail.gmail.com>
 <51DEE7BD.1010407@braverock.com>
Message-ID: <CAEESW=Ut4OjE12d5kMK9ts=YJ=0qb3mp9ysFW72tNe3uN-V-Vg@mail.gmail.com>

Brian,

Thanks for the insight and thoughts into making fn_map work with
constrained_objective.

Just to clarify a few points and make sure my understanding is correct. I
am mainly thinking in terms of how this will apply to DEoptim and random
for optimize methods.

constrained_objective is the "fn" argument to DEoptim. The function "fn"
needs to return a single argument, which in our case is the return value
from constrained_objective, out. The store_output object is useful for us
to track the weights vector before and after it is transformed at each
iteration, but does not affect the optimization.

The transformed weights from the output of fn_map will be used to calculate
the objective measures and the return value, out. Does the optimizer, e.g.
DEoptim, keep track of the adjusted weights or the original weights?

Thanks,
Ross

On Thu, Jul 11, 2013 at 12:13 PM, Brian G. Peterson <brian at braverock.com>wrote:

> On 07/10/2013 07:54 PM, Ross Bennett wrote:
>
>> I'd like to begin work on making changes to the code to accept the new
>> portfolio object. My plan is to first make the changes to
>> constrained_objective, optimize.portfolio, and random_portfolios to
>> accept a portfolio object instead of a constraint object. I'll make "v2"
>> versions of the functions so that I can test against the current
>> versions. If our next step is to integrate the mapping function into
>>   constrained_objective, I think it would be good to have a working
>> version of constrained_objective that accepts a portfolio object because
>> fn_map also accepts a portfolio object.
>>
>> Is this a good time to make these changes? Any thoughts/concerns?
>>
>
> As I already said in a private email, and as Ross has already started
> doing, I think we're in a good position to move forward to the _v2 steps
> that Ross has proposed.
>
> I promised to talk about integrating fn_map into constrained_objective().
>
> There are a handful of potential pitfalls from modifying the weights
> vector inside the objective function.
>
> The largest of these is that the optimizer won't know what the 'real'
> weights (parameters) vector is.  A related pitfall is that we need to know
> what the 'right' (adjusted) weighs vector is after we adjust it.
>
> The solution to this lies in the 'store_output' option.  If store_output
> is TRUE, constrained_objective() will store the (transformed/adjusted)
> weights vector.
>
> That is just a list.  We'd need to add a slot to that list for the
> un-transformed weights vector, but that's about it.  (feel free to add
> anything else you think might be useful too).
>
> on line 95 of the current code (constrained_objective.R r2538), on line
> 95, starts a block for the fn parameters 'normalize'.  I think that
> integrating fn_map means replacing the body of the if block from line 95 to
> the else clause on line 118 can be replaced with an appropriate call to
> fn_map.
>
> Some additional wrinkles:
>
> optimize.portfolio.**rebalancing:
> the environment for storing the extra output will exist on each of the
> workers, we'll need to write a custom .combine function for foreach to
> collect all the extra info and put it in the master process' environment,
> but we can cover that after it works for the main case.
>
> DEoptim (in parallel mode):
> DEoptim's parallel mode may also need a custom .combine function for its
> calls to foreach.
>
> Other:
> As a reminder, I'll have very limited access to email from Sun Jul 14th to
> Sun Jul 21st.  From the 21st-224th I'll be spending time with Doug and Guy,
> and expect to spend a lot of time talking and thinking about
> PortfolioAnalytics.
>
>
> Regards,
>
> Brian
>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130711/1cb50302/attachment.html>

From brian at braverock.com  Thu Jul 11 22:28:17 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Thu, 11 Jul 2013 15:28:17 -0500
Subject: [GSoC-PortA] Incorporating New Portfolio Object
In-Reply-To: <CAEESW=Ut4OjE12d5kMK9ts=YJ=0qb3mp9ysFW72tNe3uN-V-Vg@mail.gmail.com>
References: <CAEESW=V42SDUejdNJAOrsg5beip6Cvirow+gKBxbVaMbHQ_v_w@mail.gmail.com>
 <51DEE7BD.1010407@braverock.com>
 <CAEESW=Ut4OjE12d5kMK9ts=YJ=0qb3mp9ysFW72tNe3uN-V-Vg@mail.gmail.com>
Message-ID: <51DF1561.9060005@braverock.com>

On 07/11/2013 02:27 PM, Ross Bennett wrote:
> Thanks for the insight and thoughts into making fn_map work with
> constrained_objective.
>
> Just to clarify a few points and make sure my understanding is
> correct. I am mainly thinking in terms of how this will apply to
> DEoptim and random for optimize methods.

For random portfolios and DEoptim, if you call fn_map inside
constrained_objective, you'll either do nothing, or you'll try to
re-adjust for any constraints which you had to relaxe the first time
through.

That bears expanding on...

In the random portfolios code, you're using fn_map when *generating* the
random portfolios.  So the weights vector is already 'as good as we
could manage'.  In this case, I'd assume that we'd then force
normalize=FALSE when calling constrained_objective().


> constrained_objective is the "fn" argument to DEoptim. The function
> "fn" needs to return a single argument, which in our case is the
> return value from constrained_objective, out.

For the DEoptim solver, we should be passing fnMap=fn_map in 
optimize.portfolio.  This will let DEoptim apply fn_map to the 
population of weight vectors in each generation.  (we may need a wrapper 
around fn_map that takes a multi-row matrix)  DEoptim will then use 
these pre-transformed vectors when calling constrained_objective(), so 
again, normalize=FALSE should be what we use from optimize.portfolio.

> The store_output object is useful for us to track the weights vector
> before and after it is transformed at each iteration, but does not
> affect the optimization.

Correct.

If we use normalize=TRUE, the *solver* will not know that we've 
internally transformed the weights vector to map to our allowed 
constraints.  In some cases, penalizing the weights vector for how much 
it fails to meet the constriants when normalize=FALSE will help guide 
the solver to make better decisions, without having to transform/map the 
weights vector (especially when the feasible space is large).  In other 
cases, penalizing the original weights vector won't help at all, 
typically when the feasible space is very irregular, there are lots of 
assets, or the optimizer has a limited knowledge of what's going on 
inside the objective function.

So, for all the other solvers, I think we need to preserve the option 
for the user to either use the mapping function fn_map with 
normalize=TRUE, or to not use them, and to then penalizethe weights vector.

I guess this suggests that the normalize=FALSE section of 
constrained_objective() needs to change as well to penalize based on the 
degree that the weights vector violates the constraints.  It already 
does this for leverage constraints, but not for the other constraint 
types. I think that there may be an argument to be made to *always* 
penalize, because the constraints may have been relaxed somewhat.  In 
that case, the 'else' clause of the normalise=TRUE/FALSE if/else block 
may come out of the else and just always be execute the penalty for 
weights that violate each constraint.

> The transformed weights from the output of fn_map will be used to
> calculate the objective measures and the return value, out. Does the
> optimizer, e.g. DEoptim, keep track of the adjusted weights or the
> original weights?

The optimizer will only keep track of the weights it sends in, and the 
singular value of 'out'.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Thu Jul 11 23:27:45 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Thu, 11 Jul 2013 16:27:45 -0500
Subject: [GSoC-PortA] Incorporating New Portfolio Object
In-Reply-To: <51DF1561.9060005@braverock.com>
References: <CAEESW=V42SDUejdNJAOrsg5beip6Cvirow+gKBxbVaMbHQ_v_w@mail.gmail.com>
 <51DEE7BD.1010407@braverock.com>
 <CAEESW=Ut4OjE12d5kMK9ts=YJ=0qb3mp9ysFW72tNe3uN-V-Vg@mail.gmail.com>
 <51DF1561.9060005@braverock.com>
Message-ID: <CAEESW=W6jwyC_wVMp_Gij+D3X1oZWbMCJuLjbsyvZfLE88CAgA@mail.gmail.com>

On Thu, Jul 11, 2013 at 3:28 PM, Brian G. Peterson <brian at braverock.com>wrote:

> On 07/11/2013 02:27 PM, Ross Bennett wrote:
>
>> Thanks for the insight and thoughts into making fn_map work with
>> constrained_objective.
>>
>> Just to clarify a few points and make sure my understanding is
>> correct. I am mainly thinking in terms of how this will apply to
>> DEoptim and random for optimize methods.
>>
>
> For random portfolios and DEoptim, if you call fn_map inside
> constrained_objective, you'll either do nothing, or you'll try to
> re-adjust for any constraints which you had to relaxe the first time
> through.
>
> That bears expanding on...
>
> In the random portfolios code, you're using fn_map when *generating* the
> random portfolios.  So the weights vector is already 'as good as we
> could manage'.  In this case, I'd assume that we'd then force
> normalize=FALSE when calling constrained_objective().
>
> Got it. The only change I made to random_portfolios_v2() and
randomize_portfolio_v2() was to accept a portfolio object. I will add
fn_map() to randomize_portfolio_v2().


>
>  constrained_objective is the "fn" argument to DEoptim. The function
>> "fn" needs to return a single argument, which in our case is the
>> return value from constrained_objective, out.
>>
>
> For the DEoptim solver, we should be passing fnMap=fn_map in
> optimize.portfolio.  This will let DEoptim apply fn_map to the population
> of weight vectors in each generation.  (we may need a wrapper around fn_map
> that takes a multi-row matrix)  DEoptim will then use these pre-transformed
> vectors when calling constrained_objective(), so again, normalize=FALSE
> should be what we use from optimize.portfolio.


Currently fn_map() returns a list containing the transformed weights as
well as constraints that may have been relaxed. For this to work with fnMap
in DEoptim, should fn_map() just return the transformed weights vector? Do
we care what the constraints have been relaxed to and need to store the
relaxed constraints or is this not important?


>
>  The store_output object is useful for us to track the weights vector
>> before and after it is transformed at each iteration, but does not
>> affect the optimization.
>>
>
> Correct.
>
> If we use normalize=TRUE, the *solver* will not know that we've internally
> transformed the weights vector to map to our allowed constraints.  In some
> cases, penalizing the weights vector for how much it fails to meet the
> constriants when normalize=FALSE will help guide the solver to make better
> decisions, without having to transform/map the weights vector (especially
> when the feasible space is large).  In other cases, penalizing the original
> weights vector won't help at all, typically when the feasible space is very
> irregular, there are lots of assets, or the optimizer has a limited
> knowledge of what's going on inside the objective function.
>
> So, for all the other solvers, I think we need to preserve the option for
> the user to either use the mapping function fn_map with normalize=TRUE, or
> to not use them, and to then penalizethe weights vector.
>
> I guess this suggests that the normalize=FALSE section of
> constrained_objective() needs to change as well to penalize based on the
> degree that the weights vector violates the constraints.  It already does
> this for leverage constraints, but not for the other constraint types. I
> think that there may be an argument to be made to *always* penalize,
> because the constraints may have been relaxed somewhat.  In that case, the
> 'else' clause of the normalise=TRUE/FALSE if/else block may come out of the
> else and just always be execute the penalty for weights that violate each
> constraint.


Thanks for the detailed explanation. That makes perfect sense, I'll add
code to constrained objective so we have the ability to *always* penalize
for the other constraints.


>
>
>  The transformed weights from the output of fn_map will be used to
>> calculate the objective measures and the return value, out. Does the
>> optimizer, e.g. DEoptim, keep track of the adjusted weights or the
>> original weights?
>>
>
> The optimizer will only keep track of the weights it sends in, and the
> singular value of 'out'.


Thanks for clarifying.

Thanks for all the guidance, I really appreciate it!

Regards,
Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130711/3c46e7b7/attachment-0001.html>

From brian at braverock.com  Fri Jul 12 17:30:52 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Fri, 12 Jul 2013 10:30:52 -0500
Subject: [GSoC-PortA] mixed integer constraints
Message-ID: <51E0212C.4040606@braverock.com>

I just realized that we haven't done any work on mixed integer 
constraints, forcing weights to be round numbers.

Trivial to do in fn_map, and supported by a handful of optimization 
solvers, but I didn't want to forget it.

Regards,

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From brian at braverock.com  Fri Jul 12 17:49:29 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Fri, 12 Jul 2013 10:49:29 -0500
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <51E0212C.4040606@braverock.com>
References: <51E0212C.4040606@braverock.com>
Message-ID: <51E02589.70100@braverock.com>

On 07/12/2013 10:30 AM, Brian G. Peterson wrote:
> I just realized that we haven't done any work on mixed integer
> constraints, forcing weights to be round numbers.
>
> Trivial to do in fn_map, and supported by a handful of optimization
> solvers, but I didn't want to forget it.

I suppose the other one to look at in this vein is cardinality constraints.

-- 
Brian

From rossbennett34 at gmail.com  Fri Jul 12 21:22:31 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Fri, 12 Jul 2013 14:22:31 -0500
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <51E02589.70100@braverock.com>
References: <51E0212C.4040606@braverock.com>
	<51E02589.70100@braverock.com>
Message-ID: <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>

Round Lot Constraints
Just thinking through out loud how to incorporate this into fn_map. The sum
of portfolio weights would still sum to near 1 for a full investment
constraint. We would need a user input value of total wealth to calculate
the lots.
The constraint could be specified with something like:
add.constraint(portfolio, type="round_lot", total_wealth=1500000)

# randomly generate some weights
set.seed(123)
tmp <- runif(5)
w <- tmp/sum(tmp)

total_wealth <- 1200000

# round the lot values to the nearest integer
lots <- round(w * total_wealth, 0)

# calculate the new weights
new_w <- lots/sum(lots)

Am I on the right track for the round lot constraints?

Cardinality Constraints
I thought I had a good handle on cardinality constraints to set a limit on
the maximum number of positions. This is implemented in random_portfolios
by using fn_map and specifying max_pos as a position limit constraint. This
is also currently working with DEoptim by using the optional function
argument fnMap=fn_map. This is not doable with quadprog. This is supported
in rglpk. I suppose we can use the fn_map function for the other solvers,
pso and GenSA. Am I missing a different application of cardinality
constraint?

P.S. Enjoy your trip to Alaska!

Regards,
Ross


On Fri, Jul 12, 2013 at 10:49 AM, Brian G. Peterson <brian at braverock.com>wrote:

> On 07/12/2013 10:30 AM, Brian G. Peterson wrote:
>
>> I just realized that we haven't done any work on mixed integer
>> constraints, forcing weights to be round numbers.
>>
>> Trivial to do in fn_map, and supported by a handful of optimization
>> solvers, but I didn't want to forget it.
>>
>
> I suppose the other one to look at in this vein is cardinality constraints.
>
> --
> Brian
>
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130712/08d95fe4/attachment.html>

From martinrd at comcast.net  Fri Jul 12 21:32:10 2013
From: martinrd at comcast.net (Doug Martin)
Date: Fri, 12 Jul 2013 12:32:10 -0700
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <51E02589.70100@braverock.com>
References: <51E0212C.4040606@braverock.com> <51E02589.70100@braverock.com>
Message-ID: <006a01ce7f36$8d9a8a40$a8cf9ec0$@comcast.net>

This one is a high priority from my point of view.  My intention is to have
a chapter on "advanced" constraints (and penalties TBD, e.g., robust
portfolio optimization a la Axioma) that includes the two basic variants of
turnover constraints, plus one with fixed costs that makes it a MIP problem,
plus best k-out-of-n assets, and maybe some.

FYI, I have written the first two sections of chapter 1 on MVO theory primer
with placeholders for the other sections (I will send it shortly), and will
start chapter 2 which uses solve.QP and Rglpk_solve_LP for long-only, box
and group constraints, now thinking to use those only through
PortfolioAnalytics and maybe add a couple of problems to use them directly.
Want to finish that chapter by end of next week, and in the process will be
mainly  about user experience aspects of PA and  make suggestions on same
(probably mostly repeats) that will be important to feel comfortable about
the text in Chapter 2.

Doug



-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Friday, July 12, 2013 8:49 AM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] mixed integer constraints

On 07/12/2013 10:30 AM, Brian G. Peterson wrote:
> I just realized that we haven't done any work on mixed integer 
> constraints, forcing weights to be round numbers.
>
> Trivial to do in fn_map, and supported by a handful of optimization 
> solvers, but I didn't want to forget it.

I suppose the other one to look at in this vein is cardinality constraints.

--
Brian
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


From martinrd at comcast.net  Fri Jul 12 21:32:10 2013
From: martinrd at comcast.net (Doug Martin)
Date: Fri, 12 Jul 2013 12:32:10 -0700
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <51E02589.70100@braverock.com>
References: <51E0212C.4040606@braverock.com> <51E02589.70100@braverock.com>
Message-ID: <006a01ce7f36$8d9a8a40$a8cf9ec0$@comcast.net>

This one is a high priority from my point of view.  My intention is to have
a chapter on "advanced" constraints (and penalties TBD, e.g., robust
portfolio optimization a la Axioma) that includes the two basic variants of
turnover constraints, plus one with fixed costs that makes it a MIP problem,
plus best k-out-of-n assets, and maybe some.

FYI, I have written the first two sections of chapter 1 on MVO theory primer
with placeholders for the other sections (I will send it shortly), and will
start chapter 2 which uses solve.QP and Rglpk_solve_LP for long-only, box
and group constraints, now thinking to use those only through
PortfolioAnalytics and maybe add a couple of problems to use them directly.
Want to finish that chapter by end of next week, and in the process will be
mainly  about user experience aspects of PA and  make suggestions on same
(probably mostly repeats) that will be important to feel comfortable about
the text in Chapter 2.

Doug



-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Friday, July 12, 2013 8:49 AM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] mixed integer constraints

On 07/12/2013 10:30 AM, Brian G. Peterson wrote:
> I just realized that we haven't done any work on mixed integer 
> constraints, forcing weights to be round numbers.
>
> Trivial to do in fn_map, and supported by a handful of optimization 
> solvers, but I didn't want to forget it.

I suppose the other one to look at in this vein is cardinality constraints.

--
Brian
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


From martinrd at comcast.net  Fri Jul 12 21:39:11 2013
From: martinrd at comcast.net (Doug Martin)
Date: Fri, 12 Jul 2013 12:39:11 -0700
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>
References: <51E0212C.4040606@braverock.com>	<51E02589.70100@braverock.com>
 <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>
Message-ID: <006b01ce7f37$8543b960$8fcb2c20$@comcast.net>

Because solve.QP does not have MIP but Rglpk_solve_LP does, my plan has been
to use the latter for illustrating MIP (unless we find a good QP solver that
does MIP and integrate it, which seems unlikely in the near term).

 

Doug

 

P.S.  I have a project started to do a careful comparison of global minimum
variance portfolios and global minimum ETL portfolios, with a focus on the
impact of their different sources of estimation error (covariance matrix in
the former and ETL in the latter, which are not directly comparable).  In
large samples they both give the same results for normally distributed asset
returns, so their finite sample differences arise from their estimation
errors differences (large dimensions produce more covariance estimation
error and small tail probabilities produce more ETL estimation error).  

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Friday, July 12, 2013 12:23 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] mixed integer constraints

 

 

Round Lot Constraints

Just thinking through out loud how to incorporate this into fn_map. The sum
of portfolio weights would still sum to near 1 for a full investment
constraint. We would need a user input value of total wealth to calculate
the lots.

The constraint could be specified with something like:

add.constraint(portfolio, type="round_lot", total_wealth=1500000)

 

# randomly generate some weights

set.seed(123)

tmp <- runif(5)

w <- tmp/sum(tmp)

 

total_wealth <- 1200000

 

# round the lot values to the nearest integer

lots <- round(w * total_wealth, 0)

 

# calculate the new weights

new_w <- lots/sum(lots)


Am I on the right track for the round lot constraints?

 

Cardinality Constraints

I thought I had a good handle on cardinality constraints to set a limit on
the maximum number of positions. This is implemented in random_portfolios by
using fn_map and specifying max_pos as a position limit constraint. This is
also currently working with DEoptim by using the optional function argument
fnMap=fn_map. This is not doable with quadprog. This is supported in rglpk.
I suppose we can use the fn_map function for the other solvers, pso and
GenSA. Am I missing a different application of cardinality constraint?

 

P.S. Enjoy your trip to Alaska!

 

Regards,

Ross

 

 

On Fri, Jul 12, 2013 at 10:49 AM, Brian G. Peterson <brian at braverock.com>
wrote:

On 07/12/2013 10:30 AM, Brian G. Peterson wrote:

I just realized that we haven't done any work on mixed integer
constraints, forcing weights to be round numbers.

Trivial to do in fn_map, and supported by a handful of optimization
solvers, but I didn't want to forget it.

 

I suppose the other one to look at in this vein is cardinality constraints.

-- 
Brian


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130712/17d619ec/attachment-0001.html>

From martinrd at comcast.net  Fri Jul 12 23:40:59 2013
From: martinrd at comcast.net (Doug Martin)
Date: Fri, 12 Jul 2013 14:40:59 -0700
Subject: [GSoC-PortA] Beginning of Chapter 1 MSY 2nd Edition
Message-ID: <008201ce7f48$896a48e0$9c3edaa0$@comcast.net>

FYI, I have attached a draft of the first parts of Chapter 1.  You will see
the placeholders for the additional sections so can see where the total
content is headed.   This Chapter 1 is somewhat like Chapter 1  (Primer)
Scherer (2010) 4th edition, except more suited to the pedagogical needs of a
graduate course and other newcomers ,and with at least a little R
throughout.  Now I'm going to focus on Chapter 2 for the coming week. as I
mentioned in an earlier email.

Brian, bon voyage - have a great week
Ross, have a great weekend.
Guy, have a read and suggest edits.

Doug
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Chapter1 07 12 13.pdf
Type: application/pdf
Size: 245693 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130712/3e2b39da/attachment-0001.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Chapter1 07 12 13.Rnw
Type: application/octet-stream
Size: 26503 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130712/3e2b39da/attachment-0001.obj>

From brian at braverock.com  Sat Jul 13 04:57:31 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Fri, 12 Jul 2013 21:57:31 -0500
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>
References: <51E0212C.4040606@braverock.com> <51E02589.70100@braverock.com>
 <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>
Message-ID: <51E0C21B.2060307@braverock.com>

On 07/12/2013 02:22 PM, Ross Bennett wrote:
>
> Round Lot Constraints
> Just thinking through out loud how to incorporate this into fn_map. The
> sum of portfolio weights would still sum to near 1 for a full investment
> constraint. We would need a user input value of total wealth to
> calculate the lots.
> The constraint could be specified with something like:
> add.constraint(portfolio, type="round_lot", total_wealth=1500000)
>
> # randomly generate some weights
> set.seed(123)
> tmp <- runif(5)
> w <- tmp/sum(tmp)
>
> total_wealth <- 1200000
>
> # round the lot values to the nearest integer
> lots <- round(w * total_wealth, 0)
>
> # calculate the new weights
> new_w <- lots/sum(lots)
>
> Am I on the right track for the round lot constraints?

Round lots can be 1, 100, 10000, 100000, 1000000, etc. depending on the 
market/asset.

Weights are typically percentages, though there's no real requirement 
that they be so.

I'm not sure that total_wealth is really required, or even really 
relevant to 'round lots'.  I think we just need a rounding factor for 
base mixed integer constraints.

If you wanted to make sure that your chosen weights would scale to round 
lots from some NAV value, that seems like a different type of 
constraint, and requires both a rounding factor for the 'round lots', as 
above, and a total wealth to make sure it is properly divisible when 
turned back into percentage weights likely to be a fraction of 1.

> Cardinality Constraints
> I thought I had a good handle on cardinality constraints to set a limit
> on the maximum number of positions. This is implemented in
> random_portfolios by using fn_map and specifying max_pos as a position
> limit constraint. This is also currently working with DEoptim by using
> the optional function argument fnMap=fn_map. This is not doable with
> quadprog. This is supported in rglpk. I suppose we can use the fn_map
> function for the other solvers, pso and GenSA. Am I missing a different
> application of cardinality constraint?

No, I just missed this code, this sounds right.  I'm not in front of the 
code right now, but have a couple questions:
- is your method compatible with the one used by glpk?

- does it support both long and short cardinality constraints? (e.g. 
from the total universe, 20 long and 20 short?)

- does it support group cardinality constraints? (e.g. max 5 technology, 
max 10 consumer goods, max 5 manufacturing, etc?)

> P.S. Enjoy your trip to Alaska!

Thanks!

Brian


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Tue Jul 16 13:42:08 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Tue, 16 Jul 2013 06:42:08 -0500
Subject: [GSoC-PortA] mixed integer constraints
In-Reply-To: <51E0C21B.2060307@braverock.com>
References: <51E0212C.4040606@braverock.com> <51E02589.70100@braverock.com>
 <CAEESW=X--ggL5xw5Q=qnuCidYtLoopdhfeRm=9WOdGOnfjavoA@mail.gmail.com>
 <51E0C21B.2060307@braverock.com>
Message-ID: <CAEESW=V5wTNzA37tcakTB2c3XSL-oPhnw5vOShHC4tC1LYVXdQ@mail.gmail.com>

On Fri, Jul 12, 2013 at 9:57 PM, Brian G. Peterson <brian at braverock.com>wrote:

>
>  Cardinality Constraints
>> I thought I had a good handle on cardinality constraints to set a limit
>> on the maximum number of positions. This is implemented in
>> random_portfolios by using fn_map and specifying max_pos as a position
>> limit constraint. This is also currently working with DEoptim by using
>> the optional function argument fnMap=fn_map. This is not doable with
>> quadprog. This is supported in rglpk. I suppose we can use the fn_map
>> function for the other solvers, pso and GenSA. Am I missing a different
>> application of cardinality constraint?
>>
>
> No, I just missed this code, this sounds right.  I'm not in front of the
> code right now, but have a couple questions:
> - is your method compatible with the one used by glpk?
>

Yes, I've implemented basic position limit constraints with max_pos for
problems to maximize return and minimize sample CVaR/ETL/ES using Rglpk.


> - does it support both long and short cardinality constraints? (e.g. from
> the total universe, 20 long and 20 short?)
>

Not quite, it just limits the total number of non-zero weights. I think I
should be able to implement this fairly easily in fn_map, but I'm not quite
sure how to set this up in Rglpk.


> - does it support group cardinality constraints? (e.g. max 5 technology,
> max 10 consumer goods, max 5 manufacturing, etc?)


No, I had not even considered this. I should be able to implement this in
the fn_map function and modify the group_fail function to check for the
number of assets per group.

Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130716/008e7b8b/attachment.html>

From martinrd at comcast.net  Wed Jul 17 02:01:47 2013
From: martinrd at comcast.net (Doug Martin)
Date: Tue, 16 Jul 2013 17:01:47 -0700
Subject: [GSoC-PortA] add.constraint and optimize.portfolio_v2
Message-ID: <009801ce8280$de6949f0$9b3bddd0$@comcast.net>

The code in testing_ROI_Martin new.R all seems to work.  So I am putting
together all the examples for a section in Chapter 2, hopefully finish by
tomorrow.  But here is a tiny usability improvement needed in the first
example with the new interface.  The following works fine:

 

pspec <- portfolio.spec(assets=funds)

pspec <- add.constraint(portfolio=pspec, type="full_investment",
enabled=TRUE)

pspec <- add.constraint(portfolio=pspec, type="box", min=-Inf, max=Inf,
enabled=TRUE)

pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
enabled=TRUE)

 

However, I had expected that the following would also work  fine, but does
not:

 

pspec <- portfolio.spec(assets=funds)

pspec <- add.constraint(portfolio=pspec, type="full_investment",
enabled=TRUE)

pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
enabled=TRUE)

 

opt <- optimize.portfolio_v2(R=returns, portfolio=pspec,
optimize_method="ROI")

Error in get_constraints(portfolio) : 

  Box constraints min and max are not enabled or passed in

 

There should be default argument values for box, min and max if not
provided, i.e., if box is not provided and group constraints are not
provided, it should assume an unconstrained portfolio except for the full
investment constraint.

 

Also, note the following behavior:

 

> args(add.constraint)

function (portfolio, type, enabled = FALSE, ..., indexnum = NULL) 

NULL

 

Is it possible for args to result in additional arguments of function that
will make more clear what they are?  Or by design are we stuck with the "."
to indicate all the other arguments possible?

 

Doug

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130716/5d7f67a7/attachment.html>

From rossbennett34 at gmail.com  Wed Jul 17 03:54:33 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Tue, 16 Jul 2013 20:54:33 -0500
Subject: [GSoC-PortA] add.constraint and optimize.portfolio_v2
In-Reply-To: <009801ce8280$de6949f0$9b3bddd0$@comcast.net>
References: <009801ce8280$de6949f0$9b3bddd0$@comcast.net>
Message-ID: <CAEESW=X5nxssSBCXHSKRq+0+TBr71TWfehg0HNQy_1cJHvRDDg@mail.gmail.com>

On Tue, Jul 16, 2013 at 7:01 PM, Doug Martin <martinrd at comcast.net> wrote:

> The code in testing_ROI_Martin new.R all seems to work.  So I am putting
> together all the examples for a section in Chapter 2, hopefully finish by
> tomorrow.  But here is a tiny usability improvement needed in the first
> example with the new interface.  The following works fine:****
>
> ** **
>
> pspec <- portfolio.spec(assets=funds)****
>
> pspec <- add.constraint(portfolio=pspec, type="full_investment",
> enabled=TRUE)****
>
> pspec <- add.constraint(portfolio=pspec, type="box", min=-Inf, max=Inf,
> enabled=TRUE)****
>
> pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
> enabled=TRUE)****
>
> ** **
>
> However, I had expected that the following would also work  fine, but does
> not:****
>
> ** **
>
> pspec <- portfolio.spec(assets=funds)****
>
> pspec <- add.constraint(portfolio=pspec, type="full_investment",
> enabled=TRUE)****
>
> pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
> enabled=TRUE)****
>
> ** **
>
> opt <- optimize.portfolio_v2(R=returns, portfolio=pspec,
> optimize_method="ROI")****
>
> Error in get_constraints(portfolio) : ****
>
>   Box constraints min and max are not enabled or passed in****
>
> ** **
>
> There should be default argument values for box, min and max if not
> provided, i.e., if box is not provided and group constraints are not
> provided, it should assume an unconstrained portfolio except for the full
> investment constraint.****
>
> **
>
I just committed a revision (r2583) that fixes this and sets default values
if leverage and box constraints are not specified. The default is set to
full investment constraint (min_sum=1 and max_sum=1) if leverage
constraints are not specified. The default is set to infinite bounds
(min=rep(-Inf, nassets) and max=rep(Inf, nassets)) if box constraints are
not specified.


> **
>
> Also, note the following behavior:****
>
> ** **
>
> > args(add.constraint)****
>
> function (portfolio, type, enabled = FALSE, ..., indexnum = NULL) ****
>
> NULL****
>
> ** **
>
> Is it possible for args to result in additional arguments of function that
> will make more clear what they are?  Or by design are we stuck with the ???
> to indicate all the other arguments possible?
>

I think we are stuck with "..." to indicate all other arguments for the
add.constraint function. However, I have links to the other functions that
add the constraints.
?add.constraint and check the "See Also" section.

Alternatively, if you know the type of constraint and the function, you can
get the args of that function.
> args(box_constraint)
function (type, assets, min, max, min_mult, max_mult, enabled = TRUE,
    ...)
NULL

Thanks for the suggestions and feedback!
Ross

****
>
> ** **
>
> Doug****
>
> ** **
>
> ** **
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130716/d4238e16/attachment-0001.html>

From martinrd at comcast.net  Wed Jul 17 20:44:59 2013
From: martinrd at comcast.net (Doug Martin)
Date: Wed, 17 Jul 2013 11:44:59 -0700
Subject: [GSoC-PortA] add.constraint and optimize.portfolio_v2
In-Reply-To: <CAEESW=X5nxssSBCXHSKRq+0+TBr71TWfehg0HNQy_1cJHvRDDg@mail.gmail.com>
References: <009801ce8280$de6949f0$9b3bddd0$@comcast.net>
 <CAEESW=X5nxssSBCXHSKRq+0+TBr71TWfehg0HNQy_1cJHvRDDg@mail.gmail.com>
Message-ID: <002401ce831d$c6f10ea0$54d32be0$@comcast.net>

Thanks!  Doug

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Tuesday, July 16, 2013 6:55 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] add.constraint and optimize.portfolio_v2

 

 

On Tue, Jul 16, 2013 at 7:01 PM, Doug Martin <martinrd at comcast.net> wrote:

The code in testing_ROI_Martin new.R all seems to work.  So I am putting
together all the examples for a section in Chapter 2, hopefully finish by
tomorrow.  But here is a tiny usability improvement needed in the first
example with the new interface.  The following works fine:

 

pspec <- portfolio.spec(assets=funds)

pspec <- add.constraint(portfolio=pspec, type="full_investment",
enabled=TRUE)

pspec <- add.constraint(portfolio=pspec, type="box", min=-Inf, max=Inf,
enabled=TRUE)

pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
enabled=TRUE)

 

However, I had expected that the following would also work  fine, but does
not:

 

pspec <- portfolio.spec(assets=funds)

pspec <- add.constraint(portfolio=pspec, type="full_investment",
enabled=TRUE)

pspec <- add.objective_v2(portfolio=pspec, type="risk", name="var",
enabled=TRUE)

 

opt <- optimize.portfolio_v2(R=returns, portfolio=pspec,
optimize_method="ROI")

Error in get_constraints(portfolio) : 

  Box constraints min and max are not enabled or passed in

 

There should be default argument values for box, min and max if not
provided, i.e., if box is not provided and group constraints are not
provided, it should assume an unconstrained portfolio except for the full
investment constraint.

 

I just committed a revision (r2583) that fixes this and sets default values
if leverage and box constraints are not specified. The default is set to
full investment constraint (min_sum=1 and max_sum=1) if leverage constraints
are not specified. The default is set to infinite bounds (min=rep(-Inf,
nassets) and max=rep(Inf, nassets)) if box constraints are not specified.

 

Also, note the following behavior:

 

> args(add.constraint)

function (portfolio, type, enabled = FALSE, ..., indexnum = NULL) 

NULL

 

Is it possible for args to result in additional arguments of function that
will make more clear what they are?  Or by design are we stuck with the "."
to indicate all the other arguments possible?

 

I think we are stuck with "..." to indicate all other arguments for the
add.constraint function. However, I have links to the other functions that
add the constraints.

?add.constraint and check the "See Also" section. 

 

Alternatively, if you know the type of constraint and the function, you can
get the args of that function.

> args(box_constraint)

function (type, assets, min, max, min_mult, max_mult, enabled = TRUE, 

    ...) 

NULL

 

Thanks for the suggestions and feedback!

Ross

 

 

Doug

 

 


_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130717/a9c23913/attachment.html>

From martinrd at comcast.net  Wed Jul 17 20:56:16 2013
From: martinrd at comcast.net (Doug Martin)
Date: Wed, 17 Jul 2013 11:56:16 -0700
Subject: [GSoC-PortA] Additional Thoughts about Handling Constraints and
	Objectives
Message-ID: <002901ce831f$5a4d6710$0ee83530$@comcast.net>

Ross and all,

 

It seems that the only way to get constraints and objectives into
optimize.portfolio_v2 currently is to first bundle them together in an
object of class portfolio (that is produced by using portfolio.spec,
add.constraint and add.objective_v2).  Is that correct?   If so, it does not
result in the most attractive work-flow in some contexts.  I think we need
to have a version of optimize.portfolio that allows for separate inclusion
of a constraint object and an objective object.  The reason is that you
often want to backtest strategies with the objective fixed and the
constraints varying, and vice versa.  When doing so you don't want to have
to create the combined object each time.  I'm guessing this would be a
fairly easy fix, i.e., keep the current behavior with portfolio with default
NULL and add the capability above???

 

Doug

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130717/08840497/attachment.html>

From brian at braverock.com  Sun Jul 28 20:30:53 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Sun, 28 Jul 2013 13:30:53 -0500
Subject: [GSoC-PortA] Constraints and Objectives Separately in
	Optimize.Portfolio
In-Reply-To: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
Message-ID: <51F5635D.1060404@braverock.com>

I think it is worth supporting, and I don't think it will be too hard.

If we allow optimize.portfolio to take portfolio( the default) as well 
as constraints and objectives arguments (with default NULL) we can 
reconstitute a 'full' portfolio object with constraints and objectives 
included inside optimize.portfolio.  I'd suggest that the behavior in 
this case would be to append the constraints list/object or objectives 
list/object to the slots in the portfolio specification.

I think this is important because while constraints, by definition, 
usually require some knowledge of the portfolio, objectives, again by 
definition, can often be completely independent of the portfolio 
specification.

Another related use case which we should begin discussing is the ability 
to store multiple constraint and objective sets.  See for example the 
various benchmark optimizations Peter and I used in the 2012 R/Finance 
seminar.

To understand the solution space, it is very common for an investor to 
calculate GMV, Markowitz, minETL, risk budget, etc portfolios.  How 
should we represent this multiple specification model?  A list of 
portfolio specifications?  I don't have an implementation answer right 
now, I'm just aware of the user case for supporting it.

Regards,

Brian

On 07/28/2013 01:17 PM, Doug Martin wrote:
> Ross,
>
> Earlier this week Brian and I discussed the desirability of being able
> to specify constraints and objectives separately in optimize.portfolio,
> and agreed that this is worth doing.  Brian, please confirm.
>
> Thanks,
>
> Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com]
> *Sent:* Thursday, July 18, 2013 9:05 AM
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Constraints and Objectives Separately in Optimize.Portfolio
>
> It is doable, but it won't be an easy fix because optimize.portfolio_v2,
> constrained_objective_v2, and the subfunctions are designed to operate
> on a single portfolio object.
>
> It is possible to change constraints or objectives without recreating
> the entire portfolio object. You can do this for constraints as well as
> objectives. See example below, will this suffice?
>
> # Load necessary packages
>
> library(PortfolioAnalytics)
>
> library(ROI)
>
> library(ROI.plugin.glpk)
>
> # Load the edhec data
>
> data(edhec)
>
> ret <- edhec[, 1:4]
>
> funds <- colnames(ret)
>
> # Specify a portfolio object
>
> pspec <- portfolio.spec(assets=funds)
>
> # Add constraints
>
> pspec <- add.constraint(portfolio=pspec, type="weight_sum",
>
>                          min_sum=0.99, max_sum=1.01, enabled=TRUE)
>
> pspec <- add.constraint(portfolio=pspec, type="box", min=0.1, max=0.4,
> enabled=TRUE)
>
> pspec <- add.objective_v2(portfolio=pspec, type="return", name="mean",
> enabled=TRUE)
>
> opt <- optimize.portfolio_v2(R=ret, portfolio=pspec, optimize_method="ROI")
>
> opt$weights
>
> # The box constraints is the 2nd element (indexnum=2) in the constraints
> list
>
> # We can update this in place by specifying the indexnum argument in
> add.constraint
>
> # The indexnum depends on the order the constraint was added
>
> pspec$constraints
>
> # Now say we wanted to change the box constraints to for specific per
> asset weights
>
> pspec <- add.constraint(portfolio=pspec, type="box",
>
>                          min=c(0.1, 0.05, 0.1, 0.15),
>
>                          max=c(0.4, 0.4, 0.5, 0.45),
>
>                          indexnum=2, enabled=TRUE)
>
> opt_new <- optimize.portfolio_v2(R=ret, portfolio=pspec,
> optimize_method="ROI")
>
> opt_new$weights
>
> Ross
>
> On Thu, Jul 18, 2013 at 10:05 AM, Doug Martin <martinrd at comcast.net
> <mailto:martinrd at comcast.net>> wrote:
>
> Do you think you can fairly easily make the fix I suggested concerning
> having optimize.portfolio take separate constraint and objective
> arguments, in case the is not already possible?
>
> Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com
> <mailto:rossbennett34 at gmail.com>]
> *Sent:* Thursday, July 18, 2013 7:24 AM
>
>
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Re: Recurrent Problem
>
> Excellent, glad to hear that fixed the problem.
>
> Ross
>
> On Thu, Jul 18, 2013 at 9:16 AM, Doug Martin <martinrd at comcast.net
> <mailto:martinrd at comcast.net>> wrote:
>
> I did as you suggested and that fixed the problem.  Thanks!  Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com
> <mailto:rossbennett34 at gmail.com>]
> *Sent:* Thursday, July 18, 2013 4:18 AM
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Re: Recurrent Problem
>
> Doug,
>
> That does seem to be the exact problem you were running into before. I
> found this discussion about dependencies for the methods package.
> http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-hasarg
>
> I'm not familiar with WinEdt and Stangle, but it might be calling
> Rscript to run the R code chunks and the methods package doesn't load
> with Rscript. Can you try putting require(methods) in the first R code
> chunk?
>
> Hope that helps,
>
> Ross
>
> On Wed, Jul 17, 2013 at 10:14 PM, Doug Martin <martinrd at comcast.net
> <mailto:martinrd at comcast.net>> wrote:
>
> Ross,
>
> When I run the attached .Rnw file from WinEdt it fails and I get the
> following _Err.log message:
>
> Error:  chunk 1
>
> Error in add.constraint(portfolio = pspec, type = "full_investment",
> enabled = TRUE) :
>
>    could not find function "hasArg"
>
> Execution halted
>
> I believe this was the same error message I got when I tried to run your
> Vignette (remember the ?could not find function ?hasArg?.
>
> If you run Stangle on the .Rnw file and execute it, it runs fine.
>
> I tried R2.15.1 as well as 3.0.1 and have the same problem.
>
> Any idea how to fix this problem?
>
> Doug
>
> P.S. I am running other R code with Sweave in the latex document for
> Chapter 1 with no problem.
>


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From martinrd at comcast.net  Sun Jul 28 20:50:20 2013
From: martinrd at comcast.net (Doug Martin)
Date: Sun, 28 Jul 2013 11:50:20 -0700
Subject: [GSoC-PortA] Constraints and Objectives Separately
	in	Optimize.Portfolio
In-Reply-To: <51F5635D.1060404@braverock.com>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
Message-ID: <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Sunday, July 28, 2013 11:31 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] Constraints and Objectives Separately in
Optimize.Portfolio

 

I think it is worth supporting, and I don't think it will be too hard.

 

If we allow optimize.portfolio to take portfolio( the default) as well as
constraints and objectives arguments (with default NULL) we can reconstitute
a 'full' portfolio object with constraints and objectives included inside
optimize.portfolio.  I'd suggest that the behavior in this case would be to
append the constraints list/object or objectives list/object to the slots in
the portfolio specification.

 

I think this is important because while constraints, by definition, usually
require some knowledge of the portfolio, objectives, again by definition,
can often be completely independent of the portfolio specification.

 

Another related use case which we should begin discussing is the ability to
store multiple constraint and objective sets.  See for example the various
benchmark optimizations Peter and I used in the 2012 R/Finance seminar.

 

To understand the solution space, it is very common for an investor to
calculate GMV, Markowitz, minETL, risk budget, etc portfolios.  How should
we represent this multiple specification model?  A list of portfolio
specifications?  I don't have an implementation answer right now, I'm just
aware of the user case for supporting it.

[Doug] Indeed, this is crucial.  I did the following with backtest code I
used in the spring - it has a function backtest.weight that generates the
time series of rebalanced weights, and looks like this locally (one of a
bunch of examples):

 

weight.mvlub <- backtest.weight(ret, t.mw, FUN =
"port.mvlub",blo,bup)$weight

weight.mveq <- backtest.weight(ret, t.mw, port.eq)$weight

 

So I'm wondering if we can't have FUN = optimize.portfolio, and pass the
arguments for constraints and/or ojbectives as in the case of blo and bup
for each instance of executing backtest.weight?  I will, as we discussed in
Hood River, end up using the PortfolioAnalytics backtest capability.

 

Best,

Doug 

 

 

Regards,

 

Brian

 

On 07/28/2013 01:17 PM, Doug Martin wrote:

> Ross,

> 

> Earlier this week Brian and I discussed the desirability of being able 

> to specify constraints and objectives separately in 

> optimize.portfolio, and agreed that this is worth doing.  Brian, please
confirm.

> 

> Thanks,

> 

> Doug

> 

> *From:*Ross Bennett [ <mailto:rossbennett34 at gmail.com>
mailto:rossbennett34 at gmail.com]

> *Sent:* Thursday, July 18, 2013 9:05 AM

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Constraints and Objectives Separately in Optimize.Portfolio

> 

> It is doable, but it won't be an easy fix because 

> optimize.portfolio_v2, constrained_objective_v2, and the subfunctions 

> are designed to operate on a single portfolio object.

> 

> It is possible to change constraints or objectives without recreating 

> the entire portfolio object. You can do this for constraints as well 

> as objectives. See example below, will this suffice?

> 

> # Load necessary packages

> 

> library(PortfolioAnalytics)

> 

> library(ROI)

> 

> library(ROI.plugin.glpk)

> 

> # Load the edhec data

> 

> data(edhec)

> 

> ret <- edhec[, 1:4]

> 

> funds <- colnames(ret)

> 

> # Specify a portfolio object

> 

> pspec <- portfolio.spec(assets=funds)

> 

> # Add constraints

> 

> pspec <- add.constraint(portfolio=pspec, type="weight_sum",

> 

>                          min_sum=0.99, max_sum=1.01, enabled=TRUE)

> 

> pspec <- add.constraint(portfolio=pspec, type="box", min=0.1, max=0.4,

> enabled=TRUE)

> 

> pspec <- add.objective_v2(portfolio=pspec, type="return", name="mean",

> enabled=TRUE)

> 

> opt <- optimize.portfolio_v2(R=ret, portfolio=pspec, 

> optimize_method="ROI")

> 

> opt$weights

> 

> # The box constraints is the 2nd element (indexnum=2) in the 

> constraints list

> 

> # We can update this in place by specifying the indexnum argument in 

> add.constraint

> 

> # The indexnum depends on the order the constraint was added

> 

> pspec$constraints

> 

> # Now say we wanted to change the box constraints to for specific per 

> asset weights

> 

> pspec <- add.constraint(portfolio=pspec, type="box",

> 

>                          min=c(0.1, 0.05, 0.1, 0.15),

> 

>                          max=c(0.4, 0.4, 0.5, 0.45),

> 

>                          indexnum=2, enabled=TRUE)

> 

> opt_new <- optimize.portfolio_v2(R=ret, portfolio=pspec,

> optimize_method="ROI")

> 

> opt_new$weights

> 

> Ross

> 

> On Thu, Jul 18, 2013 at 10:05 AM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> Do you think you can fairly easily make the fix I suggested concerning 

> having optimize.portfolio take separate constraint and objective 

> arguments, in case the is not already possible?

> 

> Doug

> 

> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 

> < <mailto:rossbennett34 at gmail.com> mailto:rossbennett34 at gmail.com>]

> *Sent:* Thursday, July 18, 2013 7:24 AM

> 

> 

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Re: Recurrent Problem

> 

> Excellent, glad to hear that fixed the problem.

> 

> Ross

> 

> On Thu, Jul 18, 2013 at 9:16 AM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> I did as you suggested and that fixed the problem.  Thanks!  Doug

> 

> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 

> < <mailto:rossbennett34 at gmail.com> mailto:rossbennett34 at gmail.com>]

> *Sent:* Thursday, July 18, 2013 4:18 AM

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Re: Recurrent Problem

> 

> Doug,

> 

> That does seem to be the exact problem you were running into before. I 

> found this discussion about dependencies for the methods package.

>  <http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h>
http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h

> asarg

> 

> I'm not familiar with WinEdt and Stangle, but it might be calling 

> Rscript to run the R code chunks and the methods package doesn't load 

> with Rscript. Can you try putting require(methods) in the first R code 

> chunk?

> 

> Hope that helps,

> 

> Ross

> 

> On Wed, Jul 17, 2013 at 10:14 PM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> Ross,

> 

> When I run the attached .Rnw file from WinEdt it fails and I get the 

> following _Err.log message:

> 

> Error:  chunk 1

> 

> Error in add.constraint(portfolio = pspec, type = "full_investment", 

> enabled = TRUE) :

> 

>    could not find function "hasArg"

> 

> Execution halted

> 

> I believe this was the same error message I got when I tried to run 

> your Vignette (remember the "could not find function "hasArg".

> 

> If you run Stangle on the .Rnw file and execute it, it runs fine.

> 

> I tried R2.15.1 as well as 3.0.1 and have the same problem.

> 

> Any idea how to fix this problem?

> 

> Doug

> 

> P.S. I am running other R code with Sweave in the latex document for 

> Chapter 1 with no problem.

> 

 

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130728/c0adffd6/attachment-0002.html>

From martinrd at comcast.net  Sun Jul 28 20:50:20 2013
From: martinrd at comcast.net (Doug Martin)
Date: Sun, 28 Jul 2013 11:50:20 -0700
Subject: [GSoC-PortA] Constraints and Objectives Separately
	in	Optimize.Portfolio
In-Reply-To: <51F5635D.1060404@braverock.com>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
Message-ID: <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>

 

 

 

-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Sunday, July 28, 2013 11:31 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] Constraints and Objectives Separately in
Optimize.Portfolio

 

I think it is worth supporting, and I don't think it will be too hard.

 

If we allow optimize.portfolio to take portfolio( the default) as well as
constraints and objectives arguments (with default NULL) we can reconstitute
a 'full' portfolio object with constraints and objectives included inside
optimize.portfolio.  I'd suggest that the behavior in this case would be to
append the constraints list/object or objectives list/object to the slots in
the portfolio specification.

 

I think this is important because while constraints, by definition, usually
require some knowledge of the portfolio, objectives, again by definition,
can often be completely independent of the portfolio specification.

 

Another related use case which we should begin discussing is the ability to
store multiple constraint and objective sets.  See for example the various
benchmark optimizations Peter and I used in the 2012 R/Finance seminar.

 

To understand the solution space, it is very common for an investor to
calculate GMV, Markowitz, minETL, risk budget, etc portfolios.  How should
we represent this multiple specification model?  A list of portfolio
specifications?  I don't have an implementation answer right now, I'm just
aware of the user case for supporting it.

[Doug] Indeed, this is crucial.  I did the following with backtest code I
used in the spring - it has a function backtest.weight that generates the
time series of rebalanced weights, and looks like this locally (one of a
bunch of examples):

 

weight.mvlub <- backtest.weight(ret, t.mw, FUN =
"port.mvlub",blo,bup)$weight

weight.mveq <- backtest.weight(ret, t.mw, port.eq)$weight

 

So I'm wondering if we can't have FUN = optimize.portfolio, and pass the
arguments for constraints and/or ojbectives as in the case of blo and bup
for each instance of executing backtest.weight?  I will, as we discussed in
Hood River, end up using the PortfolioAnalytics backtest capability.

 

Best,

Doug 

 

 

Regards,

 

Brian

 

On 07/28/2013 01:17 PM, Doug Martin wrote:

> Ross,

> 

> Earlier this week Brian and I discussed the desirability of being able 

> to specify constraints and objectives separately in 

> optimize.portfolio, and agreed that this is worth doing.  Brian, please
confirm.

> 

> Thanks,

> 

> Doug

> 

> *From:*Ross Bennett [ <mailto:rossbennett34 at gmail.com>
mailto:rossbennett34 at gmail.com]

> *Sent:* Thursday, July 18, 2013 9:05 AM

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Constraints and Objectives Separately in Optimize.Portfolio

> 

> It is doable, but it won't be an easy fix because 

> optimize.portfolio_v2, constrained_objective_v2, and the subfunctions 

> are designed to operate on a single portfolio object.

> 

> It is possible to change constraints or objectives without recreating 

> the entire portfolio object. You can do this for constraints as well 

> as objectives. See example below, will this suffice?

> 

> # Load necessary packages

> 

> library(PortfolioAnalytics)

> 

> library(ROI)

> 

> library(ROI.plugin.glpk)

> 

> # Load the edhec data

> 

> data(edhec)

> 

> ret <- edhec[, 1:4]

> 

> funds <- colnames(ret)

> 

> # Specify a portfolio object

> 

> pspec <- portfolio.spec(assets=funds)

> 

> # Add constraints

> 

> pspec <- add.constraint(portfolio=pspec, type="weight_sum",

> 

>                          min_sum=0.99, max_sum=1.01, enabled=TRUE)

> 

> pspec <- add.constraint(portfolio=pspec, type="box", min=0.1, max=0.4,

> enabled=TRUE)

> 

> pspec <- add.objective_v2(portfolio=pspec, type="return", name="mean",

> enabled=TRUE)

> 

> opt <- optimize.portfolio_v2(R=ret, portfolio=pspec, 

> optimize_method="ROI")

> 

> opt$weights

> 

> # The box constraints is the 2nd element (indexnum=2) in the 

> constraints list

> 

> # We can update this in place by specifying the indexnum argument in 

> add.constraint

> 

> # The indexnum depends on the order the constraint was added

> 

> pspec$constraints

> 

> # Now say we wanted to change the box constraints to for specific per 

> asset weights

> 

> pspec <- add.constraint(portfolio=pspec, type="box",

> 

>                          min=c(0.1, 0.05, 0.1, 0.15),

> 

>                          max=c(0.4, 0.4, 0.5, 0.45),

> 

>                          indexnum=2, enabled=TRUE)

> 

> opt_new <- optimize.portfolio_v2(R=ret, portfolio=pspec,

> optimize_method="ROI")

> 

> opt_new$weights

> 

> Ross

> 

> On Thu, Jul 18, 2013 at 10:05 AM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> Do you think you can fairly easily make the fix I suggested concerning 

> having optimize.portfolio take separate constraint and objective 

> arguments, in case the is not already possible?

> 

> Doug

> 

> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 

> < <mailto:rossbennett34 at gmail.com> mailto:rossbennett34 at gmail.com>]

> *Sent:* Thursday, July 18, 2013 7:24 AM

> 

> 

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Re: Recurrent Problem

> 

> Excellent, glad to hear that fixed the problem.

> 

> Ross

> 

> On Thu, Jul 18, 2013 at 9:16 AM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> I did as you suggested and that fixed the problem.  Thanks!  Doug

> 

> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 

> < <mailto:rossbennett34 at gmail.com> mailto:rossbennett34 at gmail.com>]

> *Sent:* Thursday, July 18, 2013 4:18 AM

> *To:* Doug Martin

> *Cc:* Guy Yollin

> *Subject:* Re: Recurrent Problem

> 

> Doug,

> 

> That does seem to be the exact problem you were running into before. I 

> found this discussion about dependencies for the methods package.

>  <http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h>
http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h

> asarg

> 

> I'm not familiar with WinEdt and Stangle, but it might be calling 

> Rscript to run the R code chunks and the methods package doesn't load 

> with Rscript. Can you try putting require(methods) in the first R code 

> chunk?

> 

> Hope that helps,

> 

> Ross

> 

> On Wed, Jul 17, 2013 at 10:14 PM, Doug Martin <martinrd at comcast.net 

> < <mailto:martinrd at comcast.net> mailto:martinrd at comcast.net>> wrote:

> 

> Ross,

> 

> When I run the attached .Rnw file from WinEdt it fails and I get the 

> following _Err.log message:

> 

> Error:  chunk 1

> 

> Error in add.constraint(portfolio = pspec, type = "full_investment", 

> enabled = TRUE) :

> 

>    could not find function "hasArg"

> 

> Execution halted

> 

> I believe this was the same error message I got when I tried to run 

> your Vignette (remember the "could not find function "hasArg".

> 

> If you run Stangle on the .Rnw file and execute it, it runs fine.

> 

> I tried R2.15.1 as well as 3.0.1 and have the same problem.

> 

> Any idea how to fix this problem?

> 

> Doug

> 

> P.S. I am running other R code with Sweave in the latex document for 

> Chapter 1 with no problem.

> 

 

 

--

Brian G. Peterson

 <http://braverock.com/brian/> http://braverock.com/brian/

Ph: 773-459-4973

IM: bgpbraverock

_______________________________________________

GSoC-PortA mailing list

 <mailto:GSoC-PortA at lists.r-forge.r-project.org>
GSoC-PortA at lists.r-forge.r-project.org

 <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130728/c0adffd6/attachment-0003.html>

From brian at braverock.com  Sun Jul 28 20:53:42 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Sun, 28 Jul 2013 13:53:42 -0500
Subject: [GSoC-PortA] Constraints and Objectives Separately
	in	Optimize.Portfolio
In-Reply-To: <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
 <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>
Message-ID: <51F568B6.5040008@braverock.com>

On 07/28/2013 01:50 PM, Doug Martin wrote:
> */[Doug] Indeed, this is crucial.  I did the following with backtest
> code I used in the spring ? it has a function backtest.weight that
> generates the time series of rebalanced weights, and looks like this
> locally (one of a bunch of examples):/*
>
> *//*
>
> */weight.mvlub <- backtest.weight(ret, t.mw, FUN =
> "port.mvlub",blo,bup)$weight/*
>
> */weight.mveq <- backtest.weight(ret, t.mw, port.eq)$weight/*
>
> *//*
>
> */So I?m wondering if we can?t have FUN = optimize.portfolio, and pass
> the arguments for constraints and/or ojbectives as in the case of blo
> and bup for each instance of executing backtest.weight?  I will, as we
> discussed in Hood River, end up using the PortfolioAnalytics backtest
> capability./*
>
> *//*
>

Well, optimize.portfolio.rebalancing already exists, and I think covers 
thae most common use case of periodic rebalancing of a portfolio for an 
'out of sample' test of your portfolio specification (w/ 
constraints/objectives).

optimize.portfolio.rebalancing already returns the weights used for each 
rebalancing period, as well as the out of sample returns of the portfolio.


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Mon Jul 29 04:38:41 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Sun, 28 Jul 2013 21:38:41 -0500
Subject: [GSoC-PortA] Constraints and Objectives Separately in
	Optimize.Portfolio
In-Reply-To: <51F568B6.5040008@braverock.com>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
 <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>
 <51F568B6.5040008@braverock.com>
Message-ID: <CAEESW=UqeaNJY-sU5yuECNbEvXs8U3BBhj8zHW-DPRrLoOpvUQ@mail.gmail.com>

There are a couple different topics in this email thread so I just want to
comment on both.

Specifying constraints and objectives separately
I'll spend some time on this to get it working. Appreciate the suggestion
from Brian to simply insert the constraints/objectives in the appropriate
slots into the portfolio object so the optimize.portfolio function and
subfunctions can still work off of the portfolio object.

Backtesting capability
The backtest.weight function we used in class was convenient in that we
could specify a function with the FUN argument, but the downside was that
we had to write functions for each optimization case (i.e. port.gmv,
port.etl, port.lo, etc.).

As Brian already mentioned, PortfolioAnalytics supports backtesting for an
out of sample test and I have modified it to work with the new portfolio
interface. This is nice and modular because the user just has to pass in
the return object, portfolio object with the constraints and objectives,
and a few other parameters to run the backtest and return the set of
weights.
opt_rebal <- optimize.portfolio.rebalancing(R=ret, portfolio=pspec,
optimize_method="DEoptim", search_size=2000, rebalance_on="quarters")

Ross

On Sun, Jul 28, 2013 at 1:53 PM, Brian G. Peterson <brian at braverock.com>wrote:

> On 07/28/2013 01:50 PM, Doug Martin wrote:
>
>> */[Doug] Indeed, this is crucial.  I did the following with backtest
>>
>> code I used in the spring ? it has a function backtest.weight that
>> generates the time series of rebalanced weights, and looks like this
>> locally (one of a bunch of examples):/*
>>
>> *//*
>>
>> */weight.mvlub <- backtest.weight(ret, t.mw, FUN =
>> "port.mvlub",blo,bup)$weight/*
>>
>> */weight.mveq <- backtest.weight(ret, t.mw, port.eq)$weight/*
>>
>> *//*
>>
>> */So I?m wondering if we can?t have FUN = optimize.portfolio, and pass
>>
>> the arguments for constraints and/or ojbectives as in the case of blo
>> and bup for each instance of executing backtest.weight?  I will, as we
>> discussed in Hood River, end up using the PortfolioAnalytics backtest
>> capability./*
>>
>> *//*
>>
>>
> Well, optimize.portfolio.rebalancing already exists, and I think covers
> thae most common use case of periodic rebalancing of a portfolio for an
> 'out of sample' test of your portfolio specification (w/
> constraints/objectives).
>
> optimize.portfolio.rebalancing already returns the weights used for each
> rebalancing period, as well as the out of sample returns of the portfolio.
>
>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130728/270be5da/attachment.html>

From martinrd at comcast.net  Mon Jul 29 17:00:41 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 29 Jul 2013 08:00:41 -0700
Subject: [GSoC-PortA] Constraints and Objectives Separately
	in	Optimize.Portfolio
In-Reply-To: <CAEESW=UqeaNJY-sU5yuECNbEvXs8U3BBhj8zHW-DPRrLoOpvUQ@mail.gmail.com>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
 <004001ce8bc3$5bf130e0$13d392a0$@comcast.net>
 <51F568B6.5040008@braverock.com>
 <CAEESW=UqeaNJY-sU5yuECNbEvXs8U3BBhj8zHW-DPRrLoOpvUQ@mail.gmail.com>
Message-ID: <002801ce8c6c$6ea189c0$4be49d40$@comcast.net>

 

 

 

From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross
Bennett
Sent: Sunday, July 28, 2013 7:39 PM
To: PortfolioAnalytics
Subject: Re: [GSoC-PortA] Constraints and Objectives Separately in
Optimize.Portfolio

 

There are a couple different topics in this email thread so I just want to
comment on both.

 

Specifying constraints and objectives separately

I'll spend some time on this to get it working. Appreciate the suggestion
from Brian to simply insert the constraints/objectives in the appropriate
slots into the portfolio object so the optimize.portfolio function and
subfunctions can still work off of the portfolio object.

 

[Doug] That will be great.

 

Backtesting capability

The backtest.weight function we used in class was convenient in that we
could specify a function with the FUN argument, but the downside was that we
had to write functions for each optimization case (i.e. port.gmv, port.etl,
port.lo, etc.).

 

[Doug] I agree.

 

As Brian already mentioned, PortfolioAnalytics supports backtesting for an
out of sample test

[Doug] And according Brian also saves everything you would want, including
optimizer trace output, etc.

and I have modified it to work with the new portfolio interface. This is
nice and modular because the user just has to pass in the return object,
portfolio object with the constraints and objectives, and a few other
parameters to run the backtest and return the set of weights.

opt_rebal <- optimize.portfolio.rebalancing(R=ret, portfolio=pspec,
optimize_method="DEoptim", search_size=2000, rebalance_on="quarters")

 

[Doug] This is also very good.  So for example with a single objective and
many constraints, just need to build a bunch of constraint objects that the
user names, and use them as inputs.

 

Ross

 

On Sun, Jul 28, 2013 at 1:53 PM, Brian G. Peterson <brian at braverock.com>
wrote:

On 07/28/2013 01:50 PM, Doug Martin wrote:

*/[Doug] Indeed, this is crucial.  I did the following with backtest


code I used in the spring - it has a function backtest.weight that
generates the time series of rebalanced weights, and looks like this

locally (one of a bunch of examples):/*

*//*

*/weight.mvlub <- backtest.weight(ret, t.mw, FUN =
"port.mvlub",blo,bup)$weight/*

*/weight.mveq <- backtest.weight(ret, t.mw, port.eq)$weight/*

*//*

*/So I'm wondering if we can't have FUN = optimize.portfolio, and pass


the arguments for constraints and/or ojbectives as in the case of blo
and bup for each instance of executing backtest.weight?  I will, as we
discussed in Hood River, end up using the PortfolioAnalytics backtest

capability./*

*//*


Well, optimize.portfolio.rebalancing already exists, and I think covers thae
most common use case of periodic rebalancing of a portfolio for an 'out of
sample' test of your portfolio specification (w/ constraints/objectives).

optimize.portfolio.rebalancing already returns the weights used for each
rebalancing period, as well as the out of sample returns of the portfolio.




-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130729/cfca8395/attachment.html>

From martinrd at comcast.net  Mon Jul 29 17:47:15 2013
From: martinrd at comcast.net (Doug Martin)
Date: Mon, 29 Jul 2013 08:47:15 -0700
Subject: [GSoC-PortA] Constraints and Objectives Separately
	in	Optimize.Portfolio
In-Reply-To: <51F5635D.1060404@braverock.com>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
Message-ID: <004101ce8c72$efbdd940$cf398bc0$@comcast.net>

Ross,

Do I recall correctly that you implemented a general linear inequality
constraint
whereby the user specifies a row vector of the A matrix.   In that case the
user
can construct rows of the A matrix for any risk factor neutralization or
constraint
they like, including portfolio market exposures via single factor model
betas,
and use of whatever other fundamental factor model "beta" types they wish,
i.e., 
columns of the B matrix.

A related question:  Have you implemented a mapping function for a general
linear inequality constraint?  For this I think you just need the pattern of
plus
and minus signs of the row vector entries in the A matrix.  Brian, is that
correct?

Doug



-----Original Message-----
From: gsoc-porta-bounces at lists.r-forge.r-project.org
[mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
G. Peterson
Sent: Sunday, July 28, 2013 11:31 AM
To: gsoc-porta at r-forge.wu-wien.ac.at
Subject: Re: [GSoC-PortA] Constraints and Objectives Separately in
Optimize.Portfolio

I think it is worth supporting, and I don't think it will be too hard.

If we allow optimize.portfolio to take portfolio( the default) as well as
constraints and objectives arguments (with default NULL) we can reconstitute
a 'full' portfolio object with constraints and objectives included inside
optimize.portfolio.  I'd suggest that the behavior in this case would be to
append the constraints list/object or objectives list/object to the slots in
the portfolio specification.

I think this is important because while constraints, by definition, usually
require some knowledge of the portfolio, objectives, again by definition,
can often be completely independent of the portfolio specification.

Another related use case which we should begin discussing is the ability to
store multiple constraint and objective sets.  See for example the various
benchmark optimizations Peter and I used in the 2012 R/Finance seminar.

To understand the solution space, it is very common for an investor to
calculate GMV, Markowitz, minETL, risk budget, etc portfolios.  How should
we represent this multiple specification model?  A list of portfolio
specifications?  I don't have an implementation answer right now, I'm just
aware of the user case for supporting it.

Regards,

Brian

On 07/28/2013 01:17 PM, Doug Martin wrote:
> Ross,
>
> Earlier this week Brian and I discussed the desirability of being able 
> to specify constraints and objectives separately in 
> optimize.portfolio, and agreed that this is worth doing.  Brian, please
confirm.
>
> Thanks,
>
> Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com]
> *Sent:* Thursday, July 18, 2013 9:05 AM
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Constraints and Objectives Separately in Optimize.Portfolio
>
> It is doable, but it won't be an easy fix because 
> optimize.portfolio_v2, constrained_objective_v2, and the subfunctions 
> are designed to operate on a single portfolio object.
>
> It is possible to change constraints or objectives without recreating 
> the entire portfolio object. You can do this for constraints as well 
> as objectives. See example below, will this suffice?
>
> # Load necessary packages
>
> library(PortfolioAnalytics)
>
> library(ROI)
>
> library(ROI.plugin.glpk)
>
> # Load the edhec data
>
> data(edhec)
>
> ret <- edhec[, 1:4]
>
> funds <- colnames(ret)
>
> # Specify a portfolio object
>
> pspec <- portfolio.spec(assets=funds)
>
> # Add constraints
>
> pspec <- add.constraint(portfolio=pspec, type="weight_sum",
>
>                          min_sum=0.99, max_sum=1.01, enabled=TRUE)
>
> pspec <- add.constraint(portfolio=pspec, type="box", min=0.1, max=0.4,
> enabled=TRUE)
>
> pspec <- add.objective_v2(portfolio=pspec, type="return", name="mean",
> enabled=TRUE)
>
> opt <- optimize.portfolio_v2(R=ret, portfolio=pspec, 
> optimize_method="ROI")
>
> opt$weights
>
> # The box constraints is the 2nd element (indexnum=2) in the 
> constraints list
>
> # We can update this in place by specifying the indexnum argument in 
> add.constraint
>
> # The indexnum depends on the order the constraint was added
>
> pspec$constraints
>
> # Now say we wanted to change the box constraints to for specific per 
> asset weights
>
> pspec <- add.constraint(portfolio=pspec, type="box",
>
>                          min=c(0.1, 0.05, 0.1, 0.15),
>
>                          max=c(0.4, 0.4, 0.5, 0.45),
>
>                          indexnum=2, enabled=TRUE)
>
> opt_new <- optimize.portfolio_v2(R=ret, portfolio=pspec,
> optimize_method="ROI")
>
> opt_new$weights
>
> Ross
>
> On Thu, Jul 18, 2013 at 10:05 AM, Doug Martin <martinrd at comcast.net 
> <mailto:martinrd at comcast.net>> wrote:
>
> Do you think you can fairly easily make the fix I suggested concerning 
> having optimize.portfolio take separate constraint and objective 
> arguments, in case the is not already possible?
>
> Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 
> <mailto:rossbennett34 at gmail.com>]
> *Sent:* Thursday, July 18, 2013 7:24 AM
>
>
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Re: Recurrent Problem
>
> Excellent, glad to hear that fixed the problem.
>
> Ross
>
> On Thu, Jul 18, 2013 at 9:16 AM, Doug Martin <martinrd at comcast.net 
> <mailto:martinrd at comcast.net>> wrote:
>
> I did as you suggested and that fixed the problem.  Thanks!  Doug
>
> *From:*Ross Bennett [mailto:rossbennett34 at gmail.com 
> <mailto:rossbennett34 at gmail.com>]
> *Sent:* Thursday, July 18, 2013 4:18 AM
> *To:* Doug Martin
> *Cc:* Guy Yollin
> *Subject:* Re: Recurrent Problem
>
> Doug,
>
> That does seem to be the exact problem you were running into before. I 
> found this discussion about dependencies for the methods package.
> http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h
> asarg
>
> I'm not familiar with WinEdt and Stangle, but it might be calling 
> Rscript to run the R code chunks and the methods package doesn't load 
> with Rscript. Can you try putting require(methods) in the first R code 
> chunk?
>
> Hope that helps,
>
> Ross
>
> On Wed, Jul 17, 2013 at 10:14 PM, Doug Martin <martinrd at comcast.net 
> <mailto:martinrd at comcast.net>> wrote:
>
> Ross,
>
> When I run the attached .Rnw file from WinEdt it fails and I get the 
> following _Err.log message:
>
> Error:  chunk 1
>
> Error in add.constraint(portfolio = pspec, type = "full_investment", 
> enabled = TRUE) :
>
>    could not find function "hasArg"
>
> Execution halted
>
> I believe this was the same error message I got when I tried to run 
> your Vignette (remember the "could not find function "hasArg".
>
> If you run Stangle on the .Rnw file and execute it, it runs fine.
>
> I tried R2.15.1 as well as 3.0.1 and have the same problem.
>
> Any idea how to fix this problem?
>
> Doug
>
> P.S. I am running other R code with Sweave in the latex document for 
> Chapter 1 with no problem.
>


--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
_______________________________________________
GSoC-PortA mailing list
GSoC-PortA at lists.r-forge.r-project.org
http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta


From rossbennett34 at gmail.com  Mon Jul 29 22:21:56 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Mon, 29 Jul 2013 15:21:56 -0500
Subject: [GSoC-PortA] Constraints and Objectives Separately in
	Optimize.Portfolio
In-Reply-To: <004101ce8c72$efbdd940$cf398bc0$@comcast.net>
References: <003101ce8bbe$d2a83850$77f8a8f0$@comcast.net>
 <51F5635D.1060404@braverock.com>
 <004101ce8c72$efbdd940$cf398bc0$@comcast.net>
Message-ID: <CAEESW=UJNHj=G--oi8gGHJn1s+MEwJ8BPgYbiMjMPQu-ZF8iYw@mail.gmail.com>

Doug,

I have not implemented that yet. We did have some initial discussions, but
nothing further. I don't recall doing this in class and haven't been able
to find any good examples. I assume this would still be in the framework of
maximizing return, maximizing quadratic utility, or minimizing variance and
simply adding betas as a "constraint" in the A matrix, correct?

Here is a quick snippet of code, is this how it would be implemented for
the example below?

library(PortfolioAnalytics)
library(Rglpk)

data(edhec)
ret <- edhec[, 1:4]

# maximize return
# subject to:
# w'1 = 1
# w' betas <= 1.2
# w_min >= 0
# w_max <= 1

# Linear objective vector
objL <- colMeans(ret)

betas <- c(0.8, 0.2, 1.1, 1.5)
# A matrix
Amat <- rbind(rep(1, 4),
              betas,
              diag(4),
              diag(4))
rhs <- c(1, 1.2, rep(0, 4), rep(1, 4))
dir <- c("==", "<=", rep(">=", 4), rep("<=", 4))

opt <- Rglpk_solve_LP(obj=objL, mat=Amat, dir=dir, rhs=rhs, max=TRUE)
opt

Thanks,
Ross


On Mon, Jul 29, 2013 at 10:47 AM, Doug Martin <martinrd at comcast.net> wrote:

> Ross,
>
> Do I recall correctly that you implemented a general linear inequality
> constraint
> whereby the user specifies a row vector of the A matrix.   In that case the
> user
> can construct rows of the A matrix for any risk factor neutralization or
> constraint
> they like, including portfolio market exposures via single factor model
> betas,
> and use of whatever other fundamental factor model "beta" types they wish,
> i.e.,
> columns of the B matrix.
>
> A related question:  Have you implemented a mapping function for a general
> linear inequality constraint?  For this I think you just need the pattern
> of
> plus
> and minus signs of the row vector entries in the A matrix.  Brian, is that
> correct?
>
> Doug
>
>
>
> -----Original Message-----
> From: gsoc-porta-bounces at lists.r-forge.r-project.org
> [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian
> G. Peterson
> Sent: Sunday, July 28, 2013 11:31 AM
> To: gsoc-porta at r-forge.wu-wien.ac.at
> Subject: Re: [GSoC-PortA] Constraints and Objectives Separately in
> Optimize.Portfolio
>
> I think it is worth supporting, and I don't think it will be too hard.
>
> If we allow optimize.portfolio to take portfolio( the default) as well as
> constraints and objectives arguments (with default NULL) we can
> reconstitute
> a 'full' portfolio object with constraints and objectives included inside
> optimize.portfolio.  I'd suggest that the behavior in this case would be to
> append the constraints list/object or objectives list/object to the slots
> in
> the portfolio specification.
>
> I think this is important because while constraints, by definition, usually
> require some knowledge of the portfolio, objectives, again by definition,
> can often be completely independent of the portfolio specification.
>
> Another related use case which we should begin discussing is the ability to
> store multiple constraint and objective sets.  See for example the various
> benchmark optimizations Peter and I used in the 2012 R/Finance seminar.
>
> To understand the solution space, it is very common for an investor to
> calculate GMV, Markowitz, minETL, risk budget, etc portfolios.  How should
> we represent this multiple specification model?  A list of portfolio
> specifications?  I don't have an implementation answer right now, I'm just
> aware of the user case for supporting it.
>
> Regards,
>
> Brian
>
> On 07/28/2013 01:17 PM, Doug Martin wrote:
> > Ross,
> >
> > Earlier this week Brian and I discussed the desirability of being able
> > to specify constraints and objectives separately in
> > optimize.portfolio, and agreed that this is worth doing.  Brian, please
> confirm.
> >
> > Thanks,
> >
> > Doug
> >
> > *From:*Ross Bennett [mailto:rossbennett34 at gmail.com]
> > *Sent:* Thursday, July 18, 2013 9:05 AM
> > *To:* Doug Martin
> > *Cc:* Guy Yollin
> > *Subject:* Constraints and Objectives Separately in Optimize.Portfolio
> >
> > It is doable, but it won't be an easy fix because
> > optimize.portfolio_v2, constrained_objective_v2, and the subfunctions
> > are designed to operate on a single portfolio object.
> >
> > It is possible to change constraints or objectives without recreating
> > the entire portfolio object. You can do this for constraints as well
> > as objectives. See example below, will this suffice?
> >
> > # Load necessary packages
> >
> > library(PortfolioAnalytics)
> >
> > library(ROI)
> >
> > library(ROI.plugin.glpk)
> >
> > # Load the edhec data
> >
> > data(edhec)
> >
> > ret <- edhec[, 1:4]
> >
> > funds <- colnames(ret)
> >
> > # Specify a portfolio object
> >
> > pspec <- portfolio.spec(assets=funds)
> >
> > # Add constraints
> >
> > pspec <- add.constraint(portfolio=pspec, type="weight_sum",
> >
> >                          min_sum=0.99, max_sum=1.01, enabled=TRUE)
> >
> > pspec <- add.constraint(portfolio=pspec, type="box", min=0.1, max=0.4,
> > enabled=TRUE)
> >
> > pspec <- add.objective_v2(portfolio=pspec, type="return", name="mean",
> > enabled=TRUE)
> >
> > opt <- optimize.portfolio_v2(R=ret, portfolio=pspec,
> > optimize_method="ROI")
> >
> > opt$weights
> >
> > # The box constraints is the 2nd element (indexnum=2) in the
> > constraints list
> >
> > # We can update this in place by specifying the indexnum argument in
> > add.constraint
> >
> > # The indexnum depends on the order the constraint was added
> >
> > pspec$constraints
> >
> > # Now say we wanted to change the box constraints to for specific per
> > asset weights
> >
> > pspec <- add.constraint(portfolio=pspec, type="box",
> >
> >                          min=c(0.1, 0.05, 0.1, 0.15),
> >
> >                          max=c(0.4, 0.4, 0.5, 0.45),
> >
> >                          indexnum=2, enabled=TRUE)
> >
> > opt_new <- optimize.portfolio_v2(R=ret, portfolio=pspec,
> > optimize_method="ROI")
> >
> > opt_new$weights
> >
> > Ross
> >
> > On Thu, Jul 18, 2013 at 10:05 AM, Doug Martin <martinrd at comcast.net
> > <mailto:martinrd at comcast.net>> wrote:
> >
> > Do you think you can fairly easily make the fix I suggested concerning
> > having optimize.portfolio take separate constraint and objective
> > arguments, in case the is not already possible?
> >
> > Doug
> >
> > *From:*Ross Bennett [mailto:rossbennett34 at gmail.com
> > <mailto:rossbennett34 at gmail.com>]
> > *Sent:* Thursday, July 18, 2013 7:24 AM
> >
> >
> > *To:* Doug Martin
> > *Cc:* Guy Yollin
> > *Subject:* Re: Recurrent Problem
> >
> > Excellent, glad to hear that fixed the problem.
> >
> > Ross
> >
> > On Thu, Jul 18, 2013 at 9:16 AM, Doug Martin <martinrd at comcast.net
> > <mailto:martinrd at comcast.net>> wrote:
> >
> > I did as you suggested and that fixed the problem.  Thanks!  Doug
> >
> > *From:*Ross Bennett [mailto:rossbennett34 at gmail.com
> > <mailto:rossbennett34 at gmail.com>]
> > *Sent:* Thursday, July 18, 2013 4:18 AM
> > *To:* Doug Martin
> > *Cc:* Guy Yollin
> > *Subject:* Re: Recurrent Problem
> >
> > Doug,
> >
> > That does seem to be the exact problem you were running into before. I
> > found this discussion about dependencies for the methods package.
> > http://grokbase.com/t/r/r-devel/1272tva0c2/rd-dependency-problem-for-h
> > asarg
> >
> > I'm not familiar with WinEdt and Stangle, but it might be calling
> > Rscript to run the R code chunks and the methods package doesn't load
> > with Rscript. Can you try putting require(methods) in the first R code
> > chunk?
> >
> > Hope that helps,
> >
> > Ross
> >
> > On Wed, Jul 17, 2013 at 10:14 PM, Doug Martin <martinrd at comcast.net
> > <mailto:martinrd at comcast.net>> wrote:
> >
> > Ross,
> >
> > When I run the attached .Rnw file from WinEdt it fails and I get the
> > following _Err.log message:
> >
> > Error:  chunk 1
> >
> > Error in add.constraint(portfolio = pspec, type = "full_investment",
> > enabled = TRUE) :
> >
> >    could not find function "hasArg"
> >
> > Execution halted
> >
> > I believe this was the same error message I got when I tried to run
> > your Vignette (remember the "could not find function "hasArg".
> >
> > If you run Stangle on the .Rnw file and execute it, it runs fine.
> >
> > I tried R2.15.1 as well as 3.0.1 and have the same problem.
> >
> > Any idea how to fix this problem?
> >
> > Doug
> >
> > P.S. I am running other R code with Sweave in the latex document for
> > Chapter 1 with no problem.
> >
>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130729/aeda75c8/attachment-0001.html>

From brian at braverock.com  Wed Jul 31 03:05:34 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Tue, 30 Jul 2013 20:05:34 -0500
Subject: [GSoC-PortA] detect v1 'constraints' object and uipgrade to v2
	'portfolio object
Message-ID: <51F862DE.30402@braverock.com>

I got an IM from a portfolio manager in Canada today.

He said:

"I had a fun time exploring new portfolio object on the weekend as I had 
to svn it to my newly formatted laptop. the concept of the portfolio 
object is really powerful, but a lot of code had to be modified for 
backwards compatibility"

Which raises the topic of the subject line.

It seems that we should detect when the user has passed a v1 
'constraints' object into optimize.portfolio, and upgrade it to the v2 
specification.

Thoughts?

Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Wed Jul 31 15:24:10 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Wed, 31 Jul 2013 08:24:10 -0500
Subject: [GSoC-PortA] detect v1 'constraints' object and uipgrade to v2
 'portfolio object
In-Reply-To: <51F862DE.30402@braverock.com>
References: <51F862DE.30402@braverock.com>
Message-ID: <CAEESW=U2JVsDKNT4vxkxiCFiW1a758Aj2qXSamxNu980yguejA@mail.gmail.com>

Brian,

I think that makes a lot of sense and can be done. It should be pretty
straightforward to do. Should I also include a message warning the user
about using the v1 constraint object?

Thanks,
Ross
On Jul 30, 2013 8:05 PM, "Brian G. Peterson" <brian at braverock.com> wrote:

> I got an IM from a portfolio manager in Canada today.
>
> He said:
>
> "I had a fun time exploring new portfolio object on the weekend as I had
> to svn it to my newly formatted laptop. the concept of the portfolio object
> is really powerful, but a lot of code had to be modified for backwards
> compatibility"
>
> Which raises the topic of the subject line.
>
> It seems that we should detect when the user has passed a v1 'constraints'
> object into optimize.portfolio, and upgrade it to the v2 specification.
>
> Thoughts?
>
> Brian
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130731/20bb6747/attachment.html>

From brian at braverock.com  Wed Jul 31 15:25:56 2013
From: brian at braverock.com (Brian G. Peterson)
Date: Wed, 31 Jul 2013 08:25:56 -0500
Subject: [GSoC-PortA] detect v1 'constraints' object and uipgrade to v2
 'portfolio object
In-Reply-To: <CAEESW=U2JVsDKNT4vxkxiCFiW1a758Aj2qXSamxNu980yguejA@mail.gmail.com>
References: <51F862DE.30402@braverock.com>
 <CAEESW=U2JVsDKNT4vxkxiCFiW1a758Aj2qXSamxNu980yguejA@mail.gmail.com>
Message-ID: <51F91064.8080605@braverock.com>

Yes, a warning to the effect of:

"v1 constraints object detected, converting to v2 portfolio specification"

seems to be in order.  At least that should tell the user that they 
might want to update some code.

Regards,

Brian


On 07/31/2013 08:24 AM, Ross Bennett wrote:
> Brian,
>
> I think that makes a lot of sense and can be done. It should be pretty
> straightforward to do. Should I also include a message warning the user
> about using the v1 constraint object?
>
> Thanks,
> Ross
>
> On Jul 30, 2013 8:05 PM, "Brian G. Peterson" <brian at braverock.com
> <mailto:brian at braverock.com>> wrote:
>
>     I got an IM from a portfolio manager in Canada today.
>
>     He said:
>
>     "I had a fun time exploring new portfolio object on the weekend as I
>     had to svn it to my newly formatted laptop. the concept of the
>     portfolio object is really powerful, but a lot of code had to be
>     modified for backwards compatibility"
>
>     Which raises the topic of the subject line.
>
>     It seems that we should detect when the user has passed a v1
>     'constraints' object into optimize.portfolio, and upgrade it to the
>     v2 specification.
>
>     Thoughts?
>
>     Brian
>
>     --
>     Brian G. Peterson
>     http://braverock.com/brian/
>     Ph: 773-459-4973 <tel:773-459-4973>
>     IM: bgpbraverock
>     _________________________________________________
>     GSoC-PortA mailing list
>     GSoC-PortA at lists.r-forge.r-__project.org
>     <mailto:GSoC-PortA at lists.r-forge.r-project.org>
>     http://lists.r-forge.r-__project.org/cgi-bin/mailman/__listinfo/gsoc-porta
>     <http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
>
>
> _______________________________________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-project.org
> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta
>


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

From rossbennett34 at gmail.com  Wed Jul 31 22:39:39 2013
From: rossbennett34 at gmail.com (Ross Bennett)
Date: Wed, 31 Jul 2013 15:39:39 -0500
Subject: [GSoC-PortA] detect v1 'constraints' object and uipgrade to v2
 'portfolio object
In-Reply-To: <51F91064.8080605@braverock.com>
References: <51F862DE.30402@braverock.com>
 <CAEESW=U2JVsDKNT4vxkxiCFiW1a758Aj2qXSamxNu980yguejA@mail.gmail.com>
 <51F91064.8080605@braverock.com>
Message-ID: <CAEESW=WhA7f+v+_aaKq3zEF6cQOYty_pZ=bes3CFkypsRPnMew@mail.gmail.com>

Brian,

I modified the code where v1 vs. v2 has an effect in an effort to maintain
backwards compatibility. With the revisions, we are actually supporting
backwards compatibility in two ways.

1. The user can specify the v1 constraint object using the old way and add
objectives with add.objective_v1. Then all that is needed is to run the
optimization with the old interface is use optimize.portfolio_v1.

2. The user can specify the v1 constraint object using the old way and add
objectives with add.objective_v1. Then the user can use optimize.portfolio
and pass the v1_constraint object as the constraints argument. The one
minor drawback is that the user still needs to specify a portfolio object
so that I can update it with the constraints and objectives from the
v1_constraint object. The portfolio object does not need to have any
constraints or objectives, so the user only has to add one line of code (in
most cases). The major upside is that this will work with the new utility
functions (print, summary, plot) that work with the v2 specification.

I also added a testing script with examples of backwards compatibility.

Let me know if you have any other suggestions regarding backwards
compatibility or if the portfolio manager that reached out to you has any
questions or feedback with the latest revisions.

Thanks,
Ross

On Wed, Jul 31, 2013 at 8:25 AM, Brian G. Peterson <brian at braverock.com>wrote:

> Yes, a warning to the effect of:
>
> "v1 constraints object detected, converting to v2 portfolio specification"
>
> seems to be in order.  At least that should tell the user that they might
> want to update some code.
>
> Regards,
>
> Brian
>
>
>
> On 07/31/2013 08:24 AM, Ross Bennett wrote:
>
>> Brian,
>>
>> I think that makes a lot of sense and can be done. It should be pretty
>> straightforward to do. Should I also include a message warning the user
>> about using the v1 constraint object?
>>
>> Thanks,
>> Ross
>>
>> On Jul 30, 2013 8:05 PM, "Brian G. Peterson" <brian at braverock.com
>> <mailto:brian at braverock.com>> wrote:
>>
>>     I got an IM from a portfolio manager in Canada today.
>>
>>     He said:
>>
>>     "I had a fun time exploring new portfolio object on the weekend as I
>>     had to svn it to my newly formatted laptop. the concept of the
>>     portfolio object is really powerful, but a lot of code had to be
>>     modified for backwards compatibility"
>>
>>     Which raises the topic of the subject line.
>>
>>     It seems that we should detect when the user has passed a v1
>>     'constraints' object into optimize.portfolio, and upgrade it to the
>>     v2 specification.
>>
>>     Thoughts?
>>
>>     Brian
>>
>>     --
>>     Brian G. Peterson
>>     http://braverock.com/brian/
>>     Ph: 773-459-4973 <tel:773-459-4973>
>>     IM: bgpbraverock
>>     ______________________________**___________________
>>     GSoC-PortA mailing list
>>     GSoC-PortA at lists.r-forge.r-__**project.org<GSoC-PortA at lists.r-forge.r-__project.org>
>>     <mailto:GSoC-PortA at lists.r-**forge.r-project.org<GSoC-PortA at lists.r-forge.r-project.org>
>> >
>>     http://lists.r-forge.r-__**project.org/cgi-bin/mailman/__**
>> listinfo/gsoc-porta<http://lists.r-forge.r-__project.org/cgi-bin/mailman/__listinfo/gsoc-porta>
>>     <http://lists.r-forge.r-**project.org/cgi-bin/mailman/**
>> listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>> >
>>
>>
>>
>>
>> ______________________________**_________________
>> GSoC-PortA mailing list
>> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
>> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**
>> listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>>
>>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
> ______________________________**_________________
> GSoC-PortA mailing list
> GSoC-PortA at lists.r-forge.r-**project.org<GSoC-PortA at lists.r-forge.r-project.org>
> http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta<http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/gsoc-porta/attachments/20130731/c5ef5c9e/attachment.html>