From brian at braverock.com Tue Jun 4 01:37:22 2013 From: brian at braverock.com (Brian G. Peterson) Date: Mon, 03 Jun 2013 18:37:22 -0500 Subject: [GSoC-PortA] Welcome to our GSoC PortfolioAnalytics project list Message-ID: <51AD28B2.5030408@braverock.com> * You're receiving this message from a private list I've set up to help keep us all coordinated over this summer's GSoC project. All of the mentors and the student are subscribed and can send mail to the list. Using the list will let everyone communicate with just one address (the list address), and will make it easier to sort these communications out in our inboxes (I'm sure all of you, like me, get lots of emails every day). It should also lessen the pressure on everyone, as any of the mentors can respond if we have an answer to a student query, when we have a moment to do so. Please use this list for all 'general' or non-time-critical discussions about the project, so that everyone stays informed. Obviously, if you are local to one of the mentors, face to face meetings are encouraged. Please do inform the rest of us of major decisions though. I anticipate that we will discuss design decisions, implementation roadblocks, and various implementation choices that need to be made as the summer progresses. These discussions should take place on this mailing list as much as possible. As you know, the schedule has several milestones: * Community Bonding Period: Today, May 27th * Official Coding Start: June 17th * Mid Term Evaluation: Jul 29th - Aug 2nd * Pencils Down: Sept 23rd * Final Evaluations Due: Sept 27th Please add these to your calendar (or add the Google Calendar overlay [0] to yours for the summer from melange) As the first phase starts today, we want to re-emphasize what our objectives are for the short term. This first phase is intended for you to get your bearings and firm up your coding plan. To do that, you'll need to identify and obtain any source and background reading materials. Start reading and and asking questions. Early on, you'll want to identify what data you need for tests and examples. Inventory any existing code. Familiarize yourself with packages you will use for development, and packages you will be committing code into. Get set up with R-forge and SVN, and make your first commits (no matter how minor). List out what you want to start implementing and why. Sketch out a project plan in more detail than you have previously. Start communicating about what you are going to do when you start coding. This project will use R-Forge [1] extensively. R-Forge provides a set of tools for source code management (Subversion) and various web-based features. To use Subversion (svn) on R-Forge you'll need toregister as a site user [2] and thenlogin [3]. If you are unfamiliar with R-Forge, you may want to review a copy of theUser's Manual [4]. To get you set up with svn commit access on R-Forge, we will need your r-forge id. R-Forge uses svn for version control, and it will be very important for everyone (both mentors and students) to quickly get adept at using svn. For more specifics about how to use svn, take a look at the book Version Control with Subversion [5]. You will need to install the client of your choice (e.g., Tortoise SVN [6] on Windows or svnX on Mac OSX) and check out the repository. Please do not hesitate to ask for help if you get stuck - this is a critical component of our workflow and will be important for keeping everyone up to date with current code. If you've previously checked out the code anonymously, you'll need to check it out again using your R-Forge id (and ssh key) before you'll be able to commit your changes. In addition, everyone should join the r-forge commit list for the project the code is being submitted into. For example, go to the returnanalytics [7] project on R-Forge. You'll see a link for mailing lists, with one public mailing list called "returnanalytics-commits". Subscribe to that, and you'll be notified by email of any commits made to the project. Please try to make commits to svn at least daily while coding. If you make an improvement and it works - check it in. This way mentors will be able to test code continuously, and we'll know quickly if something is broken. We suggest an iterative approach to development: first make it work, then make it work *correctly*, and finally make it work fast (if needed). Do not try to make it perfect or even pretty before checking something in. Make sure you provide a log message for each commit. Look at the log of the repository you will be working with to get a feel for the logging style. Make small changes, frequently. We know from past years that students who make incremental, small progress have a much greater chance of successfully finishing the summer. Document as you write. It is really important to write the documentation as you write functions, perhaps even *before* you write the function, at least to describe what it should do. When documenting a function: * make sure equations are correct and cited, * make sure all user-facing functions have examples, * make sure you know the expected results of the examples (these will become tests), * make sure relevant literature is cited everywhere, and * apply a standard mathematical notation. In most cases, follow the notation from the original paper. We either have or are currently converting all of our packages' documentation to roxygen2 [8], an in-source 'literate programming' [9] documentation system for generating Rd, collation, and NAMESPACE files. What that means is that the documentation will be in the same file as the functions (as comments before each function) which will make writing and synchronizing the documentation easier for everyone. Every function file will have the documentation and roxygen tags in the file, and roxygenize() will be run before the package build process to generate the Rd documentation files required by R. Roxygen2 is available on CRAN. Equations in documentation should have both full LaTeX code for printing in the PDF and a text representation that will be used in the console help. Use: \eqn{\LaTeX}{ascii} or \deqn{\LaTeX}{ascii} Greek letters will also be rendered in the HTML help. However, the only way to get the full mathematical equation layout is in the PDF rendered from LaTeX. For more information on documentation and R package development in general, read 'Writing R Extensions '[10]. Although preferences for code style do vary, when there are a number of contributors to the package it can be important for readability and future maintainability of the code. You should strive (as much as is practical) to match the style in the existing code. When in doubt, rely on Google's R Style Guide [11] or ask the mailing list. Everyone should know how to build packages from source, although once-daily builds may be available on R-Forge. A *nix machine should have everything needed (see Appendix A of 'R Installation and Administration '[13]), but a regular Windows machine will not. Windows users will need to install RTools [12], a collection of resources for building packages for R under Microsoft Windows (see Appendix D of 'R Installation and Administration '[13]). Once all tools are in place, you should be able to build the package by opening a shell, moving to the directory of the package, and typing 'R CMD INSTALL packagename'. The R-Forge Manual provides more detail in section 4. I know this was a lot of information, but we thought it would make sense to get most of it out of the way immediately, in a format that is easily referred to throughout the summer. Please don't hesitate to use the list to ask any questions, that's what it is here for. Regards, Brian References: [0] Google Calendar overlay http://www.google-melange.com/gsoc/events/google/gsoc2013 [1] R-Forge http://r-forge.r-project.org [2] R-Forge registration https://r-forge.r-project.org/account/register.php [3] R-Forge login https://r-forge.r-project.org/account/login.php [4] R-Forge User Manual http://download.r-forge.r-project.org/R-Forge_Manual.pdf [5] SVN Book http://svnbook.red-bean.com [6] Tortoise SVN http://tortoisesvn.tigris.org [7] ReturnAnalytics on R-Forge https://r-forge.r-project.org/projects/returnanalytics/ [8] roxygen2 http://cran.r-project.org/web/packages/roxygen2/index.html [9] literate programming http://en.wikipedia.org/wiki/Literate_programming [10] Writing R Extensions http://cran.r-project.org/doc/manuals/R-exts.pdf [11] Google's R style guide http://google-styleguide.googlecode.com/svn/trunk/google-r-style.html [12] Rtools http://cran.r-project.org/bin/windows/Rtools/ [13] R Installation and Administration http://cran.r-project.org/doc/manuals/R-admin.pdf * -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Wed Jun 5 13:22:20 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Wed, 5 Jun 2013 06:22:20 -0500 Subject: [GSoC-PortA] Schedule for PortfolioAnalytics Coding Message-ID: All, In my GSOC proposal, the order of what components I planned to write code for was as follows: 1. Utility Functions (print, summary, plot, etc.) 2. Constraints 3. Example Functionality In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first? Thanks, Ross Bennett -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Wed Jun 5 16:23:57 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 5 Jun 2013 07:23:57 -0700 Subject: [GSoC-PortA] Schedule for PortfolioAnalytics Coding In-Reply-To: References: Message-ID: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> Ross and all, I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week). With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Wednesday, June 05, 2013 4:22 AM To: gsoc-porta at lists.r-forge.r-project.org Subject: [GSoC-PortA] Schedule for PortfolioAnalytics Coding All, In my GSOC proposal, the order of what components I planned to write code for was as follows: 1. Utility Functions (print, summary, plot, etc.) 2. Constraints 3. Example Functionality In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first? Thanks, Ross Bennett -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.r Type: application/octet-stream Size: 11673 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.test.r Type: application/octet-stream Size: 5858 bytes Desc: not available URL: From rossbennett34 at gmail.com Thu Jun 6 13:54:41 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Thu, 6 Jun 2013 06:54:41 -0500 Subject: [GSoC-PortA] Schedule for PortfolioAnalytics Coding In-Reply-To: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> Message-ID: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin wrote: > Ross and all,**** > > ** ** > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last year?s GSoC > should be the first priority. How planning on that week after next (Guy, > Eric and I all have finals week to get out of the way next week).**** > > ** ** > > With regard to constraints, that approach that Kirk Li (UW Statistics > Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross > you already have the code which I posted to the class web site. I have > attached it here for others on this project to have a look at. Let?s > review that after the end of next week when final exams and the spring > quarter are completely behind us.**** > > ** ** > > Doug**** > > ** ** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding**** > > ** ** > > All,**** > > ** ** > > In my GSOC proposal, the order of what components I planned to write code > for was as follows:**** > > 1. Utility Functions (print, summary, plot, etc.)**** > 2. Constraints**** > 3. Example Functionality**** > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints first. > Would you like me to submit a revised schedule and implementation plan with > constraints first?**** > > ** ** > > Thanks,**** > > Ross Bennett**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Mon Jun 17 13:51:10 2013 From: brian at braverock.com (Brian G. Peterson) Date: Mon, 17 Jun 2013 06:51:10 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> Message-ID: <51BEF82E.5030402@braverock.com> I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems that > there is already a lot of the infrastructure in place for the constraint > object and the optimizer so I don't want to maintain the interface to > the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets > gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, > min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object > groups <- c(3,3,3,4) > gen.constr$groups <- groups > gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we > would remove box constraints from the constraint() function and create > an add.constraint() function to add/update box, group, and other > constraint types as applicable. > > # Create the initial constraint object > # Pass in a character vector of asset names for assets > # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) > gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less than 0.5 > gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets > # Specify the group weights as a vector > # The first 3 assets shall have a group weight of greater than 0.15 and > less than 0.3 > # The next 3 assets shall have a group weight of greater than 0.1 and > less than 0.35 > # The last 4 assets shall have a group weight greater than 0 and less > than 0.4 > constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we can > revisit this so that I can hit the ground running when finals are over > and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year?s GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let?s review that after the end of next week > when final exams and the spring quarter are completely behind us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ From rossbennett34 at gmail.com Mon Jun 17 16:09:28 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 09:09:28 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51BEF82E.5030402@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote: > >> Professor Martin, >> >> Thank you for the feedback. Based on the suggestions from yourself and >> Brian Peterson, I will start with the constraints portion. I'll spend >> some more time looking at Kirk Li's code to understand it better and >> think about how to best integrate into PortfolioAnalytics. I should be >> able to use some ideas from it. >> >> I have been reviewing the code for PortfolioAnalytics and it seems that >> there is already a lot of the infrastructure in place for the constraint >> object and the optimizer so I don't want to maintain the interface to >> the optimize.portfolio function as much as possible. >> >> As an example, here is a code snippet of how box constraints and group >> constraints are currently specified. >> # General constraints object >> # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for >> assets >> gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, >> min_sum=1, max_sum=1) >> >> # Add group constraints to the gen.constr object >> groups <- c(3,3,3,4) >> gen.constr$groups <- groups >> gen.constr$cLO <- rep(0.15,length(groups)) >> gen.constr$cUP <- rep(0.30,length(groups)) >> >> Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we >> would remove box constraints from the constraint() function and create >> an add.constraint() function to add/update box, group, and other >> constraint types as applicable. >> >> # Create the initial constraint object >> # Pass in a character vector of asset names for assets >> # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) >> gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) >> >> # Add box constraints with asset weights greater than 0.1 and less than >> 0.5 >> gen.constr <- add.constraint(constraints=**gen.constr, type="box", >> min=0.1, max=0.5) >> >> # Group constraints for 10 assets >> # Separate assets into groups of 3, 3, and 4 assets >> # Specify the group weights as a vector >> # The first 3 assets shall have a group weight of greater than 0.15 and >> less than 0.3 >> # The next 3 assets shall have a group weight of greater than 0.1 and >> less than 0.35 >> # The last 4 assets shall have a group weight greater than 0 and less >> than 0.4 >> constr <- add.constraint(constraints=**constr, type="group", >> groups=c(3, 3, 4), >> group_min=c(0.15, 0.1, 0), >> group_max=c(0.3, 0.35, 0.4)) >> >> This keeps the constraint object and interface to optimize.portfolio() >> consistent with the existing code, but simplifies it and makes adding >> box and group constraints very similar. >> >> I will continue to review the PortfolioAnalytics source code and we can >> revisit this so that I can hit the ground running when finals are over >> and start as soon as possible. >> >> Regards, >> Ross Bennett >> >> >> On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > > wrote: >> >> Ross and all,____ >> >> __ __ >> >> I think starting with the constraints is a good idea. But I think a >> review of where PortfolioAnalytics functionality is as of last >> year?s GSoC should be the first priority. How planning on that week >> after next (Guy, Eric and I all have finals week to get out of the >> way next week).____ >> >> __ __ >> >> With regard to constraints, that approach that Kirk Li (UW >> Statistics Ph.D. student and AMATH 543 TA this spring) developed >> looks useful. Ross you already have the code which I posted to the >> class web site. I have attached it here for others on this project >> to have a look at. Let?s review that after the end of next week >> when final exams and the spring quarter are completely behind us.____ >> >> __ __ >> >> Doug____ >> >> __ __ >> >> __ __ >> >> *From:*gsoc-porta-bounces@**lists.r-forge.r-project.org >> >> > >> [mailto:gsoc-porta-bounces@**lists.r-forge.r-project.org >> >] >> *On Behalf >> Of *Ross Bennett >> *Sent:* Wednesday, June 05, 2013 4:22 AM >> *To:* gsoc-porta at lists.r-forge.r-**project.org >> >> > >> *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ >> >> __ __ >> >> All,____ >> >> __ __ >> >> In my GSOC proposal, the order of what components I planned to write >> code for was as follows:____ >> >> 1. Utility Functions (print, summary, plot, etc.)____ >> 2. Constraints____ >> 3. Example Functionality____ >> >> In speaking with Brian Peterson at R/Finance, he suggested that the >> constraints portion be done first. I am fine with doing constraints >> first. Would you like me to submit a revised schedule and >> implementation plan with constraints first?____ >> >> __ __ >> >> Thanks,____ >> >> Ross Bennett____ >> > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 17 16:09:28 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 09:09:28 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51BEF82E.5030402@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote: > >> Professor Martin, >> >> Thank you for the feedback. Based on the suggestions from yourself and >> Brian Peterson, I will start with the constraints portion. I'll spend >> some more time looking at Kirk Li's code to understand it better and >> think about how to best integrate into PortfolioAnalytics. I should be >> able to use some ideas from it. >> >> I have been reviewing the code for PortfolioAnalytics and it seems that >> there is already a lot of the infrastructure in place for the constraint >> object and the optimizer so I don't want to maintain the interface to >> the optimize.portfolio function as much as possible. >> >> As an example, here is a code snippet of how box constraints and group >> constraints are currently specified. >> # General constraints object >> # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for >> assets >> gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, >> min_sum=1, max_sum=1) >> >> # Add group constraints to the gen.constr object >> groups <- c(3,3,3,4) >> gen.constr$groups <- groups >> gen.constr$cLO <- rep(0.15,length(groups)) >> gen.constr$cUP <- rep(0.30,length(groups)) >> >> Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we >> would remove box constraints from the constraint() function and create >> an add.constraint() function to add/update box, group, and other >> constraint types as applicable. >> >> # Create the initial constraint object >> # Pass in a character vector of asset names for assets >> # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) >> gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) >> >> # Add box constraints with asset weights greater than 0.1 and less than >> 0.5 >> gen.constr <- add.constraint(constraints=**gen.constr, type="box", >> min=0.1, max=0.5) >> >> # Group constraints for 10 assets >> # Separate assets into groups of 3, 3, and 4 assets >> # Specify the group weights as a vector >> # The first 3 assets shall have a group weight of greater than 0.15 and >> less than 0.3 >> # The next 3 assets shall have a group weight of greater than 0.1 and >> less than 0.35 >> # The last 4 assets shall have a group weight greater than 0 and less >> than 0.4 >> constr <- add.constraint(constraints=**constr, type="group", >> groups=c(3, 3, 4), >> group_min=c(0.15, 0.1, 0), >> group_max=c(0.3, 0.35, 0.4)) >> >> This keeps the constraint object and interface to optimize.portfolio() >> consistent with the existing code, but simplifies it and makes adding >> box and group constraints very similar. >> >> I will continue to review the PortfolioAnalytics source code and we can >> revisit this so that I can hit the ground running when finals are over >> and start as soon as possible. >> >> Regards, >> Ross Bennett >> >> >> On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > > wrote: >> >> Ross and all,____ >> >> __ __ >> >> I think starting with the constraints is a good idea. But I think a >> review of where PortfolioAnalytics functionality is as of last >> year?s GSoC should be the first priority. How planning on that week >> after next (Guy, Eric and I all have finals week to get out of the >> way next week).____ >> >> __ __ >> >> With regard to constraints, that approach that Kirk Li (UW >> Statistics Ph.D. student and AMATH 543 TA this spring) developed >> looks useful. Ross you already have the code which I posted to the >> class web site. I have attached it here for others on this project >> to have a look at. Let?s review that after the end of next week >> when final exams and the spring quarter are completely behind us.____ >> >> __ __ >> >> Doug____ >> >> __ __ >> >> __ __ >> >> *From:*gsoc-porta-bounces@**lists.r-forge.r-project.org >> >> > >> [mailto:gsoc-porta-bounces@**lists.r-forge.r-project.org >> >] >> *On Behalf >> Of *Ross Bennett >> *Sent:* Wednesday, June 05, 2013 4:22 AM >> *To:* gsoc-porta at lists.r-forge.r-**project.org >> >> > >> *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ >> >> __ __ >> >> All,____ >> >> __ __ >> >> In my GSOC proposal, the order of what components I planned to write >> code for was as follows:____ >> >> 1. Utility Functions (print, summary, plot, etc.)____ >> 2. Constraints____ >> 3. Example Functionality____ >> >> In speaking with Brian Peterson at R/Finance, he suggested that the >> constraints portion be done first. I am fine with doing constraints >> first. Would you like me to submit a revised schedule and >> implementation plan with constraints first?____ >> >> __ __ >> >> Thanks,____ >> >> Ross Bennett____ >> > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Mon Jun 17 16:57:54 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 17 Jun 2013 07:57:54 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: <000601ce6b6b$18205d50$486117f0$@comcast.net> Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Mon Jun 17 16:57:54 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 17 Jun 2013 07:57:54 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: <000601ce6b6b$18205d50$486117f0$@comcast.net> Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 17 19:26:37 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 12:26:37 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <000601ce6b6b$18205d50$486117f0$@comcast.net> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> Message-ID: All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() - Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() - Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() - Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() - Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() - Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() - Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() - Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: > Brian and Ross,**** > > ** ** > > I couldn?t agree more with the proposal to separate constraints and > objectives (Brian, as you and I have discussed earlier this year or last > year).**** > > ** ** > > I will have further comments on checking validity of adding constraints > late tomorrow or early Wed. after I have completed grading of my spring > quarter course.**** > > ** ** > > Doug**** > > ** ** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 7:09 AM > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > ** ** > > Brian,**** > > ** ** > > I really like your thoughts on separating constraints and objectives. I > learned objectives and constraints in the same way that you mention them. > Intuitively, it is easier to understand that the user would create the > constraint object, the objective object, and then combine in a portfolio > object.**** > > ** ** > > Separating out the objectives and creating templates of objectives for > minimum variance and other basic objective functions should make it easy to > understand for the user as well how it is handled with the solvers.**** > > ** ** > > The way the constraints object looks today seems to be very similar to > what you have in mind for the portfoliospec object. Was that your intent so > that the solvers see pretty much the same object in the call to > optimize.portfolio? Only now the portfoliospec object will be passed > instead of the constraints object, is my understanding of your intentions > correct?**** > > ** ** > > This is what I am thinking the code will look *something* like to run a > simple optimization problem.**** > > ** ** > > #### Example R code**** > > ** ** > > # Specify a constraints object and add box constraints**** > > gen.constr = constraint(assets, min_sum=1, max_sum=1)**** > > gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, > type="box")**** > > ** ** > > # Specify an objective object for global minimum variance**** > > gen.obj = objective(type="GMV")**** > > ** ** > > # Specify a portfolio object**** > > port.spec = portfoliospec(gen.constr, gen.obj)**** > > ** ** > > # Run the optimization**** > > solution = optimize.portfolio(data, port.spec)**** > > #####**** > > ** ** > > Thanks,**** > > Ross Bennett**** > > ** ** > > On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson > wrote:**** > > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote:**** > > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems that > there is already a lot of the infrastructure in place for the constraint > object and the optimizer so I don't want to maintain the interface to > the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets > gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, > min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object > groups <- c(3,3,3,4) > gen.constr$groups <- groups > gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we > would remove box constraints from the constraint() function and create > an add.constraint() function to add/update box, group, and other > constraint types as applicable. > > # Create the initial constraint object > # Pass in a character vector of asset names for assets > # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) > gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less than 0.5 > gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets > # Specify the group weights as a vector > # The first 3 assets shall have a group weight of greater than 0.15 and > less than 0.3 > # The next 3 assets shall have a group weight of greater than 0.1 and > less than 0.35 > # The last 4 assets shall have a group weight greater than 0 and less > than 0.4 > constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we can > revisit this so that I can hit the ground running when finals are over > and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year?s GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let?s review that after the end of next week > when final exams and the spring quarter are completely behind us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > ** ** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 17 19:26:37 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 12:26:37 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <000601ce6b6b$18205d50$486117f0$@comcast.net> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> Message-ID: All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() - Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() - Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() - Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() - Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() - Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() - Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() - Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: > Brian and Ross,**** > > ** ** > > I couldn?t agree more with the proposal to separate constraints and > objectives (Brian, as you and I have discussed earlier this year or last > year).**** > > ** ** > > I will have further comments on checking validity of adding constraints > late tomorrow or early Wed. after I have completed grading of my spring > quarter course.**** > > ** ** > > Doug**** > > ** ** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 7:09 AM > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > ** ** > > Brian,**** > > ** ** > > I really like your thoughts on separating constraints and objectives. I > learned objectives and constraints in the same way that you mention them. > Intuitively, it is easier to understand that the user would create the > constraint object, the objective object, and then combine in a portfolio > object.**** > > ** ** > > Separating out the objectives and creating templates of objectives for > minimum variance and other basic objective functions should make it easy to > understand for the user as well how it is handled with the solvers.**** > > ** ** > > The way the constraints object looks today seems to be very similar to > what you have in mind for the portfoliospec object. Was that your intent so > that the solvers see pretty much the same object in the call to > optimize.portfolio? Only now the portfoliospec object will be passed > instead of the constraints object, is my understanding of your intentions > correct?**** > > ** ** > > This is what I am thinking the code will look *something* like to run a > simple optimization problem.**** > > ** ** > > #### Example R code**** > > ** ** > > # Specify a constraints object and add box constraints**** > > gen.constr = constraint(assets, min_sum=1, max_sum=1)**** > > gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, > type="box")**** > > ** ** > > # Specify an objective object for global minimum variance**** > > gen.obj = objective(type="GMV")**** > > ** ** > > # Specify a portfolio object**** > > port.spec = portfoliospec(gen.constr, gen.obj)**** > > ** ** > > # Run the optimization**** > > solution = optimize.portfolio(data, port.spec)**** > > #####**** > > ** ** > > Thanks,**** > > Ross Bennett**** > > ** ** > > On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson > wrote:**** > > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote:**** > > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems that > there is already a lot of the infrastructure in place for the constraint > object and the optimizer so I don't want to maintain the interface to > the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets > gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, > min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object > groups <- c(3,3,3,4) > gen.constr$groups <- groups > gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we > would remove box constraints from the constraint() function and create > an add.constraint() function to add/update box, group, and other > constraint types as applicable. > > # Create the initial constraint object > # Pass in a character vector of asset names for assets > # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) > gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less than 0.5 > gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets > # Specify the group weights as a vector > # The first 3 assets shall have a group weight of greater than 0.15 and > less than 0.3 > # The next 3 assets shall have a group weight of greater than 0.1 and > less than 0.35 > # The last 4 assets shall have a group weight greater than 0 and less > than 0.4 > constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we can > revisit this so that I can hit the ground running when finals are over > and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year?s GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let?s review that after the end of next week > when final exams and the spring quarter are completely behind us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > ** ** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Mon Jun 17 20:17:33 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 17 Jun 2013 11:17:33 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> Message-ID: <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Ross and Brian, Ross, have you or Brian had a chance to take a careful look at the code (attached) that Kirk Li developed for my class this spring to see if the ideas there may be useful? I note that add.constraint needs to check for compatibility with the existing constraint object, and give a sensible message if not. See for example the segment "# sum constraint are not combinable with propcost constraint" just below Scenario 10 in the attached constraint.sets.test.r. At the moment I have forgotten what the constraint set conventions are in PortfolioAnalytics, and will look at it again on Wed. But I think we need the something like the types that are in constraint.sets.r attached. Ross, since you have seen these before in computing homework problems, so you may already have some thoughts about this? FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use solve.QP and Rglpk_solve_LP for a little bit in the first computing chapter, but then quickly move to PortfolioAnalytics for everything we do that will be supported by PortfolioAnlytics. The goal is that MPO2 will equally well support instructional pedagogy and real-world applications use. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 10:27 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() * Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() * Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() * Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() * Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() * Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() * Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() * Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.r Type: application/octet-stream Size: 11673 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.test.r Type: application/octet-stream Size: 5851 bytes Desc: not available URL: From martinrd at comcast.net Mon Jun 17 20:17:33 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 17 Jun 2013 11:17:33 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> Message-ID: <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Ross and Brian, Ross, have you or Brian had a chance to take a careful look at the code (attached) that Kirk Li developed for my class this spring to see if the ideas there may be useful? I note that add.constraint needs to check for compatibility with the existing constraint object, and give a sensible message if not. See for example the segment "# sum constraint are not combinable with propcost constraint" just below Scenario 10 in the attached constraint.sets.test.r. At the moment I have forgotten what the constraint set conventions are in PortfolioAnalytics, and will look at it again on Wed. But I think we need the something like the types that are in constraint.sets.r attached. Ross, since you have seen these before in computing homework problems, so you may already have some thoughts about this? FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use solve.QP and Rglpk_solve_LP for a little bit in the first computing chapter, but then quickly move to PortfolioAnalytics for everything we do that will be supported by PortfolioAnlytics. The goal is that MPO2 will equally well support instructional pedagogy and real-world applications use. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 10:27 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() * Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() * Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() * Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() * Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() * Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() * Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() * Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.r Type: application/octet-stream Size: 11673 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: constraint.sets.test.r Type: application/octet-stream Size: 5851 bytes Desc: not available URL: From rossbennett34 at gmail.com Mon Jun 17 21:36:28 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 14:36:28 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Message-ID: See comments inline below. Regards, Ross Bennett On Mon, Jun 17, 2013 at 1:17 PM, Doug Martin wrote: > Ross and Brian,**** > > ** ** > > Ross, have you or Brian had a chance to take a careful look at the code > (attached) that Kirk Li developed for my class this spring to see if the > ideas there may be useful? > I have gone through the code from Kirk Li, but will take closer look to make sure I did not miss anything. > **** > > ** ** > > I note that *add.constraint* needs to check for compatibility with the > existing constraint object, and give a sensible message if not. See for > example the segment ?# sum constraint are not combinable with propcost > constraint? just below Scenario 10 in the attached * > constraint.sets.test.r.* > The way Kirk's code checks for compatibility constraints should be helpful and hopefully I can borrow some of the ideas. > **** > > ** ** > > At the moment I have forgotten what the constraint set conventions are in > PortfolioAnalytics, and will look at it again on Wed. But I think we need > the something like the types that are in *constraint.sets.r* attached. > Ross, since you have seen these before in computing homework problems, so > you may already have some thoughts about this? > The way that constraints.sets.R handles the constraints is similar to PortfolioAnalytics in that the different constraints are stored in a list object. The main differences I see is how the constraints are combined and passed to the solver. The different functions for the constraints such as cset.box() and cset.lo() do a lot of work setting up the A matrix and b vector for solve.QP(). In PortfolioAnalytics, this is handled inside of optimize.portfolio() and handled differently based on the chosen optimizer. In Kirk Li's code, all of the constraints are fed into gmv(), which if I recall correctly is just a wrapper around solve.QP(). So this means the constraints are pretty specific to one solver. The way the constraints are specified in PortfolioAnalytics currently, the same constraint object can be passed to the DEoptim, ROI (Rglpk and quadprog), Random Portfolios, pso, and GenSA solvers. > **** > > ** ** > > FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use > solve.QP and Rglpk_solve_LP for a little bit in the first computing > chapter, but then quickly move to PortfolioAnalytics for everything we do > that will be supported by PortfolioAnlytics. The goal is that MPO2 will > equally well support instructional pedagogy and real-world applications use. > Very cool that you plan on using PortfolioAnalytics in the 2nd edition of your book! > **** > > ** ** > > Doug**** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 10:27 AM > > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > ** ** > > All,**** > > ** ** > > With finals being over (for me at least, I do realize that Prof. Yollin > and Prof. Martin still have grading) I am anxious to get started today with > the first day of coding. **** > > ** ** > > When I am writing or editing code, should I be working directly in the > PortfolioAnalytics/R files or should I work in the sandbox folder so that I > have a safe area to "play" without worrying about messing up existing code? > **** > > ** ** > > With the recent discussions of separating the objectives from the > constraints and adding a portfoliospec object, here is a list of > functions/features I am proposing to add to accomplish this.**** > > ** ** > > constraint_v2()**** > > - Constraints object constructor as an S3 object that will hold the > constraints. Similar to the current constraints_v1 object, but will keep > separate from objectives.**** > > add.constraint()**** > > - Helper function to add constraints to the constraint_v2 object such > as box, group, etc. constraints.**** > > box_constraint()**** > > - Function to be called by add.constraint that will set minimum and > maximum weight for box constraints**** > > group_constraint()**** > > - Function to be called by add.constraint that will set minimum and > maximum group weights for group constraints**** > > objective_v2()**** > > - Objective object constructor as an S3 object that will hold the > objectives. Similar to the current constraints_v1 object, but will be > separate from constraints.**** > > add.objective()**** > > - Helper function to add objectives to the objective object.**** > > Note that there is currently an objective constructor of class=objclass > and add.objective() function. These currently update the constraints_v1 > object with the objectives. I should be able to re-use most of these and > make revisions so that they update the objectives object.**** > > ** ** > > portfoliospec()**** > > - Portfolio specification constructor that holds the constraints and > objectives. A constraint_v2 object and objective_v2 object will be passed > in to create the portfoliospec object.**** > > may need additional helper functions for portfoliospec that do checks on > the portfoliospec and/or the constraints and objectives objects that are > passed in.**** > > ** ** > > optimize.portfolio() will have to be modified so that it takes in a > portfoliospec object instead of a constraints object.**** > > ** ** > > Thoughts?**** > > ** ** > > Ross Bennett**** > > ** ** > > ** ** > > On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: > **** > > Brian and Ross,**** > > **** > > I couldn?t agree more with the proposal to separate constraints and > objectives (Brian, as you and I have discussed earlier this year or last > year).**** > > **** > > I will have further comments on checking validity of adding constraints > late tomorrow or early Wed. after I have completed grading of my spring > quarter course.**** > > **** > > Doug**** > > **** > > **** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 7:09 AM > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > **** > > Brian,**** > > **** > > I really like your thoughts on separating constraints and objectives. I > learned objectives and constraints in the same way that you mention them. > Intuitively, it is easier to understand that the user would create the > constraint object, the objective object, and then combine in a portfolio > object.**** > > **** > > Separating out the objectives and creating templates of objectives for > minimum variance and other basic objective functions should make it easy to > understand for the user as well how it is handled with the solvers.**** > > **** > > The way the constraints object looks today seems to be very similar to > what you have in mind for the portfoliospec object. Was that your intent so > that the solvers see pretty much the same object in the call to > optimize.portfolio? Only now the portfoliospec object will be passed > instead of the constraints object, is my understanding of your intentions > correct?**** > > **** > > This is what I am thinking the code will look *something* like to run a > simple optimization problem.**** > > **** > > #### Example R code**** > > **** > > # Specify a constraints object and add box constraints**** > > gen.constr = constraint(assets, min_sum=1, max_sum=1)**** > > gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, > type="box")**** > > **** > > # Specify an objective object for global minimum variance**** > > gen.obj = objective(type="GMV")**** > > **** > > # Specify a portfolio object**** > > port.spec = portfoliospec(gen.constr, gen.obj)**** > > **** > > # Run the optimization**** > > solution = optimize.portfolio(data, port.spec)**** > > #####**** > > **** > > Thanks,**** > > Ross Bennett**** > > **** > > On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson > wrote:**** > > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote:**** > > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems that > there is already a lot of the infrastructure in place for the constraint > object and the optimizer so I don't want to maintain the interface to > the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets > gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, > min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object > groups <- c(3,3,3,4) > gen.constr$groups <- groups > gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we > would remove box constraints from the constraint() function and create > an add.constraint() function to add/update box, group, and other > constraint types as applicable. > > # Create the initial constraint object > # Pass in a character vector of asset names for assets > # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) > gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less than 0.5 > gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets > # Specify the group weights as a vector > # The first 3 assets shall have a group weight of greater than 0.15 and > less than 0.3 > # The next 3 assets shall have a group weight of greater than 0.1 and > less than 0.35 > # The last 4 assets shall have a group weight greater than 0 and less > than 0.4 > constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we can > revisit this so that I can hit the ground running when finals are over > and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year?s GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let?s review that after the end of next week > when final exams and the spring quarter are completely behind us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > **** > > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > ** ** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 17 21:36:28 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 17 Jun 2013 14:36:28 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Message-ID: See comments inline below. Regards, Ross Bennett On Mon, Jun 17, 2013 at 1:17 PM, Doug Martin wrote: > Ross and Brian,**** > > ** ** > > Ross, have you or Brian had a chance to take a careful look at the code > (attached) that Kirk Li developed for my class this spring to see if the > ideas there may be useful? > I have gone through the code from Kirk Li, but will take closer look to make sure I did not miss anything. > **** > > ** ** > > I note that *add.constraint* needs to check for compatibility with the > existing constraint object, and give a sensible message if not. See for > example the segment ?# sum constraint are not combinable with propcost > constraint? just below Scenario 10 in the attached * > constraint.sets.test.r.* > The way Kirk's code checks for compatibility constraints should be helpful and hopefully I can borrow some of the ideas. > **** > > ** ** > > At the moment I have forgotten what the constraint set conventions are in > PortfolioAnalytics, and will look at it again on Wed. But I think we need > the something like the types that are in *constraint.sets.r* attached. > Ross, since you have seen these before in computing homework problems, so > you may already have some thoughts about this? > The way that constraints.sets.R handles the constraints is similar to PortfolioAnalytics in that the different constraints are stored in a list object. The main differences I see is how the constraints are combined and passed to the solver. The different functions for the constraints such as cset.box() and cset.lo() do a lot of work setting up the A matrix and b vector for solve.QP(). In PortfolioAnalytics, this is handled inside of optimize.portfolio() and handled differently based on the chosen optimizer. In Kirk Li's code, all of the constraints are fed into gmv(), which if I recall correctly is just a wrapper around solve.QP(). So this means the constraints are pretty specific to one solver. The way the constraints are specified in PortfolioAnalytics currently, the same constraint object can be passed to the DEoptim, ROI (Rglpk and quadprog), Random Portfolios, pso, and GenSA solvers. > **** > > ** ** > > FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use > solve.QP and Rglpk_solve_LP for a little bit in the first computing > chapter, but then quickly move to PortfolioAnalytics for everything we do > that will be supported by PortfolioAnlytics. The goal is that MPO2 will > equally well support instructional pedagogy and real-world applications use. > Very cool that you plan on using PortfolioAnalytics in the 2nd edition of your book! > **** > > ** ** > > Doug**** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 10:27 AM > > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > ** ** > > All,**** > > ** ** > > With finals being over (for me at least, I do realize that Prof. Yollin > and Prof. Martin still have grading) I am anxious to get started today with > the first day of coding. **** > > ** ** > > When I am writing or editing code, should I be working directly in the > PortfolioAnalytics/R files or should I work in the sandbox folder so that I > have a safe area to "play" without worrying about messing up existing code? > **** > > ** ** > > With the recent discussions of separating the objectives from the > constraints and adding a portfoliospec object, here is a list of > functions/features I am proposing to add to accomplish this.**** > > ** ** > > constraint_v2()**** > > - Constraints object constructor as an S3 object that will hold the > constraints. Similar to the current constraints_v1 object, but will keep > separate from objectives.**** > > add.constraint()**** > > - Helper function to add constraints to the constraint_v2 object such > as box, group, etc. constraints.**** > > box_constraint()**** > > - Function to be called by add.constraint that will set minimum and > maximum weight for box constraints**** > > group_constraint()**** > > - Function to be called by add.constraint that will set minimum and > maximum group weights for group constraints**** > > objective_v2()**** > > - Objective object constructor as an S3 object that will hold the > objectives. Similar to the current constraints_v1 object, but will be > separate from constraints.**** > > add.objective()**** > > - Helper function to add objectives to the objective object.**** > > Note that there is currently an objective constructor of class=objclass > and add.objective() function. These currently update the constraints_v1 > object with the objectives. I should be able to re-use most of these and > make revisions so that they update the objectives object.**** > > ** ** > > portfoliospec()**** > > - Portfolio specification constructor that holds the constraints and > objectives. A constraint_v2 object and objective_v2 object will be passed > in to create the portfoliospec object.**** > > may need additional helper functions for portfoliospec that do checks on > the portfoliospec and/or the constraints and objectives objects that are > passed in.**** > > ** ** > > optimize.portfolio() will have to be modified so that it takes in a > portfoliospec object instead of a constraints object.**** > > ** ** > > Thoughts?**** > > ** ** > > Ross Bennett**** > > ** ** > > ** ** > > On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: > **** > > Brian and Ross,**** > > **** > > I couldn?t agree more with the proposal to separate constraints and > objectives (Brian, as you and I have discussed earlier this year or last > year).**** > > **** > > I will have further comments on checking validity of adding constraints > late tomorrow or early Wed. after I have completed grading of my spring > quarter course.**** > > **** > > Doug**** > > **** > > **** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 17, 2013 7:09 AM > *To:* PortfolioAnalytics > *Cc:* gsoc-porta at r-forge.wu-wien.ac.at > *Subject:* Re: [GSoC-PortA] structure of the specification objects**** > > **** > > Brian,**** > > **** > > I really like your thoughts on separating constraints and objectives. I > learned objectives and constraints in the same way that you mention them. > Intuitively, it is easier to understand that the user would create the > constraint object, the objective object, and then combine in a portfolio > object.**** > > **** > > Separating out the objectives and creating templates of objectives for > minimum variance and other basic objective functions should make it easy to > understand for the user as well how it is handled with the solvers.**** > > **** > > The way the constraints object looks today seems to be very similar to > what you have in mind for the portfoliospec object. Was that your intent so > that the solvers see pretty much the same object in the call to > optimize.portfolio? Only now the portfoliospec object will be passed > instead of the constraints object, is my understanding of your intentions > correct?**** > > **** > > This is what I am thinking the code will look *something* like to run a > simple optimization problem.**** > > **** > > #### Example R code**** > > **** > > # Specify a constraints object and add box constraints**** > > gen.constr = constraint(assets, min_sum=1, max_sum=1)**** > > gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, > type="box")**** > > **** > > # Specify an objective object for global minimum variance**** > > gen.obj = objective(type="GMV")**** > > **** > > # Specify a portfolio object**** > > port.spec = portfoliospec(gen.constr, gen.obj)**** > > **** > > # Run the optimization**** > > solution = optimize.portfolio(data, port.spec)**** > > #####**** > > **** > > Thanks,**** > > Ross Bennett**** > > **** > > On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson > wrote:**** > > I've been thinking a lot about this over the last week, and I'm going to > propose an architecture for discussion. > > Right now, we have an object of type 'constraints' which also includes > 'objectives'. > > I have always logically separated objectives from constraints in my head > by saying that > - constraints are something that are: > 1. operable only on the weight vector and > 2. metadata information about the portfolio (e.g. groups, asset names) > 3. describe the *inputs* to the objective function > > - objectives are part of the *objective function* that > 1. you calculate on the weight vector > 2. describe business objectives for the *outcome* of the optimization > > I think we need to split these apart explicitly, and make the > distinctions, and why they exist, more obvious to the user. > > First, I think we need a container object of type 'portfolio' or > 'portfoliospec' that know what assets you have, and how you want to group > them (e.g. sectors, size, whatever). This is data about the portfolio, and > is *independent* of and constraints or objectives you might have on that > portfolio. > > Next, I think we need a constraints object. Constraints, by their nature, > are tied to the portfolio that they are created for. They require > knowledge of the portfolio metadata, or at the very least things like box > constraints are typically derived from the size of the portfolio. > > Finally, I think we need a separate 'objectives' object. To have > objectives objects, with names, that specify things like minimum variance, > maximum Sharpe ratio, or risk parity objectives. These can be completely > separate from the assets in the portfolio, and we could provide a number of > template objective objects with the package. I think the current way > objectives are added and manipulated is fine, we just need to break it off > a bit. > > For the template objectives that can be acted on by specific more > efficient solvers, I think separating the objectives out will make it > easier to identify that. > > optimize.portfolio needs all three of these components, even today, to do > anything. So I think we need to sort out how to specify them, then how to > combine them. > > I suggest that we start by getting these specifications right first. Then > we can start surgery in other parts of the code to use the new > specification forms. > > My current thoughts are that a portfolio *contains* one or more > constraints objects, since the constraints objects are dependent on the > portfolio composition. Then you optimize a portfolio by passing a > portfolio specification into the optimizer, and telling it which > constraints object to use (if you have more than one attached to the > portfolio) and what objectives object you want to apply. > > Hopefully this is clear. It's a proposal, so comments/critique/ideas > always welcome. > > Today is officially the first day of coding, so we need to get moving. > > Regards, > > Brian > > > > > On 06/06/2013 06:54 AM, Ross Bennett wrote:**** > > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems that > there is already a lot of the infrastructure in place for the constraint > object and the optimizer so I don't want to maintain the interface to > the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets > gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, > min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object > groups <- c(3,3,3,4) > gen.constr$groups <- groups > gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we > would remove box constraints from the constraint() function and create > an add.constraint() function to add/update box, group, and other > constraint types as applicable. > > # Create the initial constraint object > # Pass in a character vector of asset names for assets > # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) > gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less than 0.5 > gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets > # Specify the group weights as a vector > # The first 3 assets shall have a group weight of greater than 0.15 and > less than 0.3 > # The next 3 assets shall have a group weight of greater than 0.1 and > less than 0.35 > # The last 4 assets shall have a group weight greater than 0 and less > than 0.4 > constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we can > revisit this so that I can hit the ground running when finals are over > and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year?s GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let?s review that after the end of next week > when final exams and the spring quarter are completely behind us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > **** > > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > ** ** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Mon Jun 17 22:41:59 2013 From: brian at braverock.com (Brian G. Peterson) Date: Mon, 17 Jun 2013 15:41:59 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> Message-ID: <51BF7497.8080802@braverock.com> On 06/17/2013 12:26 PM, Ross Bennett wrote: > With finals being over (for me at least, I do realize that Prof. Yollin > and Prof. Martin still have grading) I am anxious to get started today > with the first day of coding. > > When I am writing or editing code, should I be working directly in the > PortfolioAnalytics/R files or should I work in the sandbox folder so > that I have a safe area to "play" without worrying about messing up > existing code? Put it in the trunk. We can't really work on this off in a corner, so it's either the trunk or a branch. I'm comfortable with the trunk. that's why we're using version control. > With the recent discussions of separating the objectives from the > constraints and adding a portfoliospec object, here is a list of > functions/features I am proposing to add to accomplish this. The list in your email looks like a good place to start. We don't really risk breaking major any existing code until we move constraint() to constraint_v1 and start messing with the insides of optimize.portfolio. we can talk more about the design for that once we have a working portfoliospec, constraint_v2, objective_v2, etc. -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From kris.boudt at kuleuven.be Tue Jun 18 10:34:50 2013 From: kris.boudt at kuleuven.be (Kris Boudt) Date: Tue, 18 Jun 2013 08:34:50 +0000 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51BEF82E.5030402@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From kris.boudt at kuleuven.be Tue Jun 18 10:34:50 2013 From: kris.boudt at kuleuven.be (Kris Boudt) Date: Tue, 18 Jun 2013 08:34:50 +0000 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51BEF82E.5030402@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> Message-ID: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > ] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From brian at braverock.com Tue Jun 18 11:28:34 2013 From: brian at braverock.com (Brian G. Peterson) Date: Tue, 18 Jun 2013 04:28:34 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Message-ID: <51C02842.3040101@braverock.com> On 06/18/2013 03:34 AM, Kris Boudt wrote: > Intrinsically, it means that what we used to call "soft constraints" > (the constraints that can be violated) are classified as > "objectives". Seems good to me, but we need to make sure that each > objective has then its multiplier/scaling factor, which by default is > 1, but for soft constraints will typically be higher. I've never liked the 'soft constraints' construct, as it's not very clear. My distinction was between things we can do *before* calling the objective function, only on the weights vector, which we'll call 'constraints' and things that we can do only by acting on the set of all possible weights vectors which satisfy the constraints (the objectives). We've always supported both multipliers and penalties for the individual objectives, from the very first version of the code. In practice, two things go into selecting a multiplier: a. the objectives must inherently be of similar scale, e.g. both returning a return number of roughly equal magnitude. if they are not of roughly equal magnitude, then the user needs to use the multiplier to adjust. b. the user must understand that to over/under weight a specific objective with respect to other objectives, they need to adjust either the multiplier or the penalty, or both. > 1. I would also include a turnover objective, whereby there seem to > be two possibilities: > > - provide directly the weights at the time of rebalancing - provide > the weights at the previous rebalancing data and include a function > that, uses the return data to calculate the weights at the time of > rebalancing. We demonstrated how to use a turnover objective function in our 2012 R/Finance seminar: http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf see page 36 with code here: https://r-forge.r-project.org/scm/viewvc.php/*checkout*/pkg/PortfolioAnalytics/sandbox/script.workshop2010.R?root=returnanalytics It would be nice to get the chart from page 36 into the package if we have time later in the summer. In practice, turnover can be dealt with either as a constraint (in some solvers) or as an objective. Doug would need to comment on using it as a constraint, I know there's a brief discussion of that in MPO(v1) or perhaps Bernd's book, both of which I have at the office right now. I've always done it as an objective, which works fine in current code. It would probably be nice to include the turnover function in the package directly and document it. > 2. In practical work, I work with a price data file that has all > possible constituents and for each selection date a list of allowed > constituents. Typically an xts with in row the selection dates and > columns the assets with 1 if included and 0 otherwise. Such a > constituents check could be in the constraints object. It seems to me that this belongs in the portfolio specification. That is to say that the constituents of the portfolio are defined in the portfolio specification as a time series. Another way to think of this, of course, is to say that an equality constraint (weight=0) is imposed on some of the overall universe of constituents at certain times. You can also do it with box constraints that vary through time. I think I prefer the first approach as more intuitive for the user, and dealing with subsetting the input data to match the allowed constituents within optimize.portfolio. We're better off never giving data to the solvers that they don't need, and if the constituent is out of the portfolio, then the subset can happen before the optimization step. That's just an initial feeling though. I think we need to think more about what it does to the code to support time-varying constraints and portfolio specifications, and then put it wherever it makes the most sense. I don't see any reason to work on that this week though, as the overall structure seems more important to me at this point, and I think we can cover this when we have a more coherent framework to work from. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From rossbennett34 at gmail.com Tue Jun 18 18:17:40 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Tue, 18 Jun 2013 11:17:40 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51C02842.3040101@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> <51C02842.3040101@braverock.com> Message-ID: Thank you for the detailed information and examples of working with turnover as an objective function and working with subsetting data based on whether an asset is in or out in a given time period. I am looking for clarification and have a couple thoughts on the structure of the specification object and how a user would go about specifying the objects. Method 1 - Create a portfolio object - Create a constraint object - Add constraints to the constraint object - Create an objective object - Add objectives to the objective object - Add the constraint object and objective object to the portfolio object This has the advantage of being more flexible to work with because of the separate constraint and objective objects, but the disadvantage is the multiple steps. The way add.constraint() and add.objective() are currently, they need knowledge of the assets for box_constraint, group_constraint(), and risk_budget_objective(). For example, the box_constraints() function uses the number of assets to make a vector of the minimum and maximum weights if a scalar is passed in for min and max. Or would it make more sense to do these checks and setup inside an update.portfolio() function. The code would look something like this pspec <- portfolio.spec(assets) constr <- constraint(pspec) constr <- add.constraint(constr, "box", min=0, max=1) obj <- objective(pspec) obj <- add.objective(obj, type="risk", name="var") pspec <- update.portfolio(pspec, constr, obj) optimize.portfolio(pspec) Method 2 - Create a portfolio object - Add constraints to the portfolio object - Add objectives to the portfolio object The code would look something like this pspec <- portfolio.spec(assets) constr <- add.constraint(pspec, "box", min=0, max=1) obj <- add.objective(obj, type="risk", name="var") optimize.portfolio(pspec) This is less flexible and wouldn't allow someone to "re-use" objective or constraints. The current objective functions could be re-used and would just be added to a list in a portfolio.spec object instead of the current constraint object. The structure of the portfolio.spec object would be similar to the current constraint object so we shouldn't need too many changes to optimize.portfolio. Please advise as to the direction I should proceed and any thoughts on this. Thanks, Ross Bennett On Tue, Jun 18, 2013 at 4:28 AM, Brian G. Peterson wrote: > On 06/18/2013 03:34 AM, Kris Boudt wrote: > >> Intrinsically, it means that what we used to call "soft constraints" >> (the constraints that can be violated) are classified as >> "objectives". Seems good to me, but we need to make sure that each >> objective has then its multiplier/scaling factor, which by default is >> 1, but for soft constraints will typically be higher. >> > > I've never liked the 'soft constraints' construct, as it's not very > clear. My distinction was between things we can do *before* calling the > objective function, only on the weights vector, which we'll call > 'constraints' and things that we can do only by acting on the set of all > possible weights vectors which satisfy the constraints (the objectives). > > We've always supported both multipliers and penalties for the individual > objectives, from the very first version of the code. In practice, two > things go into selecting a multiplier: > > a. the objectives must inherently be of similar scale, e.g. both returning > a return number of roughly equal magnitude. if they are not of roughly > equal magnitude, then the user needs to use the multiplier to adjust. > > b. the user must understand that to over/under weight a specific objective > with respect to other objectives, they need to adjust either the multiplier > or the penalty, or both. > > > 1. I would also include a turnover objective, whereby there seem to >> be two possibilities: >> >> - provide directly the weights at the time of rebalancing - provide >> the weights at the previous rebalancing data and include a function >> that, uses the return data to calculate the weights at the time of >> rebalancing. >> > > We demonstrated how to use a turnover objective function in our 2012 > R/Finance seminar: > > http://www.rinfinance.com/**agenda/2010/Carl+Peterson+**Boudt_Tutorial.pdf > > see page 36 > > with code here: > https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** > PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics > > It would be nice to get the chart from page 36 into the package if we have > time later in the summer. > > In practice, turnover can be dealt with either as a constraint (in some > solvers) or as an objective. Doug would need to comment on using it as a > constraint, I know there's a brief discussion of that in MPO(v1) or perhaps > Bernd's book, both of which I have at the office right now. > > I've always done it as an objective, which works fine in current code. > > It would probably be nice to include the turnover function in the package > directly and document it. > > > 2. In practical work, I work with a price data file that has all >> possible constituents and for each selection date a list of allowed >> constituents. Typically an xts with in row the selection dates and >> columns the assets with 1 if included and 0 otherwise. Such a >> constituents check could be in the constraints object. >> > > It seems to me that this belongs in the portfolio specification. That is > to say that the constituents of the portfolio are defined in the portfolio > specification as a time series. > > Another way to think of this, of course, is to say that an equality > constraint (weight=0) is imposed on some of the overall universe of > constituents at certain times. You can also do it with box constraints > that vary through time. > > I think I prefer the first approach as more intuitive for the user, and > dealing with subsetting the input data to match the allowed constituents > within optimize.portfolio. We're better off never giving data to the > solvers that they don't need, and if the constituent is out of the > portfolio, then the subset can happen before the optimization step. That's > just an initial feeling though. > > I think we need to think more about what it does to the code to support > time-varying constraints and portfolio specifications, and then put it > wherever it makes the most sense. I don't see any reason to work on that > this week though, as the overall structure seems more important to me at > this point, and I think we can cover this when we have a more coherent > framework to work from. > > Regards, > > Brian > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Tue Jun 18 18:17:40 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Tue, 18 Jun 2013 11:17:40 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51C02842.3040101@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> <51C02842.3040101@braverock.com> Message-ID: Thank you for the detailed information and examples of working with turnover as an objective function and working with subsetting data based on whether an asset is in or out in a given time period. I am looking for clarification and have a couple thoughts on the structure of the specification object and how a user would go about specifying the objects. Method 1 - Create a portfolio object - Create a constraint object - Add constraints to the constraint object - Create an objective object - Add objectives to the objective object - Add the constraint object and objective object to the portfolio object This has the advantage of being more flexible to work with because of the separate constraint and objective objects, but the disadvantage is the multiple steps. The way add.constraint() and add.objective() are currently, they need knowledge of the assets for box_constraint, group_constraint(), and risk_budget_objective(). For example, the box_constraints() function uses the number of assets to make a vector of the minimum and maximum weights if a scalar is passed in for min and max. Or would it make more sense to do these checks and setup inside an update.portfolio() function. The code would look something like this pspec <- portfolio.spec(assets) constr <- constraint(pspec) constr <- add.constraint(constr, "box", min=0, max=1) obj <- objective(pspec) obj <- add.objective(obj, type="risk", name="var") pspec <- update.portfolio(pspec, constr, obj) optimize.portfolio(pspec) Method 2 - Create a portfolio object - Add constraints to the portfolio object - Add objectives to the portfolio object The code would look something like this pspec <- portfolio.spec(assets) constr <- add.constraint(pspec, "box", min=0, max=1) obj <- add.objective(obj, type="risk", name="var") optimize.portfolio(pspec) This is less flexible and wouldn't allow someone to "re-use" objective or constraints. The current objective functions could be re-used and would just be added to a list in a portfolio.spec object instead of the current constraint object. The structure of the portfolio.spec object would be similar to the current constraint object so we shouldn't need too many changes to optimize.portfolio. Please advise as to the direction I should proceed and any thoughts on this. Thanks, Ross Bennett On Tue, Jun 18, 2013 at 4:28 AM, Brian G. Peterson wrote: > On 06/18/2013 03:34 AM, Kris Boudt wrote: > >> Intrinsically, it means that what we used to call "soft constraints" >> (the constraints that can be violated) are classified as >> "objectives". Seems good to me, but we need to make sure that each >> objective has then its multiplier/scaling factor, which by default is >> 1, but for soft constraints will typically be higher. >> > > I've never liked the 'soft constraints' construct, as it's not very > clear. My distinction was between things we can do *before* calling the > objective function, only on the weights vector, which we'll call > 'constraints' and things that we can do only by acting on the set of all > possible weights vectors which satisfy the constraints (the objectives). > > We've always supported both multipliers and penalties for the individual > objectives, from the very first version of the code. In practice, two > things go into selecting a multiplier: > > a. the objectives must inherently be of similar scale, e.g. both returning > a return number of roughly equal magnitude. if they are not of roughly > equal magnitude, then the user needs to use the multiplier to adjust. > > b. the user must understand that to over/under weight a specific objective > with respect to other objectives, they need to adjust either the multiplier > or the penalty, or both. > > > 1. I would also include a turnover objective, whereby there seem to >> be two possibilities: >> >> - provide directly the weights at the time of rebalancing - provide >> the weights at the previous rebalancing data and include a function >> that, uses the return data to calculate the weights at the time of >> rebalancing. >> > > We demonstrated how to use a turnover objective function in our 2012 > R/Finance seminar: > > http://www.rinfinance.com/**agenda/2010/Carl+Peterson+**Boudt_Tutorial.pdf > > see page 36 > > with code here: > https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** > PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics > > It would be nice to get the chart from page 36 into the package if we have > time later in the summer. > > In practice, turnover can be dealt with either as a constraint (in some > solvers) or as an objective. Doug would need to comment on using it as a > constraint, I know there's a brief discussion of that in MPO(v1) or perhaps > Bernd's book, both of which I have at the office right now. > > I've always done it as an objective, which works fine in current code. > > It would probably be nice to include the turnover function in the package > directly and document it. > > > 2. In practical work, I work with a price data file that has all >> possible constituents and for each selection date a list of allowed >> constituents. Typically an xts with in row the selection dates and >> columns the assets with 1 if included and 0 otherwise. Such a >> constituents check could be in the constraints object. >> > > It seems to me that this belongs in the portfolio specification. That is > to say that the constituents of the portfolio are defined in the portfolio > specification as a time series. > > Another way to think of this, of course, is to say that an equality > constraint (weight=0) is imposed on some of the overall universe of > constituents at certain times. You can also do it with box constraints > that vary through time. > > I think I prefer the first approach as more intuitive for the user, and > dealing with subsetting the input data to match the allowed constituents > within optimize.portfolio. We're better off never giving data to the > solvers that they don't need, and if the constituent is out of the > portfolio, then the subset can happen before the optimization step. That's > just an initial feeling though. > > I think we need to think more about what it does to the code to support > time-varying constraints and portfolio specifications, and then put it > wherever it makes the most sense. I don't see any reason to work on that > this week though, as the overall structure seems more important to me at > this point, and I think we can cover this when we have a more coherent > framework to work from. > > Regards, > > Brian > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Wed Jun 19 03:35:50 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Tue, 18 Jun 2013 20:35:50 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> <51C02842.3040101@braverock.com> Message-ID: All, I have made a series of commits this evening to add constraint objects and objective objects to a portfolio object. See the file sandbox/testing_portfolio_specification.R for a demonstration of how it works. Now that I have fully working portfolio object, I think we are at a pretty good point with the framework for creating the portfolio specification object for feedback on the good parts and the bad parts. I look forward to your feedback. Regards, Ross Bennett On Tue, Jun 18, 2013 at 11:17 AM, Ross Bennett wrote: > Thank you for the detailed information and examples of working with > turnover as an objective function and working with subsetting data based on > whether an asset is in or out in a given time period. > > I am looking for clarification and have a couple thoughts on the structure > of the specification object and how a user would go about specifying the > objects. > > Method 1 > > - Create a portfolio object > - Create a constraint object > - Add constraints to the constraint object > - Create an objective object > - Add objectives to the objective object > - Add the constraint object and objective object to the portfolio > object > > This has the advantage of being more flexible to work with because of the > separate constraint and objective objects, but the disadvantage is the > multiple steps. The way add.constraint() and add.objective() are currently, > they need knowledge of the assets for box_constraint, group_constraint(), > and risk_budget_objective(). For example, the box_constraints() function > uses the number of assets to make a vector of the minimum and maximum > weights if a scalar is passed in for min and max. Or would it make more > sense to do these checks and setup inside an update.portfolio() function. > > The code would look something like this > pspec <- portfolio.spec(assets) > constr <- constraint(pspec) > constr <- add.constraint(constr, "box", min=0, max=1) > obj <- objective(pspec) > obj <- add.objective(obj, type="risk", name="var") > pspec <- update.portfolio(pspec, constr, obj) > optimize.portfolio(pspec) > > Method 2 > > - Create a portfolio object > - Add constraints to the portfolio object > - Add objectives to the portfolio object > > The code would look something like this > pspec <- portfolio.spec(assets) > constr <- add.constraint(pspec, "box", min=0, max=1) > obj <- add.objective(obj, type="risk", name="var") > optimize.portfolio(pspec) > > This is less flexible and wouldn't allow someone to "re-use" objective or > constraints. The current objective functions could be re-used and would > just be added to a list in a portfolio.spec object instead of the current > constraint object. The structure of the portfolio.spec object would be > similar to the current constraint object so we shouldn't need too many > changes to optimize.portfolio. > > Please advise as to the direction I should proceed and any thoughts on > this. > > Thanks, > Ross Bennett > > > On Tue, Jun 18, 2013 at 4:28 AM, Brian G. Peterson wrote: > >> On 06/18/2013 03:34 AM, Kris Boudt wrote: >> >>> Intrinsically, it means that what we used to call "soft constraints" >>> (the constraints that can be violated) are classified as >>> "objectives". Seems good to me, but we need to make sure that each >>> objective has then its multiplier/scaling factor, which by default is >>> 1, but for soft constraints will typically be higher. >>> >> >> I've never liked the 'soft constraints' construct, as it's not very >> clear. My distinction was between things we can do *before* calling the >> objective function, only on the weights vector, which we'll call >> 'constraints' and things that we can do only by acting on the set of all >> possible weights vectors which satisfy the constraints (the objectives). >> >> We've always supported both multipliers and penalties for the individual >> objectives, from the very first version of the code. In practice, two >> things go into selecting a multiplier: >> >> a. the objectives must inherently be of similar scale, e.g. both >> returning a return number of roughly equal magnitude. if they are not of >> roughly equal magnitude, then the user needs to use the multiplier to >> adjust. >> >> b. the user must understand that to over/under weight a specific >> objective with respect to other objectives, they need to adjust either the >> multiplier or the penalty, or both. >> >> >> 1. I would also include a turnover objective, whereby there seem to >>> be two possibilities: >>> >>> - provide directly the weights at the time of rebalancing - provide >>> the weights at the previous rebalancing data and include a function >>> that, uses the return data to calculate the weights at the time of >>> rebalancing. >>> >> >> We demonstrated how to use a turnover objective function in our 2012 >> R/Finance seminar: >> >> http://www.rinfinance.com/**agenda/2010/Carl+Peterson+** >> Boudt_Tutorial.pdf >> >> see page 36 >> >> with code here: >> https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** >> PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics >> >> It would be nice to get the chart from page 36 into the package if we >> have time later in the summer. >> >> In practice, turnover can be dealt with either as a constraint (in some >> solvers) or as an objective. Doug would need to comment on using it as a >> constraint, I know there's a brief discussion of that in MPO(v1) or perhaps >> Bernd's book, both of which I have at the office right now. >> >> I've always done it as an objective, which works fine in current code. >> >> It would probably be nice to include the turnover function in the package >> directly and document it. >> >> >> 2. In practical work, I work with a price data file that has all >>> possible constituents and for each selection date a list of allowed >>> constituents. Typically an xts with in row the selection dates and >>> columns the assets with 1 if included and 0 otherwise. Such a >>> constituents check could be in the constraints object. >>> >> >> It seems to me that this belongs in the portfolio specification. That is >> to say that the constituents of the portfolio are defined in the portfolio >> specification as a time series. >> >> Another way to think of this, of course, is to say that an equality >> constraint (weight=0) is imposed on some of the overall universe of >> constituents at certain times. You can also do it with box constraints >> that vary through time. >> >> I think I prefer the first approach as more intuitive for the user, and >> dealing with subsetting the input data to match the allowed constituents >> within optimize.portfolio. We're better off never giving data to the >> solvers that they don't need, and if the constituent is out of the >> portfolio, then the subset can happen before the optimization step. That's >> just an initial feeling though. >> >> I think we need to think more about what it does to the code to support >> time-varying constraints and portfolio specifications, and then put it >> wherever it makes the most sense. I don't see any reason to work on that >> this week though, as the overall structure seems more important to me at >> this point, and I think we can cover this when we have a more coherent >> framework to work from. >> >> Regards, >> >> Brian >> >> >> -- >> Brian G. Peterson >> http://braverock.com/brian/ >> Ph: 773-459-4973 >> IM: bgpbraverock >> ______________________________**_________________ >> GSoC-PortA mailing list >> GSoC-PortA at lists.r-forge.r-**project.org >> http://lists.r-forge.r-**project.org/cgi-bin/mailman/** >> listinfo/gsoc-porta >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Wed Jun 19 03:35:50 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Tue, 18 Jun 2013 20:35:50 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> <51C02842.3040101@braverock.com> Message-ID: All, I have made a series of commits this evening to add constraint objects and objective objects to a portfolio object. See the file sandbox/testing_portfolio_specification.R for a demonstration of how it works. Now that I have fully working portfolio object, I think we are at a pretty good point with the framework for creating the portfolio specification object for feedback on the good parts and the bad parts. I look forward to your feedback. Regards, Ross Bennett On Tue, Jun 18, 2013 at 11:17 AM, Ross Bennett wrote: > Thank you for the detailed information and examples of working with > turnover as an objective function and working with subsetting data based on > whether an asset is in or out in a given time period. > > I am looking for clarification and have a couple thoughts on the structure > of the specification object and how a user would go about specifying the > objects. > > Method 1 > > - Create a portfolio object > - Create a constraint object > - Add constraints to the constraint object > - Create an objective object > - Add objectives to the objective object > - Add the constraint object and objective object to the portfolio > object > > This has the advantage of being more flexible to work with because of the > separate constraint and objective objects, but the disadvantage is the > multiple steps. The way add.constraint() and add.objective() are currently, > they need knowledge of the assets for box_constraint, group_constraint(), > and risk_budget_objective(). For example, the box_constraints() function > uses the number of assets to make a vector of the minimum and maximum > weights if a scalar is passed in for min and max. Or would it make more > sense to do these checks and setup inside an update.portfolio() function. > > The code would look something like this > pspec <- portfolio.spec(assets) > constr <- constraint(pspec) > constr <- add.constraint(constr, "box", min=0, max=1) > obj <- objective(pspec) > obj <- add.objective(obj, type="risk", name="var") > pspec <- update.portfolio(pspec, constr, obj) > optimize.portfolio(pspec) > > Method 2 > > - Create a portfolio object > - Add constraints to the portfolio object > - Add objectives to the portfolio object > > The code would look something like this > pspec <- portfolio.spec(assets) > constr <- add.constraint(pspec, "box", min=0, max=1) > obj <- add.objective(obj, type="risk", name="var") > optimize.portfolio(pspec) > > This is less flexible and wouldn't allow someone to "re-use" objective or > constraints. The current objective functions could be re-used and would > just be added to a list in a portfolio.spec object instead of the current > constraint object. The structure of the portfolio.spec object would be > similar to the current constraint object so we shouldn't need too many > changes to optimize.portfolio. > > Please advise as to the direction I should proceed and any thoughts on > this. > > Thanks, > Ross Bennett > > > On Tue, Jun 18, 2013 at 4:28 AM, Brian G. Peterson wrote: > >> On 06/18/2013 03:34 AM, Kris Boudt wrote: >> >>> Intrinsically, it means that what we used to call "soft constraints" >>> (the constraints that can be violated) are classified as >>> "objectives". Seems good to me, but we need to make sure that each >>> objective has then its multiplier/scaling factor, which by default is >>> 1, but for soft constraints will typically be higher. >>> >> >> I've never liked the 'soft constraints' construct, as it's not very >> clear. My distinction was between things we can do *before* calling the >> objective function, only on the weights vector, which we'll call >> 'constraints' and things that we can do only by acting on the set of all >> possible weights vectors which satisfy the constraints (the objectives). >> >> We've always supported both multipliers and penalties for the individual >> objectives, from the very first version of the code. In practice, two >> things go into selecting a multiplier: >> >> a. the objectives must inherently be of similar scale, e.g. both >> returning a return number of roughly equal magnitude. if they are not of >> roughly equal magnitude, then the user needs to use the multiplier to >> adjust. >> >> b. the user must understand that to over/under weight a specific >> objective with respect to other objectives, they need to adjust either the >> multiplier or the penalty, or both. >> >> >> 1. I would also include a turnover objective, whereby there seem to >>> be two possibilities: >>> >>> - provide directly the weights at the time of rebalancing - provide >>> the weights at the previous rebalancing data and include a function >>> that, uses the return data to calculate the weights at the time of >>> rebalancing. >>> >> >> We demonstrated how to use a turnover objective function in our 2012 >> R/Finance seminar: >> >> http://www.rinfinance.com/**agenda/2010/Carl+Peterson+** >> Boudt_Tutorial.pdf >> >> see page 36 >> >> with code here: >> https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** >> PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics >> >> It would be nice to get the chart from page 36 into the package if we >> have time later in the summer. >> >> In practice, turnover can be dealt with either as a constraint (in some >> solvers) or as an objective. Doug would need to comment on using it as a >> constraint, I know there's a brief discussion of that in MPO(v1) or perhaps >> Bernd's book, both of which I have at the office right now. >> >> I've always done it as an objective, which works fine in current code. >> >> It would probably be nice to include the turnover function in the package >> directly and document it. >> >> >> 2. In practical work, I work with a price data file that has all >>> possible constituents and for each selection date a list of allowed >>> constituents. Typically an xts with in row the selection dates and >>> columns the assets with 1 if included and 0 otherwise. Such a >>> constituents check could be in the constraints object. >>> >> >> It seems to me that this belongs in the portfolio specification. That is >> to say that the constituents of the portfolio are defined in the portfolio >> specification as a time series. >> >> Another way to think of this, of course, is to say that an equality >> constraint (weight=0) is imposed on some of the overall universe of >> constituents at certain times. You can also do it with box constraints >> that vary through time. >> >> I think I prefer the first approach as more intuitive for the user, and >> dealing with subsetting the input data to match the allowed constituents >> within optimize.portfolio. We're better off never giving data to the >> solvers that they don't need, and if the constituent is out of the >> portfolio, then the subset can happen before the optimization step. That's >> just an initial feeling though. >> >> I think we need to think more about what it does to the code to support >> time-varying constraints and portfolio specifications, and then put it >> wherever it makes the most sense. I don't see any reason to work on that >> this week though, as the overall structure seems more important to me at >> this point, and I think we can cover this when we have a more coherent >> framework to work from. >> >> Regards, >> >> Brian >> >> >> -- >> Brian G. Peterson >> http://braverock.com/brian/ >> Ph: 773-459-4973 >> IM: bgpbraverock >> ______________________________**_________________ >> GSoC-PortA mailing list >> GSoC-PortA at lists.r-forge.r-**project.org >> http://lists.r-forge.r-**project.org/cgi-bin/mailman/** >> listinfo/gsoc-porta >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at braverock.com Wed Jun 19 16:23:18 2013 From: peter at braverock.com (Peter Carl) Date: Wed, 19 Jun 2013 09:23:18 -0500 Subject: [GSoC-PortA] Use case for PortfolioAnalytics Message-ID: Brian suggested that I send a few thoughts on a use-case for PortfolioAnalytics. I'm going to use a real situation, although I'm going to skip over some of the details about *why* some of these issues exist. Apologies in advance for the length - I couldn't restrain myself from talking more broadly around the specifics I lay out here. The workflow I'm presenting here is similar to the one developed in presentations that Brian has already pointed out: - http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf - http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf Many of the things I say here are reflected in those presentations, but I hope I can emphasize a few points here. In this use case, I'm more concerned about describing the decisions than the workflow or the specific data. Portfolio construction is still a frequently neglected part of the investment process. It has, in practice, come a long way since the days of Jessie Livermore-style overbetting (he lost 90% his wealth in 1907 on a blown cotton trade). That said, investors still don't always understand the specific trade-offs they are making in optimization, or the sensitivity the techniques show to the data and estimation error around the inputs. Many, many investors prefer to use simple rules-of-thumb and common sense instead - which work to keep the portfolio out of hot water - not realizing that they could add actually add value during this step by explicitly expressing their objectives and constraints and generating alternatives. In going through this exercise of constructing a portfolio from individual elements, there are a few issues that are important to me as a decision-maker. In general, I want: - to be approximately correct rather than precisely wrong (accuracy of the method should be considered in the context of estimation error); - to define risk as potential loss rather than volatility (even Markowitz wanted this); - the flexibility to define any kind of objective (in fact, I want to compare several); - the ability to combine constraints (I have specific position-level reasons for doing so); - a framework for considering different sets of portfolio constraints for comparison through time; and - to build intuition about optimization through visualization. Many people would add that they are sensitive to the calculation time, as well. Given the rebalancing frequency proposed in this case, calculation time doesn't matter as much in this case. 0. Understand the portfolio constituents Often left unsaid, but this step affects judgments about what comes after. In this case, I have three categories of commodities futures traders - one of which tends to be extremely volatile but uncorrelated among the peers (sector specialists). Another category shows high correlation among themselves, but run relatively low volatility strategies (trend followers). Those strategies can show autocorrelation, however, and can find themselves in extended drawdowns. The third category of manager are somewhere in between those two (macro). We tend to think of these in groups, even where managers don't fit perfectly. 1. Set limits that are position-specific The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: - individual level (e.g., no single manager can represent more than 15% of AUM) or - group level (e.g., no group of managers using Systematic strategies can represent more than 30% of AUM). There may be a specific direction of change: - Managers may not be able to accept additional capital, so that the position may only be reduced from current levels; - Managers may be receiving an allocation subject to a minimum amount of investment; - Positions may be subject to gates or lockups such that the investment may not be decreased; or - We simply don't want too much manager risk, so that if they imploded our losses would be limited to the amount invested (e.g., 10% of capital). These may be specified across all funds, for a sub-set of funds, or for each individual fund. In my specific case, I have an individual maximum for each type of manager (e.g., a sector specialist may be no more than 10% of the portfolio). I am not setting limits to the group today, but I may want to in the future (e.g., no more than 60% to the sector specialists). I usually have a minimum investment level below which I want to drop the investment (usually I want a position larger than 1-2%). I can't short in this portfolio (individual weight must be greater than 0%). 2. Set limits that are portfolio-specific I have different sets of constraints around the sum of the portfolio weights. In some cases, all my managers' weights must sum to (about) 100%. In this particular case, I can add leverage in such a way that the sum can be greater than 100%. There's always an upper limit to that leverage as well (e.g., no more than 200%). I also want to target a particular portfolio volatility (>10%), with the view that volatility is also an element of upside potential. In this case, competitive products have a volatility between 10-16%, and since investors use a heuristic of fees relative to the level of volatility I want to make sure the portfolio volatility is roughly in that range. So far, all of these issues represent constraints in the context that Brian already discussed, in that they: - operate only on the weight vector (or a calculation of its components); - may include metadata about the components (the type of manager); or - describe the inputs to the objective function (target portfolio volatility and/or return level). 3. Choose measures for constructing portfolio objectives >From there, I then have a variety of measures by which I want to evalute (rank) portfolios. An objective may be composed of several measures, which then have to be combined in a way that makes sense. That is, the measures may need to be normalized such that they relate to one another proportionally. I may want to vary my objectives by the amount of estimation I'm required to do: - if I don't want to do any estimation, I can think about a 1/N portfolio. That's also a good benchmark to determine whether I can add value in portfolio construction; OR - if I believe my volatility estimate, then I want a 1/sigma portfolio; OR - if I have confidence in both my volatility and correlation estimates, then I'm able to calculate: + minimum variance or minimum CVaR portfolio; + equal volatility or CVaR contribution portfolio; OR - If I have good estimates for higher moments, I can do the same with modified CVaR; OR - If I have an estimate for mean return I can examine the tradeoff between return and risk (e.g., MSRP). As we move down this list, additional estimates are required and we lose degrees of freedom. I'm going to ignore the issues of forecast horizon here. In this particular use case, I want to: - equalize the (ex-ante) contribution of volatility to the portfolio among it's constituent managers. - target a volatility level (as described above). 4. Examine other objectives But I also want to understand that portfolio in the context of the other possible objectives I could have, namely: - Equal contribution to: + Weight (1/N); + Variance without correlation (1/sigma); ! + Variance with correlation; <- my objective + Risk (mETL). - Reward to Risk: + Mean-Variance; + Mean-mETL. - Minimum: + Variance; + Risk (mETL); Even though I have a preference for one objective, I want to know about other objectives as well. Those might be considered allocation benchmarks, but they serve a role as markers (or buoys) in the constraint space rather than as 'benchmarks' for performance. Each of these is best expressed as an objective, of course, since they are each: - calculated on a weight vector; and - describe a portfolio objective for the outcome of the optimization (e.g., it describes the portfolio that is the minimum variance portfolio within the constraint space). The portfolios when compared this way can reveal some interesting aspects of the underlying decision - say, when comparing a 1/sigma portfolio to one calculated with correlation estimates one can see the impact of the trade-offs made for the correlation among the constituents. In my experience, constraints are typically very position-specific and must be related directly to the data of the instrument in question; objectives tend to be related to the portfolio in the aggregate. This brief description probably doesn't go far enough into the broader workflow. It's pretty obvious and it isn't as linear as I'm representing it, but I'll state it anyway: 1. organize data 2. describe the data statistically 3. develop methods for construction (broken down in the steps above) 4. get results 5. analyze results 6. interact with collaborators This broader context suggests a few more requirements for PortfolioAnalytics from the user's perspective. One is that the traceability of the solutions has to be maintained and well documented (What did I do? Where did the data come from? How were the results calculated? With what parameters?). We can end up with a great deal of derived data, and the user will probably need some helper functions to navigate through it all. Second, the results need to be well organized such that they can be easily compared, especially visually. Currently, this requires a great deal of manual manipulation to organize the data and make the objectives comparable. This also requires some thought when specifying objectives: calculations that need to be done for comparison that don't carry any weight in the objective itself (e.g., portfolio variance is calculated for a mean-CVaR objective, but has a multiplier of zero). Third, the sensitivity of the results should be addressable. This is probably a topic unto itself, because it can be manifested in several ways. Along one of these dimensions is the ability to examine portfolios that are not the optimal, but are "close-by," in that they rank below the optimal (e.g., the 10 nearest portfolios in RP). Food for thought, pcc -- Peter Carl http://www.braverock.com/peter From rossbennett34 at gmail.com Wed Jun 19 23:40:25 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Wed, 19 Jun 2013 16:40:25 -0500 Subject: [GSoC-PortA] Use case for PortfolioAnalytics In-Reply-To: References: Message-ID: Peter, Thank you very much for sending, that was very interesting and enlightening for me from a real use case perspective. No apology necessary regarding the length, I really appreciate your detailed thoughts on the points you discussed. I will use this as a reference as I work on this and may come back to you for additional clarification on certain points. I have a few questions regarding point 1. Set limits that are position-specific. Your text is italics and mine is inline below. *The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: - individual level (e.g., no single manager can represent more than 15% of AUM) or* This is equivalent to setting box constraints and is currently implemented for all solvers, correct? * - group level (e.g., no group of managers using Systematic strategies** can represent more than 30% of AUM).* Group constraints are only implemented in the ROI optimization method. Could we use the fnMap function for DEoptim or add group constraints to the randomize portfolio function? *There may be a specific direction of change: - Managers may not be able to accept additional capital, so that the position may only be reduced from current levels; - Managers may be receiving an allocation subject to a minimum amount of investment; - Positions may be subject to gates or lockups such that the investment may not be decreased; or* These seem similar to a turnover constraint. Similar to my question on group constraints, is it feasible to implement these constraints using fnMap with DEoptim? Thanks again! Ross On Wed, Jun 19, 2013 at 9:23 AM, Peter Carl wrote: > Brian suggested that I send a few thoughts on a use-case for > PortfolioAnalytics. I'm going to use a real situation, although I'm going > to skip over some of the details about *why* some of these issues exist. > Apologies in advance for the length - I couldn't restrain myself from > talking more broadly around the specifics I lay out here. > > The workflow I'm presenting here is similar to the one developed in > presentations that Brian has already pointed out: > - http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf > - http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf > Many of the things I say here are reflected in those presentations, but I > hope I can emphasize a few points here. In this use case, I'm more > concerned about describing the decisions than the workflow or the specific > data. > > Portfolio construction is still a frequently neglected part of the > investment process. It has, in practice, come a long way since the days > of Jessie Livermore-style overbetting (he lost 90% his wealth in 1907 on a > blown cotton trade). That said, investors still don't always understand > the specific trade-offs they are making in optimization, or the > sensitivity the techniques show to the data and estimation error around > the inputs. Many, many investors prefer to use simple rules-of-thumb and > common sense instead - which work to keep the portfolio out of hot water - > not realizing that they could add actually add value during this step by > explicitly expressing their objectives and constraints and generating > alternatives. > > In going through this exercise of constructing a portfolio from individual > elements, there are a few issues that are important to me as a > decision-maker. In general, I want: > - to be approximately correct rather than precisely wrong (accuracy of > the method should be considered in the context of estimation error); > - to define risk as potential loss rather than volatility (even > Markowitz wanted this); > - the flexibility to define any kind of objective (in fact, I want to > compare several); > - the ability to combine constraints (I have specific position-level > reasons for doing so); > - a framework for considering different sets of portfolio > constraints for comparison through time; and > - to build intuition about optimization through visualization. > > Many people would add that they are sensitive to the calculation time, as > well. Given the rebalancing frequency proposed in this case, calculation > time doesn't matter as much in this case. > > 0. Understand the portfolio constituents > > Often left unsaid, but this step affects judgments about what comes after. > In this case, I have three categories of commodities futures traders - > one of which tends to be extremely volatile but uncorrelated among the > peers (sector specialists). Another category shows high correlation among > themselves, but run relatively low volatility strategies (trend > followers). Those strategies can show autocorrelation, however, and can > find themselves in extended drawdowns. The third category of manager are > somewhere in between those two (macro). We tend to think of these in > groups, even where managers don't fit perfectly. > > 1. Set limits that are position-specific > > The first step is to specify individual position limits. In my more > specific case, where I'm allocating with actual managers (as opposed to > the EDHEC indexes in the 2012 presentation), those may come at the: > - individual level (e.g., no single manager can represent more than 15% > of AUM) or > - group level (e.g., no group of managers using Systematic strategies > can represent more than 30% of AUM). > There may be a specific direction of change: > - Managers may not be able to accept additional capital, so that the > position may only be reduced from current levels; > - Managers may be receiving an allocation subject to a minimum amount of > investment; > - Positions may be subject to gates or lockups such that the investment > may not be decreased; or > - We simply don't want too much manager risk, so that if they imploded > our losses would be limited to the amount invested (e.g., 10% of > capital). > > These may be specified across all funds, for a sub-set of funds, or for > each individual fund. In my specific case, I have an individual maximum > for each type of manager (e.g., a sector specialist may be no more than > 10% of the portfolio). I am not setting limits to the group today, but I > may want to in the future (e.g., no more than 60% to the sector > specialists). I usually have a minimum investment level below which I > want to drop the investment (usually I want a position larger than 1-2%). > I can't short in this portfolio (individual weight must be greater than > 0%). > > 2. Set limits that are portfolio-specific > > I have different sets of constraints around the sum of the portfolio > weights. In some cases, all my managers' weights must sum to (about) > 100%. In this particular case, I can add leverage in such a way that the > sum can be greater than 100%. There's always an upper limit to that > leverage as well (e.g., no more than 200%). > > I also want to target a particular portfolio volatility (>10%), with the > view that volatility is also an element of upside potential. In this > case, competitive products have a volatility between 10-16%, and since > investors use a heuristic of fees relative to the level of volatility I > want to make sure the portfolio volatility is roughly in that range. > > > So far, all of these issues represent constraints in the context that > Brian already discussed, in that they: > - operate only on the weight vector (or a calculation of its components); > - may include metadata about the components (the type of manager); or > - describe the inputs to the objective function (target portfolio > volatility and/or return level). > > 3. Choose measures for constructing portfolio objectives > > From there, I then have a variety of measures by which I want to evalute > (rank) portfolios. An objective may be composed of several measures, > which then have to be combined in a way that makes sense. That is, the > measures may need to be normalized such that they relate to one another > proportionally. > > I may want to vary my objectives by the amount of estimation I'm required > to do: > > - if I don't want to do any estimation, I can think about a 1/N > portfolio. That's also a good benchmark to determine whether I can add > value in portfolio construction; OR > > - if I believe my volatility estimate, then I want a 1/sigma portfolio; > OR > > - if I have confidence in both my volatility and correlation estimates, > then I'm able to calculate: > + minimum variance or minimum CVaR portfolio; > + equal volatility or CVaR contribution portfolio; OR > > - If I have good estimates for higher moments, I can do the same with > modified CVaR; OR > > - If I have an estimate for mean return I can examine the tradeoff > between return and risk (e.g., MSRP). > > As we move down this list, additional estimates are required and we lose > degrees of freedom. I'm going to ignore the issues of forecast horizon > here. > > In this particular use case, I want to: > - equalize the (ex-ante) contribution of volatility to the portfolio > among it's constituent managers. > - target a volatility level (as described above). > > 4. Examine other objectives > > But I also want to understand that portfolio in the context of the other > possible objectives I could have, namely: > > - Equal contribution to: > + Weight (1/N); > + Variance without correlation (1/sigma); > ! + Variance with correlation; <- my objective > + Risk (mETL). > - Reward to Risk: > + Mean-Variance; > + Mean-mETL. > - Minimum: > + Variance; > + Risk (mETL); > > Even though I have a preference for one objective, I want to know about > other objectives as well. Those might be considered allocation > benchmarks, but they serve a role as markers (or buoys) in the constraint > space rather than as 'benchmarks' for performance. > > Each of these is best expressed as an objective, of course, since they are > each: > - calculated on a weight vector; and > - describe a portfolio objective for the outcome of the optimization > (e.g., it describes the portfolio that is the minimum variance portfolio > within the constraint space). > > The portfolios when compared this way can reveal some interesting aspects > of the underlying decision - say, when comparing a 1/sigma portfolio to > one calculated with correlation estimates one can see the impact of the > trade-offs made for the correlation among the constituents. > > In my experience, constraints are typically very position-specific and > must be related directly to the data of the instrument in question; > objectives tend to be related to the portfolio in the aggregate. > > > This brief description probably doesn't go far enough into the broader > workflow. It's pretty obvious and it isn't as linear as I'm representing > it, but I'll state it anyway: > > 1. organize data > 2. describe the data statistically > 3. develop methods for construction (broken down in the steps above) > 4. get results > 5. analyze results > 6. interact with collaborators > > This broader context suggests a few more requirements for > PortfolioAnalytics from the user's perspective. One is that the > traceability of the solutions has to be maintained and well documented > (What did I do? Where did the data come from? How were the results > calculated? With what parameters?). We can end up with a great deal of > derived data, and the user will probably need some helper functions to > navigate through it all. > > Second, the results need to be well organized such that they can be easily > compared, especially visually. Currently, this requires a great deal of > manual manipulation to organize the data and make the objectives > comparable. This also requires some thought when specifying objectives: > calculations that need to be done for comparison that don't carry any > weight in the objective itself (e.g., portfolio variance is calculated for > a mean-CVaR objective, but has a multiplier of zero). > > Third, the sensitivity of the results should be addressable. This is > probably a topic unto itself, because it can be manifested in several > ways. Along one of these dimensions is the ability to examine portfolios > that are not the optimal, but are "close-by," in that they rank below the > optimal (e.g., the 10 nearest portfolios in RP). > > Food for thought, > > pcc > -- > Peter Carl > http://www.braverock.com/peter > > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at braverock.com Thu Jun 20 03:32:12 2013 From: peter at braverock.com (Peter Carl) Date: Wed, 19 Jun 2013 20:32:12 -0500 Subject: [GSoC-PortA] Use case for PortfolioAnalytics Message-ID: <48eeeb76bc77947bcadc9d9805336ab6.squirrel@mail.braverock.com> Whoops - resending to the list as well. pcc -- Peter Carl http://www.braverock.com/peter > This is equivalent to setting box constraints and is currently implemented for all solvers, correct? Yes. > Could we use the fnMap function for DEoptim or add group constraints to the randomize portfolio function? I'm not familiar with the fnMap function - maybe Brian has a take there. Within RP, group constraints should be relatively easy to add. > These seem similar to a turnover constraint. Similar to my question on group constraints, is it feasible to implement these constraints using fnMap with DEoptim? Not really - at least, not as I think about turnover. In my mind, a turnover constraint applies at the aggregate portfolio level - e.g., no more than 25% total turnover during a period. These are individual position limits that can be described in inequalities: <= current position, = current position, or >= minimum investment position not described by policy (imposed by manager). HTH, pcc -- Peter Carl http://www.braverock.com/peter > Peter, > > Thank you very much for sending, that was very interesting and > enlightening > for me from a real use case perspective. No apology necessary regarding the > length, I really appreciate your detailed thoughts on the points you discussed. I will use this as a reference as I work on this and may come back to you for additional clarification on certain points. > > I have a few questions regarding point 1. Set limits that are > position-specific. Your text is italics and mine is inline below. > > *The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: > - individual level (e.g., no single manager can represent more than 15% > of AUM) or* > This is equivalent to setting box constraints and is currently implemented for all solvers, correct? > > * - group level (e.g., no group of managers using Systematic strategies** > can represent more than 30% of AUM).* > Group constraints are only implemented in the ROI optimization method. Could we use the fnMap function for DEoptim or add group constraints to the > randomize portfolio function? > > *There may be a specific direction of change: > - Managers may not be able to accept additional capital, so that the > position may only be reduced from current levels; > - Managers may be receiving an allocation subject to a minimum amount of > investment; > - Positions may be subject to gates or lockups such that the investment > may not be decreased; or* > These seem similar to a turnover constraint. Similar to my question on group constraints, is it feasible to implement these constraints using fnMap with DEoptim? > > > Thanks again! > Ross > > > On Wed, Jun 19, 2013 at 9:23 AM, Peter Carl wrote: > >> Brian suggested that I send a few thoughts on a use-case for >> PortfolioAnalytics. I'm going to use a real situation, although I'm going >> to skip over some of the details about *why* some of these issues exist. Apologies in advance for the length - I couldn't restrain myself from talking more broadly around the specifics I lay out here. >> >> The workflow I'm presenting here is similar to the one developed in presentations that Brian has already pointed out: >> - http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf - http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf Many of the things I say here are reflected in those presentations, but I >> hope I can emphasize a few points here. In this use case, I'm more concerned about describing the decisions than the workflow or the specific >> data. >> >> Portfolio construction is still a frequently neglected part of the investment process. It has, in practice, come a long way since the days of Jessie Livermore-style overbetting (he lost 90% his wealth in 1907 on a >> blown cotton trade). That said, investors still don't always understand the specific trade-offs they are making in optimization, or the sensitivity the techniques show to the data and estimation error around the inputs. Many, many investors prefer to use simple rules-of-thumb and >> common sense instead - which work to keep the portfolio out of hot water - >> not realizing that they could add actually add value during this step by explicitly expressing their objectives and constraints and generating alternatives. >> >> In going through this exercise of constructing a portfolio from individual >> elements, there are a few issues that are important to me as a >> decision-maker. In general, I want: >> - to be approximately correct rather than precisely wrong (accuracy of >> the method should be considered in the context of estimation error); >> - to define risk as potential loss rather than volatility (even >> Markowitz wanted this); >> - the flexibility to define any kind of objective (in fact, I want to >> compare several); >> - the ability to combine constraints (I have specific position-level >> reasons for doing so); >> - a framework for considering different sets of portfolio >> constraints for comparison through time; and >> - to build intuition about optimization through visualization. >> >> Many people would add that they are sensitive to the calculation time, as >> well. Given the rebalancing frequency proposed in this case, >> calculation >> time doesn't matter as much in this case. >> >> 0. Understand the portfolio constituents >> >> Often left unsaid, but this step affects judgments about what comes after. >> In this case, I have three categories of commodities futures traders - >> one of which tends to be extremely volatile but uncorrelated among the peers (sector specialists). Another category shows high correlation among >> themselves, but run relatively low volatility strategies (trend followers). Those strategies can show autocorrelation, however, and can find themselves in extended drawdowns. The third category of manager are >> somewhere in between those two (macro). We tend to think of these in groups, even where managers don't fit perfectly. >> >> 1. Set limits that are position-specific >> >> The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: >> - individual level (e.g., no single manager can represent more than >> 15% >> of AUM) or >> - group level (e.g., no group of managers using Systematic strategies >> can represent more than 30% of AUM). >> There may be a specific direction of change: >> - Managers may not be able to accept additional capital, so that the >> position may only be reduced from current levels; >> - Managers may be receiving an allocation subject to a minimum amount >> of >> investment; >> - Positions may be subject to gates or lockups such that the >> investment >> may not be decreased; or >> - We simply don't want too much manager risk, so that if they imploded >> our losses would be limited to the amount invested (e.g., 10% of capital). >> >> These may be specified across all funds, for a sub-set of funds, or for each individual fund. In my specific case, I have an individual maximum for each type of manager (e.g., a sector specialist may be no more than 10% of the portfolio). I am not setting limits to the group today, but I >> may want to in the future (e.g., no more than 60% to the sector specialists). I usually have a minimum investment level below which I want to drop the investment (usually I want a position larger than 1-2%). >> I can't short in this portfolio (individual weight must be greater than 0%). >> >> 2. Set limits that are portfolio-specific >> >> I have different sets of constraints around the sum of the portfolio weights. In some cases, all my managers' weights must sum to (about) 100%. In this particular case, I can add leverage in such a way that the >> sum can be greater than 100%. There's always an upper limit to that leverage as well (e.g., no more than 200%). >> >> I also want to target a particular portfolio volatility (>10%), with the view that volatility is also an element of upside potential. In this case, competitive products have a volatility between 10-16%, and since investors use a heuristic of fees relative to the level of volatility I want to make sure the portfolio volatility is roughly in that range. >> >> >> So far, all of these issues represent constraints in the context that Brian already discussed, in that they: >> - operate only on the weight vector (or a calculation of its >> components); >> - may include metadata about the components (the type of manager); or - describe the inputs to the objective function (target portfolio >> volatility and/or return level). >> >> 3. Choose measures for constructing portfolio objectives >> >> From there, I then have a variety of measures by which I want to evalute (rank) portfolios. An objective may be composed of several measures, which then have to be combined in a way that makes sense. That is, the measures may need to be normalized such that they relate to one another proportionally. >> >> I may want to vary my objectives by the amount of estimation I'm required >> to do: >> >> - if I don't want to do any estimation, I can think about a 1/N >> portfolio. That's also a good benchmark to determine whether I can add value in portfolio construction; OR >> >> - if I believe my volatility estimate, then I want a 1/sigma >> portfolio; >> OR >> >> - if I have confidence in both my volatility and correlation >> estimates, >> then I'm able to calculate: >> + minimum variance or minimum CVaR portfolio; >> + equal volatility or CVaR contribution portfolio; OR >> >> - If I have good estimates for higher moments, I can do the same with >> modified CVaR; OR >> >> - If I have an estimate for mean return I can examine the tradeoff >> between return and risk (e.g., MSRP). >> >> As we move down this list, additional estimates are required and we lose degrees of freedom. I'm going to ignore the issues of forecast horizon here. >> >> In this particular use case, I want to: >> - equalize the (ex-ante) contribution of volatility to the portfolio >> among it's constituent managers. >> - target a volatility level (as described above). >> >> 4. Examine other objectives >> >> But I also want to understand that portfolio in the context of the other possible objectives I could have, namely: >> >> - Equal contribution to: >> + Weight (1/N); >> + Variance without correlation (1/sigma); >> ! + Variance with correlation; <- my objective >> + Risk (mETL). >> - Reward to Risk: >> + Mean-Variance; >> + Mean-mETL. >> - Minimum: >> + Variance; >> + Risk (mETL); >> >> Even though I have a preference for one objective, I want to know about other objectives as well. Those might be considered allocation benchmarks, but they serve a role as markers (or buoys) in the >> constraint >> space rather than as 'benchmarks' for performance. >> >> Each of these is best expressed as an objective, of course, since they are >> each: >> - calculated on a weight vector; and >> - describe a portfolio objective for the outcome of the optimization >> (e.g., it describes the portfolio that is the minimum variance portfolio within the constraint space). >> >> The portfolios when compared this way can reveal some interesting aspects >> of the underlying decision - say, when comparing a 1/sigma portfolio to one calculated with correlation estimates one can see the impact of the trade-offs made for the correlation among the constituents. >> >> In my experience, constraints are typically very position-specific and must be related directly to the data of the instrument in question; objectives tend to be related to the portfolio in the aggregate. >> >> >> This brief description probably doesn't go far enough into the broader workflow. It's pretty obvious and it isn't as linear as I'm >> representing >> it, but I'll state it anyway: >> >> 1. organize data >> 2. describe the data statistically >> 3. develop methods for construction (broken down in the steps above) 4. get results >> 5. analyze results >> 6. interact with collaborators >> >> This broader context suggests a few more requirements for >> PortfolioAnalytics from the user's perspective. One is that the traceability of the solutions has to be maintained and well documented (What did I do? Where did the data come from? How were the results calculated? With what parameters?). We can end up with a great deal of derived data, and the user will probably need some helper functions to navigate through it all. >> >> Second, the results need to be well organized such that they can be easily >> compared, especially visually. Currently, this requires a great deal of manual manipulation to organize the data and make the objectives comparable. This also requires some thought when specifying objectives: calculations that need to be done for comparison that don't carry any weight in the objective itself (e.g., portfolio variance is calculated for >> a mean-CVaR objective, but has a multiplier of zero). >> >> Third, the sensitivity of the results should be addressable. This is probably a topic unto itself, because it can be manifested in several ways. Along one of these dimensions is the ability to examine >> portfolios >> that are not the optimal, but are "close-by," in that they rank below the >> optimal (e.g., the 10 nearest portfolios in RP). >> >> Food for thought, >> >> pcc >> -- >> Peter Carl >> http://www.braverock.com/peter >> >> >> _______________________________________________ >> GSoC-PortA mailing list >> GSoC-PortA at lists.r-forge.r-project.org >> http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta >> > From martinrd at comcast.net Thu Jun 20 05:41:07 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:41:07 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Message-ID: <018501ce6d68$0c0d7d00$24287700$@comcast.net> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 12:36 PM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects See comments inline below. Regards, Ross Bennett On Mon, Jun 17, 2013 at 1:17 PM, Doug Martin wrote: Ross and Brian, Ross, have you or Brian had a chance to take a careful look at the code (attached) that Kirk Li developed for my class this spring to see if the ideas there may be useful? I have gone through the code from Kirk Li, but will take closer look to make sure I did not miss anything. I note that add.constraint needs to check for compatibility with the existing constraint object, and give a sensible message if not. See for example the segment "# sum constraint are not combinable with propcost constraint" just below Scenario 10 in the attached constraint.sets.test.r. The way Kirk's code checks for compatibility constraints should be helpful and hopefully I can borrow some of the ideas. At the moment I have forgotten what the constraint set conventions are in PortfolioAnalytics, and will look at it again on Wed. But I think we need the something like the types that are in constraint.sets.r attached. Ross, since you have seen these before in computing homework problems, so you may already have some thoughts about this? The way that constraints.sets.R handles the constraints is similar to PortfolioAnalytics in that the different constraints are stored in a list object. The main differences I see is how the constraints are combined and passed to the solver. The different functions for the constraints such as cset.box() and cset.lo() do a lot of work setting up the A matrix and b vector for solve.QP(). [Doug] Indeed. In PortfolioAnalytics, this is handled inside of optimize.portfolio() and handled differently based on the chosen optimizer. [Doug] I need to look at optimize.portfolio again In Kirk Li's code, all of the constraints are fed into gmv(), which if I recall correctly is just a wrapper around solve.QP(). So this means the constraints are pretty specific to one solver. The way the constraints are specified in PortfolioAnalytics currently, the same constraint object can be passed to the DEoptim, ROI (Rglpk and quadprog), Random Portfolios, pso, and GenSA solvers. [Doug] In the next few days I'll look at the way it is done in PA. However, I seem to recall that neither general linear inequality constraints or group constraints were supported. Am I recalling correctly? If so we need to agree on a convention for doing so. For the case of general linear inequality constraints I like the convention of specifying equalities and inequalities in Rglpk_solve_LP. Let's get this convention agreed upon and then move on to the groups constraints. Also, I seem to recall that at the end of last summer there was still a problem in calling solve.QP and Rglpk_solve_LP, and it involved having to use ROI and create a mapping from the PortfolioAnalytics to the solve.QP and Rglpk_solve_LP conventions. Brian do you recall the situation as of the end of last summer (I think you had thought a bit about fixing this small problem)? Doug FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use solve.QP and Rglpk_solve_LP for a little bit in the first computing chapter, but then quickly move to PortfolioAnalytics for everything we do that will be supported by PortfolioAnlytics. The goal is that MPO2 will equally well support instructional pedagogy and real-world applications use. Very cool that you plan on using PortfolioAnalytics in the 2nd edition of your book! Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 10:27 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() * Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() * Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() * Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() * Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() * Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() * Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() * Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Thu Jun 20 05:41:07 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:41:07 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <000601ce6b6b$18205d50$486117f0$@comcast.net> <008001ce6b86$fc3c4100$f4b4c300$@comcast.net> Message-ID: <018501ce6d68$0c0d7d00$24287700$@comcast.net> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 12:36 PM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects See comments inline below. Regards, Ross Bennett On Mon, Jun 17, 2013 at 1:17 PM, Doug Martin wrote: Ross and Brian, Ross, have you or Brian had a chance to take a careful look at the code (attached) that Kirk Li developed for my class this spring to see if the ideas there may be useful? I have gone through the code from Kirk Li, but will take closer look to make sure I did not miss anything. I note that add.constraint needs to check for compatibility with the existing constraint object, and give a sensible message if not. See for example the segment "# sum constraint are not combinable with propcost constraint" just below Scenario 10 in the attached constraint.sets.test.r. The way Kirk's code checks for compatibility constraints should be helpful and hopefully I can borrow some of the ideas. At the moment I have forgotten what the constraint set conventions are in PortfolioAnalytics, and will look at it again on Wed. But I think we need the something like the types that are in constraint.sets.r attached. Ross, since you have seen these before in computing homework problems, so you may already have some thoughts about this? The way that constraints.sets.R handles the constraints is similar to PortfolioAnalytics in that the different constraints are stored in a list object. The main differences I see is how the constraints are combined and passed to the solver. The different functions for the constraints such as cset.box() and cset.lo() do a lot of work setting up the A matrix and b vector for solve.QP(). [Doug] Indeed. In PortfolioAnalytics, this is handled inside of optimize.portfolio() and handled differently based on the chosen optimizer. [Doug] I need to look at optimize.portfolio again In Kirk Li's code, all of the constraints are fed into gmv(), which if I recall correctly is just a wrapper around solve.QP(). So this means the constraints are pretty specific to one solver. The way the constraints are specified in PortfolioAnalytics currently, the same constraint object can be passed to the DEoptim, ROI (Rglpk and quadprog), Random Portfolios, pso, and GenSA solvers. [Doug] In the next few days I'll look at the way it is done in PA. However, I seem to recall that neither general linear inequality constraints or group constraints were supported. Am I recalling correctly? If so we need to agree on a convention for doing so. For the case of general linear inequality constraints I like the convention of specifying equalities and inequalities in Rglpk_solve_LP. Let's get this convention agreed upon and then move on to the groups constraints. Also, I seem to recall that at the end of last summer there was still a problem in calling solve.QP and Rglpk_solve_LP, and it involved having to use ROI and create a mapping from the PortfolioAnalytics to the solve.QP and Rglpk_solve_LP conventions. Brian do you recall the situation as of the end of last summer (I think you had thought a bit about fixing this small problem)? Doug FYI, in the 2nd edition MPO2 of the MPO book I intend to briefly use solve.QP and Rglpk_solve_LP for a little bit in the first computing chapter, but then quickly move to PortfolioAnalytics for everything we do that will be supported by PortfolioAnlytics. The goal is that MPO2 will equally well support instructional pedagogy and real-world applications use. Very cool that you plan on using PortfolioAnalytics in the 2nd edition of your book! Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 10:27 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects All, With finals being over (for me at least, I do realize that Prof. Yollin and Prof. Martin still have grading) I am anxious to get started today with the first day of coding. When I am writing or editing code, should I be working directly in the PortfolioAnalytics/R files or should I work in the sandbox folder so that I have a safe area to "play" without worrying about messing up existing code? With the recent discussions of separating the objectives from the constraints and adding a portfoliospec object, here is a list of functions/features I am proposing to add to accomplish this. constraint_v2() * Constraints object constructor as an S3 object that will hold the constraints. Similar to the current constraints_v1 object, but will keep separate from objectives. add.constraint() * Helper function to add constraints to the constraint_v2 object such as box, group, etc. constraints. box_constraint() * Function to be called by add.constraint that will set minimum and maximum weight for box constraints group_constraint() * Function to be called by add.constraint that will set minimum and maximum group weights for group constraints objective_v2() * Objective object constructor as an S3 object that will hold the objectives. Similar to the current constraints_v1 object, but will be separate from constraints. add.objective() * Helper function to add objectives to the objective object. Note that there is currently an objective constructor of class=objclass and add.objective() function. These currently update the constraints_v1 object with the objectives. I should be able to re-use most of these and make revisions so that they update the objectives object. portfoliospec() * Portfolio specification constructor that holds the constraints and objectives. A constraint_v2 object and objective_v2 object will be passed in to create the portfoliospec object. may need additional helper functions for portfoliospec that do checks on the portfoliospec and/or the constraints and objectives objects that are passed in. optimize.portfolio() will have to be modified so that it takes in a portfoliospec object instead of a constraints object. Thoughts? Ross Bennett On Mon, Jun 17, 2013 at 9:57 AM, Doug Martin wrote: Brian and Ross, I couldn't agree more with the proposal to separate constraints and objectives (Brian, as you and I have discussed earlier this year or last year). I will have further comments on checking validity of adding constraints late tomorrow or early Wed. after I have completed grading of my spring quarter course. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 17, 2013 7:09 AM To: PortfolioAnalytics Cc: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Brian, I really like your thoughts on separating constraints and objectives. I learned objectives and constraints in the same way that you mention them. Intuitively, it is easier to understand that the user would create the constraint object, the objective object, and then combine in a portfolio object. Separating out the objectives and creating templates of objectives for minimum variance and other basic objective functions should make it easy to understand for the user as well how it is handled with the solvers. The way the constraints object looks today seems to be very similar to what you have in mind for the portfoliospec object. Was that your intent so that the solvers see pretty much the same object in the call to optimize.portfolio? Only now the portfoliospec object will be passed instead of the constraints object, is my understanding of your intentions correct? This is what I am thinking the code will look *something* like to run a simple optimization problem. #### Example R code # Specify a constraints object and add box constraints gen.constr = constraint(assets, min_sum=1, max_sum=1) gen.constr = add.constraint(gen.constr, min_weight=0.10, max_weight=0.4, type="box") # Specify an objective object for global minimum variance gen.obj = objective(type="GMV") # Specify a portfolio object port.spec = portfoliospec(gen.constr, gen.obj) # Run the optimization solution = optimize.portfolio(data, port.spec) ##### Thanks, Ross Bennett On Mon, Jun 17, 2013 at 6:51 AM, Brian G. Peterson wrote: I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: Professor Martin, Thank you for the feedback. Based on the suggestions from yourself and Brian Peterson, I will start with the constraints portion. I'll spend some more time looking at Kirk Li's code to understand it better and think about how to best integrate into PortfolioAnalytics. I should be able to use some ideas from it. I have been reviewing the code for PortfolioAnalytics and it seems that there is already a lot of the infrastructure in place for the constraint object and the optimizer so I don't want to maintain the interface to the optimize.portfolio function as much as possible. As an example, here is a code snippet of how box constraints and group constraints are currently specified. # General constraints object # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max =0.5, min_sum=1, max_sum=1) # Add group constraints to the gen.constr object groups <- c(3,3,3,4) gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) gen.constr$cUP <- rep(0.30,length(groups)) Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, we would remove box constraints from the constraint() function and create an add.constraint() function to add/update box, group, and other constraint types as applicable. # Create the initial constraint object # Pass in a character vector of asset names for assets # Portfolio weights must sum to 1 (min_sum=1 and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), min_sum=1, max_sum=1) # Add box constraints with asset weights greater than 0.1 and less than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", min=0.1, max=0.5) # Group constraints for 10 assets # Separate assets into groups of 3, 3, and 4 assets # Specify the group weights as a vector # The first 3 assets shall have a group weight of greater than 0.15 and less than 0.3 # The next 3 assets shall have a group weight of greater than 0.1 and less than 0.35 # The last 4 assets shall have a group weight greater than 0 and less than 0.4 constr <- add.constraint(constraints=constr, type="group", groups=c(3, 3, 4), group_min=c(0.15, 0.1, 0), group_max=c(0.3, 0.35, 0.4)) This keeps the constraint object and interface to optimize.portfolio() consistent with the existing code, but simplifies it and makes adding box and group constraints very similar. I will continue to review the PortfolioAnalytics source code and we can revisit this so that I can hit the ground running when finals are over and start as soon as possible. Regards, Ross Bennett On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin > wrote: Ross and all,____ __ __ I think starting with the constraints is a good idea. But I think a review of where PortfolioAnalytics functionality is as of last year's GSoC should be the first priority. How planning on that week after next (Guy, Eric and I all have finals week to get out of the way next week).____ __ __ With regard to constraints, that approach that Kirk Li (UW Statistics Ph.D. student and AMATH 543 TA this spring) developed looks useful. Ross you already have the code which I posted to the class web site. I have attached it here for others on this project to have a look at. Let's review that after the end of next week when final exams and the spring quarter are completely behind us.____ __ __ Doug____ __ __ __ __ *From:*gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org ] *On Behalf Of *Ross Bennett *Sent:* Wednesday, June 05, 2013 4:22 AM *To:* gsoc-porta at lists.r-forge.r-project.org *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ __ __ All,____ __ __ In my GSOC proposal, the order of what components I planned to write code for was as follows:____ 1. Utility Functions (print, summary, plot, etc.)____ 2. Constraints____ 3. Example Functionality____ In speaking with Brian Peterson at R/Finance, he suggested that the constraints portion be done first. I am fine with doing constraints first. Would you like me to submit a revised schedule and implementation plan with constraints first?____ __ __ Thanks,____ Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Thu Jun 20 05:47:56 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:47:56 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Message-ID: <018a01ce6d69$0011e710$0035b530$@comcast.net> -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Kris Boudt Sent: Tuesday, June 18, 2013 1:35 AM To: PortfolioAnalytics; gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: [Doug] It should be noted that turnover can be handled through either through constraints or a penalty in the objective (see the attached slides). Last summer I was not very happy with the objective penalty behavior but need to revisit it. - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. [Doug] We definitely need both of the above options, the second obviously for back-testing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. [Doug] This is a good idea. [Doug] Thanks. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [ mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin < mailto:martinrd at comcast.net>> wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org> > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > < mailto:gsoc-porta at lists.r-forge.r-project.org> > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Thu Jun 20 05:47:56 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:47:56 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Message-ID: <018a01ce6d69$0011e710$0035b530$@comcast.net> -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Kris Boudt Sent: Tuesday, June 18, 2013 1:35 AM To: PortfolioAnalytics; gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: [Doug] It should be noted that turnover can be handled through either through constraints or a penalty in the objective (see the attached slides). Last summer I was not very happy with the objective penalty behavior but need to revisit it. - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. [Doug] We definitely need both of the above options, the second obviously for back-testing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. [Doug] This is a good idea. [Doug] Thanks. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [ mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin < mailto:martinrd at comcast.net>> wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org> > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > < mailto:gsoc-porta at lists.r-forge.r-project.org> > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Thu Jun 20 05:50:18 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:50:18 -0700 Subject: [GSoC-PortA] Use case for PortfolioAnalytics In-Reply-To: References: Message-ID: <019501ce6d69$57fd1300$07f73900$@comcast.net> Peter, Great set of comments and suggestions. At the moment I would just say that our challenge is to figure out which of all the possibilities to prioritize and implement sooner rather than later. I will come back later with some thoughts about this. Thanks, Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Peter Carl Sent: Wednesday, June 19, 2013 7:23 AM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] Use case for PortfolioAnalytics Brian suggested that I send a few thoughts on a use-case for PortfolioAnalytics. I'm going to use a real situation, although I'm going to skip over some of the details about *why* some of these issues exist. Apologies in advance for the length - I couldn't restrain myself from talking more broadly around the specifics I lay out here. The workflow I'm presenting here is similar to the one developed in presentations that Brian has already pointed out: - http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf - http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf Many of the things I say here are reflected in those presentations, but I hope I can emphasize a few points here. In this use case, I'm more concerned about describing the decisions than the workflow or the specific data. Portfolio construction is still a frequently neglected part of the investment process. It has, in practice, come a long way since the days of Jessie Livermore-style overbetting (he lost 90% his wealth in 1907 on a blown cotton trade). That said, investors still don't always understand the specific trade-offs they are making in optimization, or the sensitivity the techniques show to the data and estimation error around the inputs. Many, many investors prefer to use simple rules-of-thumb and common sense instead - which work to keep the portfolio out of hot water - not realizing that they could add actually add value during this step by explicitly expressing their objectives and constraints and generating alternatives. In going through this exercise of constructing a portfolio from individual elements, there are a few issues that are important to me as a decision-maker. In general, I want: - to be approximately correct rather than precisely wrong (accuracy of the method should be considered in the context of estimation error); - to define risk as potential loss rather than volatility (even Markowitz wanted this); - the flexibility to define any kind of objective (in fact, I want to compare several); - the ability to combine constraints (I have specific position-level reasons for doing so); - a framework for considering different sets of portfolio constraints for comparison through time; and - to build intuition about optimization through visualization. Many people would add that they are sensitive to the calculation time, as well. Given the rebalancing frequency proposed in this case, calculation time doesn't matter as much in this case. 0. Understand the portfolio constituents Often left unsaid, but this step affects judgments about what comes after. In this case, I have three categories of commodities futures traders - one of which tends to be extremely volatile but uncorrelated among the peers (sector specialists). Another category shows high correlation among themselves, but run relatively low volatility strategies (trend followers). Those strategies can show autocorrelation, however, and can find themselves in extended drawdowns. The third category of manager are somewhere in between those two (macro). We tend to think of these in groups, even where managers don't fit perfectly. 1. Set limits that are position-specific The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: - individual level (e.g., no single manager can represent more than 15% of AUM) or - group level (e.g., no group of managers using Systematic strategies can represent more than 30% of AUM). There may be a specific direction of change: - Managers may not be able to accept additional capital, so that the position may only be reduced from current levels; - Managers may be receiving an allocation subject to a minimum amount of investment; - Positions may be subject to gates or lockups such that the investment may not be decreased; or - We simply don't want too much manager risk, so that if they imploded our losses would be limited to the amount invested (e.g., 10% of capital). These may be specified across all funds, for a sub-set of funds, or for each individual fund. In my specific case, I have an individual maximum for each type of manager (e.g., a sector specialist may be no more than 10% of the portfolio). I am not setting limits to the group today, but I may want to in the future (e.g., no more than 60% to the sector specialists). I usually have a minimum investment level below which I want to drop the investment (usually I want a position larger than 1-2%). I can't short in this portfolio (individual weight must be greater than 0%). 2. Set limits that are portfolio-specific I have different sets of constraints around the sum of the portfolio weights. In some cases, all my managers' weights must sum to (about) 100%. In this particular case, I can add leverage in such a way that the sum can be greater than 100%. There's always an upper limit to that leverage as well (e.g., no more than 200%). I also want to target a particular portfolio volatility (>10%), with the view that volatility is also an element of upside potential. In this case, competitive products have a volatility between 10-16%, and since investors use a heuristic of fees relative to the level of volatility I want to make sure the portfolio volatility is roughly in that range. So far, all of these issues represent constraints in the context that Brian already discussed, in that they: - operate only on the weight vector (or a calculation of its components); - may include metadata about the components (the type of manager); or - describe the inputs to the objective function (target portfolio volatility and/or return level). 3. Choose measures for constructing portfolio objectives >From there, I then have a variety of measures by which I want to evalute (rank) portfolios. An objective may be composed of several measures, which then have to be combined in a way that makes sense. That is, the measures may need to be normalized such that they relate to one another proportionally. I may want to vary my objectives by the amount of estimation I'm required to do: - if I don't want to do any estimation, I can think about a 1/N portfolio. That's also a good benchmark to determine whether I can add value in portfolio construction; OR - if I believe my volatility estimate, then I want a 1/sigma portfolio; OR - if I have confidence in both my volatility and correlation estimates, then I'm able to calculate: + minimum variance or minimum CVaR portfolio; + equal volatility or CVaR contribution portfolio; OR - If I have good estimates for higher moments, I can do the same with modified CVaR; OR - If I have an estimate for mean return I can examine the tradeoff between return and risk (e.g., MSRP). As we move down this list, additional estimates are required and we lose degrees of freedom. I'm going to ignore the issues of forecast horizon here. In this particular use case, I want to: - equalize the (ex-ante) contribution of volatility to the portfolio among it's constituent managers. - target a volatility level (as described above). 4. Examine other objectives But I also want to understand that portfolio in the context of the other possible objectives I could have, namely: - Equal contribution to: + Weight (1/N); + Variance without correlation (1/sigma); ! + Variance with correlation; <- my objective + Risk (mETL). - Reward to Risk: + Mean-Variance; + Mean-mETL. - Minimum: + Variance; + Risk (mETL); Even though I have a preference for one objective, I want to know about other objectives as well. Those might be considered allocation benchmarks, but they serve a role as markers (or buoys) in the constraint space rather than as 'benchmarks' for performance. Each of these is best expressed as an objective, of course, since they are each: - calculated on a weight vector; and - describe a portfolio objective for the outcome of the optimization (e.g., it describes the portfolio that is the minimum variance portfolio within the constraint space). The portfolios when compared this way can reveal some interesting aspects of the underlying decision - say, when comparing a 1/sigma portfolio to one calculated with correlation estimates one can see the impact of the trade-offs made for the correlation among the constituents. In my experience, constraints are typically very position-specific and must be related directly to the data of the instrument in question; objectives tend to be related to the portfolio in the aggregate. This brief description probably doesn't go far enough into the broader workflow. It's pretty obvious and it isn't as linear as I'm representing it, but I'll state it anyway: 1. organize data 2. describe the data statistically 3. develop methods for construction (broken down in the steps above) 4. get results 5. analyze results 6. interact with collaborators This broader context suggests a few more requirements for PortfolioAnalytics from the user's perspective. One is that the traceability of the solutions has to be maintained and well documented (What did I do? Where did the data come from? How were the results calculated? With what parameters?). We can end up with a great deal of derived data, and the user will probably need some helper functions to navigate through it all. Second, the results need to be well organized such that they can be easily compared, especially visually. Currently, this requires a great deal of manual manipulation to organize the data and make the objectives comparable. This also requires some thought when specifying objectives: calculations that need to be done for comparison that don't carry any weight in the objective itself (e.g., portfolio variance is calculated for a mean-CVaR objective, but has a multiplier of zero). Third, the sensitivity of the results should be addressable. This is probably a topic unto itself, because it can be manifested in several ways. Along one of these dimensions is the ability to examine portfolios that are not the optimal, but are "close-by," in that they rank below the optimal (e.g., the 10 nearest portfolios in RP). Food for thought, pcc -- Peter Carl http://www.braverock.com/peter _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From martinrd at comcast.net Thu Jun 20 05:50:18 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 20:50:18 -0700 Subject: [GSoC-PortA] Use case for PortfolioAnalytics In-Reply-To: References: Message-ID: <019501ce6d69$57fd1300$07f73900$@comcast.net> Peter, Great set of comments and suggestions. At the moment I would just say that our challenge is to figure out which of all the possibilities to prioritize and implement sooner rather than later. I will come back later with some thoughts about this. Thanks, Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Peter Carl Sent: Wednesday, June 19, 2013 7:23 AM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] Use case for PortfolioAnalytics Brian suggested that I send a few thoughts on a use-case for PortfolioAnalytics. I'm going to use a real situation, although I'm going to skip over some of the details about *why* some of these issues exist. Apologies in advance for the length - I couldn't restrain myself from talking more broadly around the specifics I lay out here. The workflow I'm presenting here is similar to the one developed in presentations that Brian has already pointed out: - http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf - http://www.rinfinance.com/agenda/2010/Carl+Peterson+Boudt_Tutorial.pdf Many of the things I say here are reflected in those presentations, but I hope I can emphasize a few points here. In this use case, I'm more concerned about describing the decisions than the workflow or the specific data. Portfolio construction is still a frequently neglected part of the investment process. It has, in practice, come a long way since the days of Jessie Livermore-style overbetting (he lost 90% his wealth in 1907 on a blown cotton trade). That said, investors still don't always understand the specific trade-offs they are making in optimization, or the sensitivity the techniques show to the data and estimation error around the inputs. Many, many investors prefer to use simple rules-of-thumb and common sense instead - which work to keep the portfolio out of hot water - not realizing that they could add actually add value during this step by explicitly expressing their objectives and constraints and generating alternatives. In going through this exercise of constructing a portfolio from individual elements, there are a few issues that are important to me as a decision-maker. In general, I want: - to be approximately correct rather than precisely wrong (accuracy of the method should be considered in the context of estimation error); - to define risk as potential loss rather than volatility (even Markowitz wanted this); - the flexibility to define any kind of objective (in fact, I want to compare several); - the ability to combine constraints (I have specific position-level reasons for doing so); - a framework for considering different sets of portfolio constraints for comparison through time; and - to build intuition about optimization through visualization. Many people would add that they are sensitive to the calculation time, as well. Given the rebalancing frequency proposed in this case, calculation time doesn't matter as much in this case. 0. Understand the portfolio constituents Often left unsaid, but this step affects judgments about what comes after. In this case, I have three categories of commodities futures traders - one of which tends to be extremely volatile but uncorrelated among the peers (sector specialists). Another category shows high correlation among themselves, but run relatively low volatility strategies (trend followers). Those strategies can show autocorrelation, however, and can find themselves in extended drawdowns. The third category of manager are somewhere in between those two (macro). We tend to think of these in groups, even where managers don't fit perfectly. 1. Set limits that are position-specific The first step is to specify individual position limits. In my more specific case, where I'm allocating with actual managers (as opposed to the EDHEC indexes in the 2012 presentation), those may come at the: - individual level (e.g., no single manager can represent more than 15% of AUM) or - group level (e.g., no group of managers using Systematic strategies can represent more than 30% of AUM). There may be a specific direction of change: - Managers may not be able to accept additional capital, so that the position may only be reduced from current levels; - Managers may be receiving an allocation subject to a minimum amount of investment; - Positions may be subject to gates or lockups such that the investment may not be decreased; or - We simply don't want too much manager risk, so that if they imploded our losses would be limited to the amount invested (e.g., 10% of capital). These may be specified across all funds, for a sub-set of funds, or for each individual fund. In my specific case, I have an individual maximum for each type of manager (e.g., a sector specialist may be no more than 10% of the portfolio). I am not setting limits to the group today, but I may want to in the future (e.g., no more than 60% to the sector specialists). I usually have a minimum investment level below which I want to drop the investment (usually I want a position larger than 1-2%). I can't short in this portfolio (individual weight must be greater than 0%). 2. Set limits that are portfolio-specific I have different sets of constraints around the sum of the portfolio weights. In some cases, all my managers' weights must sum to (about) 100%. In this particular case, I can add leverage in such a way that the sum can be greater than 100%. There's always an upper limit to that leverage as well (e.g., no more than 200%). I also want to target a particular portfolio volatility (>10%), with the view that volatility is also an element of upside potential. In this case, competitive products have a volatility between 10-16%, and since investors use a heuristic of fees relative to the level of volatility I want to make sure the portfolio volatility is roughly in that range. So far, all of these issues represent constraints in the context that Brian already discussed, in that they: - operate only on the weight vector (or a calculation of its components); - may include metadata about the components (the type of manager); or - describe the inputs to the objective function (target portfolio volatility and/or return level). 3. Choose measures for constructing portfolio objectives >From there, I then have a variety of measures by which I want to evalute (rank) portfolios. An objective may be composed of several measures, which then have to be combined in a way that makes sense. That is, the measures may need to be normalized such that they relate to one another proportionally. I may want to vary my objectives by the amount of estimation I'm required to do: - if I don't want to do any estimation, I can think about a 1/N portfolio. That's also a good benchmark to determine whether I can add value in portfolio construction; OR - if I believe my volatility estimate, then I want a 1/sigma portfolio; OR - if I have confidence in both my volatility and correlation estimates, then I'm able to calculate: + minimum variance or minimum CVaR portfolio; + equal volatility or CVaR contribution portfolio; OR - If I have good estimates for higher moments, I can do the same with modified CVaR; OR - If I have an estimate for mean return I can examine the tradeoff between return and risk (e.g., MSRP). As we move down this list, additional estimates are required and we lose degrees of freedom. I'm going to ignore the issues of forecast horizon here. In this particular use case, I want to: - equalize the (ex-ante) contribution of volatility to the portfolio among it's constituent managers. - target a volatility level (as described above). 4. Examine other objectives But I also want to understand that portfolio in the context of the other possible objectives I could have, namely: - Equal contribution to: + Weight (1/N); + Variance without correlation (1/sigma); ! + Variance with correlation; <- my objective + Risk (mETL). - Reward to Risk: + Mean-Variance; + Mean-mETL. - Minimum: + Variance; + Risk (mETL); Even though I have a preference for one objective, I want to know about other objectives as well. Those might be considered allocation benchmarks, but they serve a role as markers (or buoys) in the constraint space rather than as 'benchmarks' for performance. Each of these is best expressed as an objective, of course, since they are each: - calculated on a weight vector; and - describe a portfolio objective for the outcome of the optimization (e.g., it describes the portfolio that is the minimum variance portfolio within the constraint space). The portfolios when compared this way can reveal some interesting aspects of the underlying decision - say, when comparing a 1/sigma portfolio to one calculated with correlation estimates one can see the impact of the trade-offs made for the correlation among the constituents. In my experience, constraints are typically very position-specific and must be related directly to the data of the instrument in question; objectives tend to be related to the portfolio in the aggregate. This brief description probably doesn't go far enough into the broader workflow. It's pretty obvious and it isn't as linear as I'm representing it, but I'll state it anyway: 1. organize data 2. describe the data statistically 3. develop methods for construction (broken down in the steps above) 4. get results 5. analyze results 6. interact with collaborators This broader context suggests a few more requirements for PortfolioAnalytics from the user's perspective. One is that the traceability of the solutions has to be maintained and well documented (What did I do? Where did the data come from? How were the results calculated? With what parameters?). We can end up with a great deal of derived data, and the user will probably need some helper functions to navigate through it all. Second, the results need to be well organized such that they can be easily compared, especially visually. Currently, this requires a great deal of manual manipulation to organize the data and make the objectives comparable. This also requires some thought when specifying objectives: calculations that need to be done for comparison that don't carry any weight in the objective itself (e.g., portfolio variance is calculated for a mean-CVaR objective, but has a multiplier of zero). Third, the sensitivity of the results should be addressable. This is probably a topic unto itself, because it can be manifested in several ways. Along one of these dimensions is the ability to examine portfolios that are not the optimal, but are "close-by," in that they rank below the optimal (e.g., the 10 nearest portfolios in RP). Food for thought, pcc -- Peter Carl http://www.braverock.com/peter _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From martinrd at comcast.net Thu Jun 20 08:30:15 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 23:30:15 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Message-ID: <01b601ce6d7f$ad306ff0$07914fd0$@comcast.net> This time with the slides. I will add the examples soon and send. Ross, you will recognize this from 543. Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Kris Boudt Sent: Tuesday, June 18, 2013 1:35 AM To: PortfolioAnalytics; gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: [Doug] It should be noted that turnover can be handled through either through constraints or a penalty in the objective (see the attached slides). Last summer I was not very happy with the objective penalty behavior but need to revisit it. - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. [Doug] We definitely need both of the above options, the second obviously for back-testing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. [Doug] This is a good idea. [Doug] Thanks. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [ mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin < mailto:martinrd at comcast.net>> wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org> > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > < mailto:gsoc-porta at lists.r-forge.r-project.org> > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5. turnover constraints vs turnover objective penalty.ppt Type: application/vnd.ms-powerpoint Size: 229888 bytes Desc: not available URL: From martinrd at comcast.net Thu Jun 20 08:30:15 2013 From: martinrd at comcast.net (Doug Martin) Date: Wed, 19 Jun 2013 23:30:15 -0700 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> Message-ID: <01b601ce6d7f$ad306ff0$07914fd0$@comcast.net> This time with the slides. I will add the examples soon and send. Ross, you will recognize this from 543. Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Kris Boudt Sent: Tuesday, June 18, 2013 1:35 AM To: PortfolioAnalytics; gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] structure of the specification objects Intrinsically, it means that what we used to call "soft constraints" (the constraints that can be violated) are classified as "objectives". Seems good to me, but we need to make sure that each objective has then its multiplier/scaling factor, which by default is 1, but for soft constraints will typically be higher. 1. I would also include a turnover objective, whereby there seem to be two possibilities: [Doug] It should be noted that turnover can be handled through either through constraints or a penalty in the objective (see the attached slides). Last summer I was not very happy with the objective penalty behavior but need to revisit it. - provide directly the weights at the time of rebalancing - provide the weights at the previous rebalancing data and include a function that, uses the return data to calculate the weights at the time of rebalancing. [Doug] We definitely need both of the above options, the second obviously for back-testing. 2. In practical work, I work with a price data file that has all possible constituents and for each selection date a list of allowed constituents. Typically an xts with in row the selection dates and columns the assets with 1 if included and 0 otherwise. Such a constituents check could be in the constraints object. [Doug] This is a good idea. [Doug] Thanks. Thanks, Kris -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [ mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Monday, June 17, 2013 1:51 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: [GSoC-PortA] structure of the specification objects I've been thinking a lot about this over the last week, and I'm going to propose an architecture for discussion. Right now, we have an object of type 'constraints' which also includes 'objectives'. I have always logically separated objectives from constraints in my head by saying that - constraints are something that are: 1. operable only on the weight vector and 2. metadata information about the portfolio (e.g. groups, asset names) 3. describe the *inputs* to the objective function - objectives are part of the *objective function* that 1. you calculate on the weight vector 2. describe business objectives for the *outcome* of the optimization I think we need to split these apart explicitly, and make the distinctions, and why they exist, more obvious to the user. First, I think we need a container object of type 'portfolio' or 'portfoliospec' that know what assets you have, and how you want to group them (e.g. sectors, size, whatever). This is data about the portfolio, and is *independent* of and constraints or objectives you might have on that portfolio. Next, I think we need a constraints object. Constraints, by their nature, are tied to the portfolio that they are created for. They require knowledge of the portfolio metadata, or at the very least things like box constraints are typically derived from the size of the portfolio. Finally, I think we need a separate 'objectives' object. To have objectives objects, with names, that specify things like minimum variance, maximum Sharpe ratio, or risk parity objectives. These can be completely separate from the assets in the portfolio, and we could provide a number of template objective objects with the package. I think the current way objectives are added and manipulated is fine, we just need to break it off a bit. For the template objectives that can be acted on by specific more efficient solvers, I think separating the objectives out will make it easier to identify that. optimize.portfolio needs all three of these components, even today, to do anything. So I think we need to sort out how to specify them, then how to combine them. I suggest that we start by getting these specifications right first. Then we can start surgery in other parts of the code to use the new specification forms. My current thoughts are that a portfolio *contains* one or more constraints objects, since the constraints objects are dependent on the portfolio composition. Then you optimize a portfolio by passing a portfolio specification into the optimizer, and telling it which constraints object to use (if you have more than one attached to the portfolio) and what objectives object you want to apply. Hopefully this is clear. It's a proposal, so comments/critique/ideas always welcome. Today is officially the first day of coding, so we need to get moving. Regards, Brian On 06/06/2013 06:54 AM, Ross Bennett wrote: > Professor Martin, > > Thank you for the feedback. Based on the suggestions from yourself and > Brian Peterson, I will start with the constraints portion. I'll spend > some more time looking at Kirk Li's code to understand it better and > think about how to best integrate into PortfolioAnalytics. I should be > able to use some ideas from it. > > I have been reviewing the code for PortfolioAnalytics and it seems > that there is already a lot of the infrastructure in place for the > constraint object and the optimizer so I don't want to maintain the > interface to the optimize.portfolio function as much as possible. > > As an example, here is a code snippet of how box constraints and group > constraints are currently specified. > # General constraints object > # Box constraints of minimum weight = 0.1 and maximum weight = 0.5 for > assets gen.constr <- constraint(assets = colnames(edhec), min=0.1, max > =0.5, min_sum=1, max_sum=1) > > # Add group constraints to the gen.constr object groups <- c(3,3,3,4) > gen.constr$groups <- groups gen.constr$cLO <- rep(0.15,length(groups)) > gen.constr$cUP <- rep(0.30,length(groups)) > > Following the outline on the GSOC2013 PortfolioAnalytics rwiki page, > we would remove box constraints from the constraint() function and > create an add.constraint() function to add/update box, group, and > other constraint types as applicable. > > # Create the initial constraint object # Pass in a character vector of > asset names for assets # Portfolio weights must sum to 1 (min_sum=1 > and max_sum=1) gen.constr <- constraint(assets = colnames(edhec), > min_sum=1, max_sum=1) > > # Add box constraints with asset weights greater than 0.1 and less > than 0.5 gen.constr <- add.constraint(constraints=gen.constr, type="box", > min=0.1, max=0.5) > > # Group constraints for 10 assets > # Separate assets into groups of 3, 3, and 4 assets # Specify the > group weights as a vector # The first 3 assets shall have a group > weight of greater than 0.15 and less than 0.3 # The next 3 assets > shall have a group weight of greater than 0.1 and less than 0.35 # The > last 4 assets shall have a group weight greater than 0 and less than > 0.4 constr <- add.constraint(constraints=constr, type="group", > groups=c(3, 3, 4), > group_min=c(0.15, 0.1, 0), > group_max=c(0.3, 0.35, 0.4)) > > This keeps the constraint object and interface to optimize.portfolio() > consistent with the existing code, but simplifies it and makes adding > box and group constraints very similar. > > I will continue to review the PortfolioAnalytics source code and we > can revisit this so that I can hit the ground running when finals are > over and start as soon as possible. > > Regards, > Ross Bennett > > > On Wed, Jun 5, 2013 at 9:23 AM, Doug Martin < mailto:martinrd at comcast.net>> wrote: > > Ross and all,____ > > __ __ > > I think starting with the constraints is a good idea. But I think a > review of where PortfolioAnalytics functionality is as of last > year's GSoC should be the first priority. How planning on that week > after next (Guy, Eric and I all have finals week to get out of the > way next week).____ > > __ __ > > With regard to constraints, that approach that Kirk Li (UW > Statistics Ph.D. student and AMATH 543 TA this spring) developed > looks useful. Ross you already have the code which I posted to the > class web site. I have attached it here for others on this project > to have a look at. Let's review that after the end of next week > when final exams and the spring quarter are completely behind > us.____ > > __ __ > > Doug____ > > __ __ > > __ __ > > *From:*gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org> > [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org > < mailto:gsoc-porta-bounces at lists.r-forge.r-project.org>] *On Behalf > Of *Ross Bennett > *Sent:* Wednesday, June 05, 2013 4:22 AM > *To:* gsoc-porta at lists.r-forge.r-project.org > < mailto:gsoc-porta at lists.r-forge.r-project.org> > *Subject:* [GSoC-PortA] Schedule for PortfolioAnalytics Coding____ > > __ __ > > All,____ > > __ __ > > In my GSOC proposal, the order of what components I planned to write > code for was as follows:____ > > 1. Utility Functions (print, summary, plot, etc.)____ > 2. Constraints____ > 3. Example Functionality____ > > In speaking with Brian Peterson at R/Finance, he suggested that the > constraints portion be done first. I am fine with doing constraints > first. Would you like me to submit a revised schedule and > implementation plan with constraints first?____ > > __ __ > > Thanks,____ > > Ross Bennett____ _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5. turnover constraints vs turnover objective penalty.ppt Type: application/vnd.ms-powerpoint Size: 229888 bytes Desc: not available URL: From rossbennett34 at gmail.com Thu Jun 20 19:02:14 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Thu, 20 Jun 2013 12:02:14 -0500 Subject: [GSoC-PortA] structure of the specification objects In-Reply-To: <51C02842.3040101@braverock.com> References: <001a01ce61f8$50ebafd0$f2c30f70$@comcast.net> <51BEF82E.5030402@braverock.com> <3717237620598B4CB591543D679DA3860361A508@ICTS-S-MBX5.luna.kuleuven.be> <51C02842.3040101@braverock.com> Message-ID: Re: Turnover Objective Function I have gone through the slides and code, but did not see an example using turnover as an objective function. http://www.rinfinance.com/**agenda/2010/Carl+Peterson+**Boudt_Tutorial.pdf https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics I have also gone through this presentation and code, but it does not look like turnover was used as an objective measure. http://www.rinfinance.com/agenda/2012/workshop/Carl+Peterson.pdf https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/sandbox/script.workshop2012.R?view=markup&root=returnanalytics Slide 36 shows a chart of Turnover from Equal Weight, but it looks to be that the turnover was calculated ex-post. Is there a different example you have where turnover is specified as an objective function for the optimization? Both presentations have some great examples and charts for visualization that I can use for later this summer. Thanks, Ross Bennett On Tue, Jun 18, 2013 at 4:28 AM, Brian G. Peterson wrote: > On 06/18/2013 03:34 AM, Kris Boudt wrote: > >> Intrinsically, it means that what we used to call "soft constraints" >> (the constraints that can be violated) are classified as >> "objectives". Seems good to me, but we need to make sure that each >> objective has then its multiplier/scaling factor, which by default is >> 1, but for soft constraints will typically be higher. >> > > I've never liked the 'soft constraints' construct, as it's not very > clear. My distinction was between things we can do *before* calling the > objective function, only on the weights vector, which we'll call > 'constraints' and things that we can do only by acting on the set of all > possible weights vectors which satisfy the constraints (the objectives). > > We've always supported both multipliers and penalties for the individual > objectives, from the very first version of the code. In practice, two > things go into selecting a multiplier: > > a. the objectives must inherently be of similar scale, e.g. both returning > a return number of roughly equal magnitude. if they are not of roughly > equal magnitude, then the user needs to use the multiplier to adjust. > > b. the user must understand that to over/under weight a specific objective > with respect to other objectives, they need to adjust either the multiplier > or the penalty, or both. > > > 1. I would also include a turnover objective, whereby there seem to >> be two possibilities: >> >> - provide directly the weights at the time of rebalancing - provide >> the weights at the previous rebalancing data and include a function >> that, uses the return data to calculate the weights at the time of >> rebalancing. >> > > We demonstrated how to use a turnover objective function in our 2012 > R/Finance seminar: > > http://www.rinfinance.com/**agenda/2010/Carl+Peterson+**Boudt_Tutorial.pdf > > see page 36 > > with code here: > https://r-forge.r-project.org/**scm/viewvc.php/*checkout*/pkg/** > PortfolioAnalytics/sandbox/**script.workshop2010.R?root=**returnanalytics > > It would be nice to get the chart from page 36 into the package if we have > time later in the summer. > > In practice, turnover can be dealt with either as a constraint (in some > solvers) or as an objective. Doug would need to comment on using it as a > constraint, I know there's a brief discussion of that in MPO(v1) or perhaps > Bernd's book, both of which I have at the office right now. > > I've always done it as an objective, which works fine in current code. > > It would probably be nice to include the turnover function in the package > directly and document it. > > > 2. In practical work, I work with a price data file that has all >> possible constituents and for each selection date a list of allowed >> constituents. Typically an xts with in row the selection dates and >> columns the assets with 1 if included and 0 otherwise. Such a >> constituents check could be in the constraints object. >> > > It seems to me that this belongs in the portfolio specification. That is > to say that the constituents of the portfolio are defined in the portfolio > specification as a time series. > > Another way to think of this, of course, is to say that an equality > constraint (weight=0) is imposed on some of the overall universe of > constituents at certain times. You can also do it with box constraints > that vary through time. > > I think I prefer the first approach as more intuitive for the user, and > dealing with subsetting the input data to match the allowed constituents > within optimize.portfolio. We're better off never giving data to the > solvers that they don't need, and if the constituent is out of the > portfolio, then the subset can happen before the optimization step. That's > just an initial feeling though. > > I think we need to think more about what it does to the code to support > time-varying constraints and portfolio specifications, and then put it > wherever it makes the most sense. I don't see any reason to work on that > this week though, as the overall structure seems more important to me at > this point, and I think we can cover this when we have a more coherent > framework to work from. > > Regards, > > Brian > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Sat Jun 22 03:31:35 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Fri, 21 Jun 2013 20:31:35 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() Message-ID: Hi All, I have a quick question on how the multiplier is used inside of the constrained_objective() function. The code below is from lines 230-248 in constrained_objective.R for objects of class return_objective and portfolio_risk_objective. # now set the new value of the objective output if(inherits(objective,"return_objective")){ if (!is.null(objective$target) & is.numeric(objective$target)){ # we have a target out = out + penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) } # target is null or doesn't exist, just maximize, or minimize violation of constraint out = out + abs(objective$multiplier)*tmp_measure } # end handling for return objectives if(inherits(objective,"portfolio_risk_objective")){ if (!is.null(objective$target) & is.numeric(objective$target)){ # we have a target out = out + penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) #should we also penalize risk too low for risk targets? or is a range another objective? # # half penalty for risk lower than target # if( prw < (.9*Riskupper) ){ out = out + .5*(penalty*( prw - Riskupper)) } } # target is null or doesn't exist, just maximize, or minimize violation of constraint out = out + abs(objective$multiplier)*tmp_measure } # univariate risk objectives When specifying objectives, if we want to maximize return, we would set multiplier=-1 and if we wanted to minimize risk we would set multiplier=1 in the objective object. Because the code below takes the absolute value of the multiplier (abs(objective$multiplier)), doesn't this make the multiplier have no effect? Any thoughts or insight is much appreciated. Thanks, Ross Bennett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sat Jun 22 05:09:58 2013 From: brian at braverock.com (Brian G. Peterson) Date: Fri, 21 Jun 2013 22:09:58 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: References: Message-ID: <51C51586.4030406@braverock.com> On 06/21/2013 08:31 PM, Ross Bennett wrote: > Hi All, > > I have a quick question on how the multiplier is used inside of the > constrained_objective() function. > > The code below is from lines 230-248 in constrained_objective.R for > objects of class return_objective and portfolio_risk_objective. > > # now set the new value of the objective output > if(inherits(objective,"return_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # end handling for return objectives > > if(inherits(objective,"portfolio_risk_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > #should we also penalize risk too low for risk targets? > or is a range another objective? > # # half penalty for risk lower than target > # if( prw < (.9*Riskupper) ){ out = out + > .5*(penalty*( prw - Riskupper)) } > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # univariate risk objectives > > When specifying objectives, if we want to maximize return, we would set > multiplier=-1 and if we wanted to minimize risk we would set > multiplier=1 in the objective object. > > Because the code below takes the absolute value of the multiplier > (abs(objective$multiplier)), doesn't this make the multiplier have no > effect? > > Any thoughts or insight is much appreciated. I think your analysis of the code seems correct. Returns are positive numbers, risk, generally negative numbers. From the svn logs, the abs was added in revision 2327 by Peter with a commit log: - added abs to multiplier when using targets to keep searches directionally correct the diff between that revision and the previous revision is here: https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/R/constrained_objective.R?root=returnanalytics&r1=2326&r2=2327 From the comment, it would seem that Peter figured that he needed abs() when there was a target, and multiplier is just a scalar. That doesn't explain why it would be applied in all cases. Maybe Peter remembers why? I'll update my development environment to current code in the morning and see if I can't run some of the code from the seminar script: /pkg/PortfolioAnalytics/sandbox/script.workshop2012.R usually, when I'm trying to understand what's going on in some of this code, I use simple objectives (maximize return, minimize risk, etc), a small portfolio of only a few assets, and random portfolios so that I can see the entire space and figure out if things are being done correctly. I'll try to construct some sort of contrived example tomorrow. Regards, - Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Sat Jun 22 05:44:33 2013 From: martinrd at comcast.net (Doug Martin) Date: Fri, 21 Jun 2013 20:44:33 -0700 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <51C51586.4030406@braverock.com> References: <51C51586.4030406@braverock.com> Message-ID: <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> Small comment: a lot (if not most) literature on risk takes risk as a positive number, so puts a minus sign on the VaR quantile, etc. I previously noticed that PerformanceAnalytics takes risk as negative, had the thought to change that. -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Friday, June 21, 2013 8:10 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() On 06/21/2013 08:31 PM, Ross Bennett wrote: > Hi All, > > I have a quick question on how the multiplier is used inside of the > constrained_objective() function. > > The code below is from lines 230-248 in constrained_objective.R for > objects of class return_objective and portfolio_risk_objective. > > # now set the new value of the objective output > if(inherits(objective,"return_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # end handling for return objectives > > if(inherits(objective,"portfolio_risk_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > #should we also penalize risk too low for risk targets? > or is a range another objective? > # # half penalty for risk lower than target > # if( prw < (.9*Riskupper) ){ out = out + > .5*(penalty*( prw - Riskupper)) } > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # univariate risk objectives > > When specifying objectives, if we want to maximize return, we would > set > multiplier=-1 and if we wanted to minimize risk we would set > multiplier=1 in the objective object. > > Because the code below takes the absolute value of the multiplier > (abs(objective$multiplier)), doesn't this make the multiplier have no > effect? > > Any thoughts or insight is much appreciated. I think your analysis of the code seems correct. Returns are positive numbers, risk, generally negative numbers. From the svn logs, the abs was added in revision 2327 by Peter with a commit log: - added abs to multiplier when using targets to keep searches directionally correct the diff between that revision and the previous revision is here: https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/R/constr ained_objective.R?root=returnanalytics&r1=2326&r2=2327 From the comment, it would seem that Peter figured that he needed abs() when there was a target, and multiplier is just a scalar. That doesn't explain why it would be applied in all cases. Maybe Peter remembers why? I'll update my development environment to current code in the morning and see if I can't run some of the code from the seminar script: /pkg/PortfolioAnalytics/sandbox/script.workshop2012.R usually, when I'm trying to understand what's going on in some of this code, I use simple objectives (maximize return, minimize risk, etc), a small portfolio of only a few assets, and random portfolios so that I can see the entire space and figure out if things are being done correctly. I'll try to construct some sort of contrived example tomorrow. Regards, - Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From martinrd at comcast.net Sat Jun 22 05:44:33 2013 From: martinrd at comcast.net (Doug Martin) Date: Fri, 21 Jun 2013 20:44:33 -0700 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <51C51586.4030406@braverock.com> References: <51C51586.4030406@braverock.com> Message-ID: <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> Small comment: a lot (if not most) literature on risk takes risk as a positive number, so puts a minus sign on the VaR quantile, etc. I previously noticed that PerformanceAnalytics takes risk as negative, had the thought to change that. -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Friday, June 21, 2013 8:10 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() On 06/21/2013 08:31 PM, Ross Bennett wrote: > Hi All, > > I have a quick question on how the multiplier is used inside of the > constrained_objective() function. > > The code below is from lines 230-248 in constrained_objective.R for > objects of class return_objective and portfolio_risk_objective. > > # now set the new value of the objective output > if(inherits(objective,"return_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # end handling for return objectives > > if(inherits(objective,"portfolio_risk_objective")){ > if (!is.null(objective$target) & > is.numeric(objective$target)){ # we have a target > out = out + > penalty*abs(objective$multiplier)*abs(tmp_measure-objective$target) > #should we also penalize risk too low for risk targets? > or is a range another objective? > # # half penalty for risk lower than target > # if( prw < (.9*Riskupper) ){ out = out + > .5*(penalty*( prw - Riskupper)) } > } > # target is null or doesn't exist, just maximize, or > minimize violation of constraint > out = out + abs(objective$multiplier)*tmp_measure > } # univariate risk objectives > > When specifying objectives, if we want to maximize return, we would > set > multiplier=-1 and if we wanted to minimize risk we would set > multiplier=1 in the objective object. > > Because the code below takes the absolute value of the multiplier > (abs(objective$multiplier)), doesn't this make the multiplier have no > effect? > > Any thoughts or insight is much appreciated. I think your analysis of the code seems correct. Returns are positive numbers, risk, generally negative numbers. From the svn logs, the abs was added in revision 2327 by Peter with a commit log: - added abs to multiplier when using targets to keep searches directionally correct the diff between that revision and the previous revision is here: https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/R/constr ained_objective.R?root=returnanalytics&r1=2326&r2=2327 From the comment, it would seem that Peter figured that he needed abs() when there was a target, and multiplier is just a scalar. That doesn't explain why it would be applied in all cases. Maybe Peter remembers why? I'll update my development environment to current code in the morning and see if I can't run some of the code from the seminar script: /pkg/PortfolioAnalytics/sandbox/script.workshop2012.R usually, when I'm trying to understand what's going on in some of this code, I use simple objectives (maximize return, minimize risk, etc), a small portfolio of only a few assets, and random portfolios so that I can see the entire space and figure out if things are being done correctly. I'll try to construct some sort of contrived example tomorrow. Regards, - Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From brian at braverock.com Sat Jun 22 05:55:19 2013 From: brian at braverock.com (Brian G. Peterson) Date: Fri, 21 Jun 2013 22:55:19 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> Message-ID: <51C52027.7080601@braverock.com> On 06/21/2013 10:44 PM, Doug Martin wrote: > Small comment: a lot (if not most) literature on risk takes risk as a > positive number, so puts a minus sign on the VaR quantile, etc. I > previously noticed that PerformanceAnalytics takes risk as negative, had the > thought to change that. PerformanceAnalytics allows it either way, user's choice. Risk is risk of *loss*. Period. Now, the math is sometimes/often easier with a positive number, I admit, but it's typical for risk reports on real portfolios to describe those risks as negative numbers. So we allow either. Cheers, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Sat Jun 22 06:52:38 2013 From: martinrd at comcast.net (Doug Martin) Date: Fri, 21 Jun 2013 21:52:38 -0700 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <51C52027.7080601@braverock.com> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> Message-ID: <002801ce6f04$526a65e0$f73f31a0$@comcast.net> Brian and Ross (and anyone else who wishes to comment), Thanks for the late evening response. Good to allow both conventions wrt the sign of risk. Will check the leading portfolio and risk products for most frequent choice, and then we should go to with the majority. My priors are that the plus sign will win. Much more importantly: I just started browsing the PortfolioAnalytics help files manual, slowly starting to refresh my memory of what little I understood last summer. Please see the attached version of August 26 (not the latest I know, but almost the last if not the last the Hezky edited)), where I have added a small number of comments here and there. See in particular my comments on optimize.portfolio. It would be very helpful if the unfinished business of making ROI invisible to the user is taken care of, and soon we just have solve.QP and Rglpk_solve_LP as optimizer choice arguments. Even before getting all the constraint object changes, because then I could check running my solve.QP and Rglpk_solve_LP based examples (Ross you are quite familiar with these) run via optimize.portfolio and box constraints. Then come back to group constraints when they are implemented. Will keep browsing the manual, and try to understand what is going on. Brian, I don't know if you (and Ross) can manage, but examples at the end of each help file would be truly helpful (no pun intended). Thanks, Doug P.S. Brian, I hope you didn't hear my phone call at 11:15 or so - I thought you were still live on email and then realized that almost 30 minutes had passed and you had probably crashed. -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Friday, June 21, 2013 8:55 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() On 06/21/2013 10:44 PM, Doug Martin wrote: > Small comment: a lot (if not most) literature on risk takes risk as a > positive number, so puts a minus sign on the VaR quantile, etc. I > previously noticed that PerformanceAnalytics takes risk as negative, > had the thought to change that. PerformanceAnalytics allows it either way, user's choice. Risk is risk of *loss*. Period. Now, the math is sometimes/often easier with a positive number, I admit, but it's typical for risk reports on real portfolios to describe those risks as negative numbers. So we allow either. Cheers, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PortfolioAnalytics.pdf Type: application/pdf Size: 174183 bytes Desc: not available URL: From martinrd at comcast.net Sat Jun 22 06:52:38 2013 From: martinrd at comcast.net (Doug Martin) Date: Fri, 21 Jun 2013 21:52:38 -0700 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <51C52027.7080601@braverock.com> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> Message-ID: <002801ce6f04$526a65e0$f73f31a0$@comcast.net> Brian and Ross (and anyone else who wishes to comment), Thanks for the late evening response. Good to allow both conventions wrt the sign of risk. Will check the leading portfolio and risk products for most frequent choice, and then we should go to with the majority. My priors are that the plus sign will win. Much more importantly: I just started browsing the PortfolioAnalytics help files manual, slowly starting to refresh my memory of what little I understood last summer. Please see the attached version of August 26 (not the latest I know, but almost the last if not the last the Hezky edited)), where I have added a small number of comments here and there. See in particular my comments on optimize.portfolio. It would be very helpful if the unfinished business of making ROI invisible to the user is taken care of, and soon we just have solve.QP and Rglpk_solve_LP as optimizer choice arguments. Even before getting all the constraint object changes, because then I could check running my solve.QP and Rglpk_solve_LP based examples (Ross you are quite familiar with these) run via optimize.portfolio and box constraints. Then come back to group constraints when they are implemented. Will keep browsing the manual, and try to understand what is going on. Brian, I don't know if you (and Ross) can manage, but examples at the end of each help file would be truly helpful (no pun intended). Thanks, Doug P.S. Brian, I hope you didn't hear my phone call at 11:15 or so - I thought you were still live on email and then realized that almost 30 minutes had passed and you had probably crashed. -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Friday, June 21, 2013 8:55 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() On 06/21/2013 10:44 PM, Doug Martin wrote: > Small comment: a lot (if not most) literature on risk takes risk as a > positive number, so puts a minus sign on the VaR quantile, etc. I > previously noticed that PerformanceAnalytics takes risk as negative, > had the thought to change that. PerformanceAnalytics allows it either way, user's choice. Risk is risk of *loss*. Period. Now, the math is sometimes/often easier with a positive number, I admit, but it's typical for risk reports on real portfolios to describe those risks as negative numbers. So we allow either. Cheers, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PortfolioAnalytics.pdf Type: application/pdf Size: 174183 bytes Desc: not available URL: From rossbennett34 at gmail.com Sat Jun 22 19:12:34 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Sat, 22 Jun 2013 12:12:34 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <002801ce6f04$526a65e0$f73f31a0$@comcast.net> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> <002801ce6f04$526a65e0$f73f31a0$@comcast.net> Message-ID: Professor Martin and others, I just added a testing script to match your examples in your 1. theory review weights constrained mvo v5.pdf slides (attached). See committed revision 2402 with sandbox/testing_ROI_Martin.R. Currently, the user just chooses optimize_method="ROI" and optimize.portfolio() takes care of choosing the ROI plugin for solve.QP or Rglpk_solve_LP based on the objectives specified. If "var" is specified as the only objective, then the plug-in selected is quadprog for solve.QP to minimize variance. If "var" and "mean" are specified as objectives, then the plug-in selected is quadprog for solve.QP to maximize quadratic utility. If "mean" is specified as the only objective, then the plug-in selected is glpk for Rglpk_solve_LP to maximize return. If "CVaR" is specified as the only objective, then the plug-in selected is glpk for Rglpk_solve_LP to minimize sample CVaR. This is nice for the user because they don't need to know when to use quadprog and vice versa for glpk. It appears that the reference manual may be out of date. I could easily make a vignette with several examples about the functionality that optimize_method="ROI" offers. I'd prefer to do this after we are all satisfied how to set the portfolio object with constraints and objectives. Regards, Ross Bennett On Fri, Jun 21, 2013 at 11:52 PM, Doug Martin wrote: > Brian and Ross (and anyone else who wishes to comment),**** > > ** ** > > Thanks for the late evening response. Good to allow both conventions wrt > the sign of risk. Will check the leading portfolio and risk products for > most frequent choice, and then we should go to with the majority. My > priors are that the plus sign will win.**** > > ** ** > > Much more importantly: I just started browsing the PortfolioAnalytics > help files manual, slowly starting to refresh my memory of what little I > understood last summer. Please see the attached version of August 26 (not > the latest I know, but almost the last if not the last the Hezky edited)), > where I have added a small number of comments here and there. See in > particular my comments on optimize.portfolio. It would be very helpful if > the unfinished business of making ROI invisible to the user is taken care > of, and soon we just have solve.QP and Rglpk_solve_LP as optimizer choice > arguments. Even before getting all the constraint object changes, because > then I could check running my solve.QP and Rglpk_solve_LP based examples > (Ross you are quite familiar with these) run via optimize.portfolio and box > constraints. Then come back to group constraints when they are implemented. > **** > > ** ** > > Will keep browsing the manual, and try to understand what is going on. > Brian, I don?t know if you (and Ross) can manage, but examples at the end > of each help file would be truly helpful (no pun intended).**** > > ** ** > > Thanks,**** > > Doug**** > > ** ** > > P.S. Brian, I hope you didn?t hear my phone call at 11:15 or so ? I > thought you were still live on email and then realized that almost 30 > minutes had passed and you had probably crashed. **** > > ** ** > > ** ** > > ** ** > > -----Original Message----- > From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. > Peterson > Sent: Friday, June 21, 2013 8:55 PM > To: gsoc-porta at r-forge.wu-wien.ac.at > Subject: Re: [GSoC-PortA] Quick question on multipliers and > constrained_objective() > > ** ** > > On 06/21/2013 10:44 PM, Doug Martin wrote:**** > > > Small comment: a lot (if not most) literature on risk takes risk as a * > *** > > > positive number, so puts a minus sign on the VaR quantile, etc. I **** > > > previously noticed that PerformanceAnalytics takes risk as negative, *** > * > > > had the thought to change that.**** > > ** ** > > PerformanceAnalytics allows it either way, user's choice.**** > > ** ** > > Risk is risk of *loss*. Period.**** > > ** ** > > Now, the math is sometimes/often easier with a positive number, I admit, > but it's typical for risk reports on real portfolios to describe those > risks as negative numbers.**** > > ** ** > > So we allow either.**** > > ** ** > > ** ** > > Cheers,**** > > ** ** > > Brian**** > > ** ** > > ** ** > > --**** > > Brian G. Peterson**** > > http://braverock.com/brian/**** > > Ph: 773-459-4973**** > > IM: bgpbraverock**** > > _______________________________________________**** > > GSoC-PortA mailing list**** > > GSoC-PortA at lists.r-forge.r-project.org**** > > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 1. theory review weights constrained mvo v5.pdf Type: application/pdf Size: 790881 bytes Desc: not available URL: From martinrd at comcast.net Mon Jun 24 02:28:15 2013 From: martinrd at comcast.net (Doug Martin) Date: Sun, 23 Jun 2013 17:28:15 -0700 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> <002801ce6f04$526a65e0$f73f31a0$@comcast.net> Message-ID: <006b01ce7071$b8727fd0$29577f70$@comcast.net> Ross and others, I will check this out. Meanwhile I offer a few comments below. Thanks, Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Saturday, June 22, 2013 10:13 AM To: PortfolioAnalytics Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() Professor Martin and others, I just added a testing script to match your examples in your 1. theory review weights constrained mvo v5.pdf slides (attached). See committed revision 2402 with sandbox/testing_ROI_Martin.R. Currently, the user just chooses optimize_method="ROI" and optimize.portfolio() takes care of choosing the ROI plugin for solve.QP or Rglpk_solve_LP based on the objectives specified. [Doug] I guess you mean that rev. 2402 still requires use of the argument optimize_method="ROI". I'm wondering if my earlier assumption from last summer that the code could be changed so that we don't have to specify ROI in an argument. Brian, what is your opinion on this? If "var" is specified as the only objective, then the plug-in selected is quadprog for solve.QP to minimize variance. [Doug] I think we need to specify the solver independently of the use of "var", with a default choice which could be solve.QP for now. I am not very confident about the quality of solve.QP. Maybe Brian knows if there are any test-bed benchmarks on it? In addition solve.QP lacks the following: (a) The ability to provide an initial set of weights, (b) Lack of a "verbose =" (or "trace =") with default F. If "var" and "mean" are specified as objectives, then the plug-in selected is quadprog for solve.QP to maximize quadratic utility. [Doug] This is a way to go. But maybe we want to be more explicit and clear by using "quadutil" or "qu" or some such? If "mean" is specified as the only objective, then the plug-in selected is glpk for Rglpk_solve_LP to maximize return. [Doug] Good If "CVaR" is specified as the only objective, then the plug-in selected is glpk for Rglpk_solve_LP to minimize sample CVaR. [Doug] Good. I would prefer to use the term ETL. This is just my habit, but I must say that CVaR was a bad naming choice by R&S, the reason being that the literal translation "conditional VaR" is not what it is about. I guess in situations like this where some People like one name and some like another, we can use aliases??? This is nice for the user because they don't need to know when to use quadprog and vice versa for glpk. It appears that the reference manual may be out of date. I could easily make a vignette with several examples about the functionality that optimize_method="ROI" offers. I'd prefer to do this after we are all satisfied how to set the portfolio object with constraints and objectives. Regards, Ross Bennett On Fri, Jun 21, 2013 at 11:52 PM, Doug Martin wrote: Brian and Ross (and anyone else who wishes to comment), Thanks for the late evening response. Good to allow both conventions wrt the sign of risk. Will check the leading portfolio and risk products for most frequent choice, and then we should go to with the majority. My priors are that the plus sign will win. Much more importantly: I just started browsing the PortfolioAnalytics help files manual, slowly starting to refresh my memory of what little I understood last summer. Please see the attached version of August 26 (not the latest I know, but almost the last if not the last the Hezky edited)), where I have added a small number of comments here and there. See in particular my comments on optimize.portfolio. It would be very helpful if the unfinished business of making ROI invisible to the user is taken care of, and soon we just have solve.QP and Rglpk_solve_LP as optimizer choice arguments. Even before getting all the constraint object changes, because then I could check running my solve.QP and Rglpk_solve_LP based examples (Ross you are quite familiar with these) run via optimize.portfolio and box constraints. Then come back to group constraints when they are implemented. Will keep browsing the manual, and try to understand what is going on. Brian, I don't know if you (and Ross) can manage, but examples at the end of each help file would be truly helpful (no pun intended). Thanks, Doug P.S. Brian, I hope you didn't hear my phone call at 11:15 or so - I thought you were still live on email and then realized that almost 30 minutes had passed and you had probably crashed. -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Friday, June 21, 2013 8:55 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] Quick question on multipliers and constrained_objective() On 06/21/2013 10:44 PM, Doug Martin wrote: > Small comment: a lot (if not most) literature on risk takes risk as a > positive number, so puts a minus sign on the VaR quantile, etc. I > previously noticed that PerformanceAnalytics takes risk as negative, > had the thought to change that. PerformanceAnalytics allows it either way, user's choice. Risk is risk of *loss*. Period. Now, the math is sometimes/often easier with a positive number, I admit, but it's typical for risk reports on real portfolios to describe those risks as negative numbers. So we allow either. Cheers, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 24 03:35:08 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Sun, 23 Jun 2013 20:35:08 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <006b01ce7071$b8727fd0$29577f70$@comcast.net> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> <002801ce6f04$526a65e0$f73f31a0$@comcast.net> <006b01ce7071$b8727fd0$29577f70$@comcast.net> Message-ID: Professor Martin, I look forward to your feedback after you have a chance to run the testing_ROI_Martin.R script. See a few additional comments below. Ross Bennett On Sun, Jun 23, 2013 at 7:28 PM, Doug Martin wrote: > Ross and others,**** > > ** ** > > I will check this out. Meanwhile I offer a few comments below.**** > > ** ** > > Thanks,**** > > Doug**** > > ** ** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Saturday, June 22, 2013 10:13 AM > *To:* PortfolioAnalytics > > *Subject:* Re: [GSoC-PortA] Quick question on multipliers and > constrained_objective()**** > > ** ** > > Professor Martin and others,**** > > ** ** > > I just added a testing script to match your examples in your 1. theory > review weights constrained mvo v5.pdf slides (attached). See committed > revision 2402 with sandbox/testing_ROI_Martin.R. Currently, the user just > chooses optimize_method="ROI" and optimize.portfolio() takes care of > choosing the ROI plugin for solve.QP or Rglpk_solve_LP based on the > objectives specified. **** > > *[Doug] I guess you mean that rev. 2402 still requires use of the > argument optimize_method=?ROI?. I?m wondering if my earlier assumption > from last summer that the code could be changed so that we don?t have to > specify ROI in an argument. * > > *Brian, what is your opinion on this?***** > > ** ** > > If "var" is specified as the only objective, then the plug-in selected is > quadprog for solve.QP to minimize variance.**** > > *[Doug] I think we need to specify the solver independently of the use of > ?var?, with a default choice which could be solve.QP for now.* > > *I am not very confident about the quality of solve.QP. Maybe Brian > knows if there are any test-bed benchmarks on it? In addition* > > *solve.QP lacks the following: (a) The ability to provide an initial set > of weights, (b) Lack of a ?verbose =? (or ?trace =?) with default F.***** > > ** ** > > If "var" and "mean" are specified as objectives, then the plug-in selected > is quadprog for solve.QP to maximize quadratic utility.**** > > *[Doug] This is a way to go. But maybe we want to be more explicit and > clear by using ?quadutil? or ?qu? or some such?* > [Ross] Brian mentioned "templated" objective functions in an earlier email and I think this could be part of doing that and should be fairly easy to implement. > **** > > ** ** > > If "mean" is specified as the only objective, then the plug-in selected is > glpk for Rglpk_solve_LP to maximize return.**** > > *[Doug] Good***** > > ** ** > > If "CVaR" is specified as the only objective, then the plug-in selected is > glpk for Rglpk_solve_LP to minimize sample CVaR.**** > > *[Doug] Good. I would prefer to use the term ETL. This is just my > habit, but I must say that CVaR was a bad naming choice by R&S,* > > *the reason being that the literal translation ?conditional VaR? is not > what it is about. I guess in situations like this where some * > > *People like one name and some like another, we can use aliases???* > [Ross] I think this would be fairly easy to change in optimize.portfolio(). Currently the function only looks for "CVaR" as an objective name. The if statements could just be extend to look for "CVaR", "ETL" or "ES" as an objective name. > **** > > ** ** > > This is nice for the user because they don't need to know when to use > quadprog and vice versa for glpk.**** > > ** ** > > It appears that the reference manual may be out of date. I could easily > make a vignette with several examples about the functionality that > optimize_method="ROI" offers. I'd prefer to do this after we are all > satisfied how to set the portfolio object with constraints and objectives. > **** > > ** ** > > Regards,**** > > Ross Bennett**** > > ** ** > > ** ** > > On Fri, Jun 21, 2013 at 11:52 PM, Doug Martin > wrote:**** > > Brian and Ross (and anyone else who wishes to comment),**** > > **** > > Thanks for the late evening response. Good to allow both conventions wrt > the sign of risk. Will check the leading portfolio and risk products for > most frequent choice, and then we should go to with the majority. My > priors are that the plus sign will win.**** > > **** > > Much more importantly: I just started browsing the PortfolioAnalytics > help files manual, slowly starting to refresh my memory of what little I > understood last summer. Please see the attached version of August 26 (not > the latest I know, but almost the last if not the last the Hezky edited)), > where I have added a small number of comments here and there. See in > particular my comments on optimize.portfolio. It would be very helpful if > the unfinished business of making ROI invisible to the user is taken care > of, and soon we just have solve.QP and Rglpk_solve_LP as optimizer choice > arguments. Even before getting all the constraint object changes, because > then I could check running my solve.QP and Rglpk_solve_LP based examples > (Ross you are quite familiar with these) run via optimize.portfolio and box > constraints. Then come back to group constraints when they are implemented. > **** > > **** > > Will keep browsing the manual, and try to understand what is going on. > Brian, I don?t know if you (and Ross) can manage, but examples at the end > of each help file would be truly helpful (no pun intended).**** > > **** > > Thanks,**** > > Doug**** > > **** > > P.S. Brian, I hope you didn?t hear my phone call at 11:15 or so ? I > thought you were still live on email and then realized that almost 30 > minutes had passed and you had probably crashed. **** > > **** > > **** > > **** > > -----Original Message----- > From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. > Peterson**** > > Sent: Friday, June 21, 2013 8:55 PM > To: gsoc-porta at r-forge.wu-wien.ac.at > Subject: Re: [GSoC-PortA] Quick question on multipliers and > constrained_objective()**** > > **** > > On 06/21/2013 10:44 PM, Doug Martin wrote:**** > > > Small comment: a lot (if not most) literature on risk takes risk as a * > *** > > > positive number, so puts a minus sign on the VaR quantile, etc. I **** > > > previously noticed that PerformanceAnalytics takes risk as negative, *** > * > > > had the thought to change that.**** > > **** > > PerformanceAnalytics allows it either way, user's choice.**** > > **** > > Risk is risk of *loss*. Period.**** > > **** > > Now, the math is sometimes/often easier with a positive number, I admit, > but it's typical for risk reports on real portfolios to describe those > risks as negative numbers.**** > > **** > > So we allow either.**** > > **** > > **** > > Cheers,**** > > **** > > Brian**** > > **** > > **** > > --**** > > Brian G. Peterson**** > > http://braverock.com/brian/**** > > Ph: 773-459-4973**** > > IM: bgpbraverock**** > > _______________________________________________**** > > GSoC-PortA mailing list**** > > GSoC-PortA at lists.r-forge.r-project.org**** > > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta**** > > ** ** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 24 04:46:16 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Sun, 23 Jun 2013 21:46:16 -0500 Subject: [GSoC-PortA] Interface to specify portfolio object, add constraints, and add objectives Message-ID: All, I've recently added a vignette to demonstrate how to create a portfolio object, add constraints, and add objectives (links below). In addition to the vignette, I have also add a testing script with more examples of adding and/or updating constraints and objectives in the portfolio object. The vignette gives step-by-step examples and description of each component: - Creating the portfolio object - Adding constraints - Adding objectives With where I am currently at, I feel like we have a coherent framework that separates constraints from objectives and stores in a portfolio object. The interface to add constraints is pretty flexible and is very similar to the current way objectives are added and stored. I would really appreciate feedback on this new interface to specify portfolio objects and adding constraints and objectives. Once we are all satisfied with this interface I can start making changes for optimize.portfolio to accept a portfolio object. Thanks, Ross Bennett testing_portfolio_specification.R https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/sandbox/testing_portfolio_specification.R?view=markup&revision=2410&root=returnanalytics portfolio_vignette.Rnw https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/sandbox/portfolio_vignette.Rnw?view=markup&revision=2405&root=returnanalytics -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Mon Jun 24 05:49:22 2013 From: martinrd at comcast.net (Doug Martin) Date: Sun, 23 Jun 2013 20:49:22 -0700 Subject: [GSoC-PortA] Interface to specify portfolio object, add constraints, and add objectives In-Reply-To: References: Message-ID: <00a101ce708d$d0f415c0$72dc4140$@comcast.net> Ross, Great to see this! Either this evening or first thing in the morning, I'll build PortfolioAnalytics and run the vignette, and take a look. Doug From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Sunday, June 23, 2013 7:46 PM To: PortfolioAnalytics Subject: [GSoC-PortA] Interface to specify portfolio object, add constraints, and add objectives All, I've recently added a vignette to demonstrate how to create a portfolio object, add constraints, and add objectives (links below). In addition to the vignette, I have also add a testing script with more examples of adding and/or updating constraints and objectives in the portfolio object. The vignette gives step-by-step examples and description of each component: - Creating the portfolio object - Adding constraints - Adding objectives With where I am currently at, I feel like we have a coherent framework that separates constraints from objectives and stores in a portfolio object. The interface to add constraints is pretty flexible and is very similar to the current way objectives are added and stored. I would really appreciate feedback on this new interface to specify portfolio objects and adding constraints and objectives. Once we are all satisfied with this interface I can start making changes for optimize.portfolio to accept a portfolio object. Thanks, Ross Bennett testing_portfolio_specification.R https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/sandbox/ testing_portfolio_specification.R?view=markup &revision=2410&root=returnanalytics portfolio_vignette.Rnw https://r-forge.r-project.org/scm/viewvc.php/pkg/PortfolioAnalytics/sandbox/ portfolio_vignette.Rnw?view=markup &revision=2405&root=returnanalytics -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Mon Jun 24 16:02:23 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 24 Jun 2013 09:02:23 -0500 Subject: [GSoC-PortA] Question on quadratic constraint specification Message-ID: Hi All, I have a quick question on quadratic constraint specification. One of the bullet points under constraints is "provide quadratic constraint specification". I did some digging for the ROI solvers and found that quadprog and glpk only handle linear constraints. The ROI plugin for cplex can handle a quadratic constraint, but this is a commercial product so I am not sure if we want to go down that road and add support for the cplex plugin for optimize_method="ROI". Are there workarounds to handle quadratic constraints in quadprog or glpk? Is the intent of providing quadratic constraint specification to allow the user to specify or define any function? Or should we limit it to pre-defined constraint types and functions that I can add? A couple use cases for a quadratic constraint that I thought of are: - Diversification (Allows the user to specify a diversification limit as a constraint). - Volatility (Allows the user to specify a target volatility (or min or max) as a constraint. (e.g. see Peter Carl's email with a portfolio specific limit to target volatility of > 10%) - Other use cases for quadratic constraints? I can add constraint functions for diversification, volatility, etc. and then this could be implemented in constrained_objective or with the mapping function that I will work on in the next section. Thanks, Ross Bennett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Mon Jun 24 17:02:00 2013 From: brian at braverock.com (Brian G. Peterson) Date: Mon, 24 Jun 2013 10:02:00 -0500 Subject: [GSoC-PortA] Quick question on multipliers and constrained_objective() In-Reply-To: <006b01ce7071$b8727fd0$29577f70$@comcast.net> References: <51C51586.4030406@braverock.com> <002101ce6efa$cfb04c40$6f10e4c0$@comcast.net> <51C52027.7080601@braverock.com> <002801ce6f04$526a65e0$f73f31a0$@comcast.net> <006b01ce7071$b8727fd0$29577f70$@comcast.net> Message-ID: <51C85F68.6000106@braverock.com> On 06/23/2013 07:28 PM, Doug Martin wrote: > I just added a testing script to match your examples in your 1. theory > review weights constrained mvo v5.pdf slides (attached). See committed > revision 2402 with sandbox/testing_ROI_Martin.R. Currently, the user > just chooses optimize_method="ROI" and optimize.portfolio() takes care > of choosing the ROI plugin for solve.QP or Rglpk_solve_LP based on the > objectives specified. > > */[Doug] I guess you mean that rev. 2402 still requires use of the > argument optimize_method=?ROI?. I?m wondering if my earlier assumption > from last summer that the code could be changed so that we don?t have to > specify ROI in an argument. /* > > */Brian, what is your opinion on this?/* > optimize_method allows the user to specify any of the many solvers supported by PortfolioAnalytics. Over time, I expect then number of solvers to continue to increase, either through addition of new solvers to ROI (which I expect) or through addition of solvers directly to PortfolioAnalytics. If and when we can detect the most efficient/accurate solver for a particular combination of objectives and constraints, I'd suggest changing the default for optimize_method from 'random' to 'auto'. If we have time to get to this this summer, that would be fantastic. Otherwise, we can come back to it later in the year. Generally, I think that more exact solvers should be used when constraints/objectives line up to something that can be solved by a linear, conical, or quadratic solver. For global solvers, 'random' gives a good intuitive overview of the entire feasible space while 'DEoptim' will provide the most accurate answer in a complex constraint or multi-objective space. My personal testing of 'soma' and 'pso' left me underwhelmed, though it is possible that has more to do with the default settings being poor for portfolio optmization than with any inherent limitations. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Mon Jun 24 17:27:58 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 24 Jun 2013 08:27:58 -0700 Subject: [GSoC-PortA] Question on quadratic constraint specification In-Reply-To: References: Message-ID: <003701ce70ef$6866df30$39349d90$@comcast.net> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 24, 2013 7:02 AM To: PortfolioAnalytics Subject: [GSoC-PortA] Question on quadratic constraint specification Hi All, I have a quick question on quadratic constraint specification. One of the bullet points under constraints is "provide quadratic constraint specification". I did some digging for the ROI solvers and found that quadprog and glpk only handle linear constraints. The ROI plugin for cplex can handle a quadratic constraint, but this is a commercial product so I am not sure if we want to go down that road and add support for the cplex plugin for optimize_method="ROI". Are there workarounds to handle quadratic constraints in quadprog or glpk? [Doug] I have thought very about the desirability of a chapter on cplex in view of the extensive use of the product. But there is so much else to do that this probably won't happen this go around Is the intent of providing quadratic constraint specification to allow the user to specify or define any function? [Doug] A generic quadratic constraint capability would be ideal. But just adding the two you list below would be a good first step. With regard to the second below, I note that Jorion showed that adding an absolute volatility risk constraint to TEV portfolios improves their performance. Or should we limit it to pre-defined constraint types and functions that I can add? A couple use cases for a quadratic constraint that I thought of are: - Diversification (Allows the user to specify a diversification limit as a constraint). - Volatility (Allows the user to specify a target volatility (or min or max) as a constraint. (e.g. see Peter Carl's email with a portfolio specific limit to target volatility of > 10%) - Other use cases for quadratic constraints? I can add constraint functions for diversification, volatility, etc. and then this could be implemented in constrained_objective or with the mapping function that I will work on in the next section. Thanks, Ross Bennett -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossbennett34 at gmail.com Tue Jun 25 02:45:07 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Mon, 24 Jun 2013 19:45:07 -0500 Subject: [GSoC-PortA] Question on quadratic constraint specification In-Reply-To: <003701ce70ef$6866df30$39349d90$@comcast.net> References: <003701ce70ef$6866df30$39349d90$@comcast.net> Message-ID: See comments below On Mon, Jun 24, 2013 at 10:27 AM, Doug Martin wrote: > ** ** > > ** ** > > ** ** > > *From:* gsoc-porta-bounces at lists.r-forge.r-project.org [mailto: > gsoc-porta-bounces at lists.r-forge.r-project.org] *On Behalf Of *Ross > Bennett > *Sent:* Monday, June 24, 2013 7:02 AM > *To:* PortfolioAnalytics > *Subject:* [GSoC-PortA] Question on quadratic constraint specification**** > > ** ** > > Hi All,**** > > ** ** > > I have a quick question on quadratic constraint specification. One of the > bullet points under constraints is "provide quadratic constraint > specification". I did some digging for the ROI solvers and found that > quadprog and glpk only handle linear constraints. The ROI plugin for cplex > can handle a quadratic constraint, but this is a commercial product so I am > not sure if we want to go down that road and add support for the cplex > plugin for optimize_method="ROI". Are there workarounds to handle quadratic > constraints in quadprog or glpk?**** > > *[Doug] I have thought very about the desirability of a chapter on cplex > in view of the extensive use of the product. But there is so much else to > do that this probably won?t happen this go around* > [Ross] It looks like the IBM CPLEX suite is free for academic use. I might be able to work on this in the fall after GSOC is complete. **** > > ** ** > > Is the intent of providing quadratic constraint specification to allow the > user to specify or define any function?**** > > *[Doug] A generic quadratic constraint capability would be ideal. But > just adding the two you list below would be a good first step. With regard > to the second below, I note that Jorion showed that adding an absolute > volatility risk constraint to TEV portfolios improves their performance.* > [Ross] Sounds good, I'll work on adding constraint types for diversification and volatility. **** > > ** ** > > Or should we limit it to pre-defined constraint types and functions that I > can add?**** > > ** ** > > A couple use cases for a quadratic constraint that I thought of are:**** > > - Diversification (Allows the user to specify a diversification limit as a > constraint).**** > > - Volatility (Allows the user to specify a target volatility (or min or > max) as a constraint. (e.g. see Peter Carl's email with a portfolio > specific limit to target volatility of > 10%)**** > > - Other use cases for quadratic constraints?**** > > ** ** > > I can add constraint functions for diversification, volatility, etc. and > then this could be implemented in constrained_objective or with the mapping > function that I will work on in the next section.**** > > ** ** > > Thanks,**** > > Ross Bennett**** > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Tue Jun 25 03:02:46 2013 From: martinrd at comcast.net (Doug Martin) Date: Mon, 24 Jun 2013 18:02:46 -0700 Subject: [GSoC-PortA] Question on quadratic constraint specification In-Reply-To: References: <003701ce70ef$6866df30$39349d90$@comcast.net> Message-ID: <000e01ce713f$b52c70f0$1f8552d0$@comcast.net> From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 24, 2013 5:45 PM To: PortfolioAnalytics Subject: Re: [GSoC-PortA] Question on quadratic constraint specification See comments below On Mon, Jun 24, 2013 at 10:27 AM, Doug Martin wrote: From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Ross Bennett Sent: Monday, June 24, 2013 7:02 AM To: PortfolioAnalytics Subject: [GSoC-PortA] Question on quadratic constraint specification Hi All, I have a quick question on quadratic constraint specification. One of the bullet points under constraints is "provide quadratic constraint specification". I did some digging for the ROI solvers and found that quadprog and glpk only handle linear constraints. The ROI plugin for cplex can handle a quadratic constraint, but this is a commercial product so I am not sure if we want to go down that road and add support for the cplex plugin for optimize_method="ROI". Are there workarounds to handle quadratic constraints in quadprog or glpk? [Doug] I have thought very about the desirability of a chapter on cplex in view of the extensive use of the product. But there is so much else to do that this probably won't happen this go around [Ross] It looks like the IBM CPLEX suite is free for academic use. I might be able to work on this in the fall after GSOC is complete. [Doug] Well, that would be serious incentive. Is the intent of providing quadratic constraint specification to allow the user to specify or define any function? [Doug] A generic quadratic constraint capability would be ideal. But just adding the two you list below would be a good first step. With regard to the second below, I note that Jorion showed that adding an absolute volatility risk constraint to TEV portfolios improves their performance. [Ross] Sounds good, I'll work on adding constraint types for diversification and volatility. [Doug] I briefly describe the Jorion 2003 result in the appendix of LSS4 attached, and have attached the paper. We did something similar with ETL based optimization in FinAnalytica's Cognity product. Or should we limit it to pre-defined constraint types and functions that I can add? A couple use cases for a quadratic constraint that I thought of are: - Diversification (Allows the user to specify a diversification limit as a constraint). - Volatility (Allows the user to specify a target volatility (or min or max) as a constraint. (e.g. see Peter Carl's email with a portfolio specific limit to target volatility of > 10%) - Other use cases for quadratic constraints? I can add constraint functions for diversification, volatility, etc. and then this could be implemented in constrained_objective or with the mapping function that I will work on in the next section. Thanks, Ross Bennett _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 4. active portfolio management v3.pdf Type: application/pdf Size: 851193 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: jorion portopt with tev constraint 2003.pdf Type: application/pdf Size: 2238160 bytes Desc: not available URL: From brian at braverock.com Fri Jun 28 20:05:27 2013 From: brian at braverock.com (Brian G. Peterson) Date: Fri, 28 Jun 2013 13:05:27 -0500 Subject: [GSoC-PortA] min/max objectives Message-ID: <51CDD067.3020708@braverock.com> I saw commit r2463, and I thought I'd comment that something looks off. right now, with targets on objectives, we're trying to get as close to the target as possible. With min/max ranges for objectives, I think we would penalize if and only if the objective is outside the min or the max. if (!is.null(objective$min) & !is.null(objective$max)){ # we have a min and max out = out + penalty * objective$multiplier * ((tmp_measure - objective$max) + (objective$min - tmp_measure)) } This code seems to always penalize. I haven't run it, so I may be reading it wrong, but it seems that supporting this would require you to first test for being outside the range, and *then* penalize the objective if and only if outside the range. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From rossbennett34 at gmail.com Fri Jun 28 20:32:55 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Fri, 28 Jun 2013 13:32:55 -0500 Subject: [GSoC-PortA] min/max objectives In-Reply-To: <51CDD067.3020708@braverock.com> References: <51CDD067.3020708@braverock.com> Message-ID: Brian, Thanks for the feedback. Yes, it does always penalize. The past few days I have been trying to use the turnover objective as a maximum allowable turnover and only penalize if turnover > objective$max, but I was getting results I was not expecting... it seemed to penalize even if the target value was greater than the calculated value. I'll test this a few different ways to try to get it working Thanks, Ross On Fri, Jun 28, 2013 at 1:05 PM, Brian G. Peterson wrote: > I saw commit r2463, and I thought I'd comment that something looks off. > > right now, with targets on objectives, we're trying to get as close to the > target as possible. > > With min/max ranges for objectives, I think we would penalize if and only > if the objective is outside the min or the max. > > if (!is.null(objective$min) & !is.null(objective$max)){ > # we have a min and max > out = out + penalty * objective$multiplier * > ((tmp_measure - objective$max) + (objective$min - tmp_measure)) > } > > This code seems to always penalize. > > I haven't run it, so I may be reading it wrong, but it seems that > supporting this would require you to first test for being outside the > range, and *then* penalize the objective if and only if outside the range. > > Regards, > > Brian > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Fri Jun 28 20:35:34 2013 From: brian at braverock.com (Brian G. Peterson) Date: Fri, 28 Jun 2013 13:35:34 -0500 Subject: [GSoC-PortA] min/max objectives In-Reply-To: References: <51CDD067.3020708@braverock.com> Message-ID: <51CDD776.10706@braverock.com> On 06/28/2013 01:32 PM, Ross Bennett wrote: > Thanks for the feedback. Yes, it does always penalize. The past few days > I have been trying to use the turnover objective as a maximum allowable > turnover and only penalize if turnover > objective$max, but I was > getting results I was not expecting... it seemed to penalize even if the > target value was greater than the calculated value. I'll test this a few > different ways to try to get it working > If you're below a threshold, you should do nothing to change the objective at all. You're below the threshold, so it shouldn't change 'out'. -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From rossbennett34 at gmail.com Fri Jun 28 20:55:14 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Fri, 28 Jun 2013 13:55:14 -0500 Subject: [GSoC-PortA] min/max objectives In-Reply-To: <51CDD776.10706@braverock.com> References: <51CDD067.3020708@braverock.com> <51CDD776.10706@braverock.com> Message-ID: Thanks for the pointer. I've made the changes in commit r2466 to first check if min or max has been violated and only then apply a penalty to change out. Thanks again! Ross On Fri, Jun 28, 2013 at 1:35 PM, Brian G. Peterson wrote: > On 06/28/2013 01:32 PM, Ross Bennett wrote: > >> Thanks for the feedback. Yes, it does always penalize. The past few days >> I have been trying to use the turnover objective as a maximum allowable >> turnover and only penalize if turnover > objective$max, but I was >> getting results I was not expecting... it seemed to penalize even if the >> target value was greater than the calculated value. I'll test this a few >> different ways to try to get it working >> >> > If you're below a threshold, you should do nothing to change the objective > at all. You're below the threshold, so it shouldn't change 'out'. > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sat Jun 29 15:45:11 2013 From: brian at braverock.com (Brian G. Peterson) Date: Sat, 29 Jun 2013 08:45:11 -0500 Subject: [GSoC-PortA] mapping function Message-ID: <51CEE4E7.8010106@braverock.com> Based on side conversations with Ross and Peter, I thought I should talk a little bit about next steps related to the mapping function. Apologies for the long email, I want to be complete, and I hope that some of this can make its way to the documentation. The purpose of the mapping function is to transform a weights vector that does not meet all the constraints into a weights vector that does meet the constraints, if one exists, hopefully with a minimum of transformation. In the random portfolios code, we've used a couple of techniques pioneered by Pat Burns. The philosophical idea is that your optimum portfolio is most likely to exist at the edges of the feasible space. At the first R/Finance conference, Pat used the analogy of a mountain lake, where the lake represents the feasible space. With a combination of lots of different constraints, the shore of the lake will not be smooth or regular. The lake (the feasible space) may not take up a large percentage of the terrain. If we randomly place a rock anywhere in the terrain, some of them will land in the lake, inside the feasible space, but most will land outside, on the slopes of the mountains that surround the lake. The goal should be to nudge these towards the shores of the lake (our feasible space). Having exhausted the analogy, let's talk details. A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224 It is possible that can use this method directly for random portfolios (and that we could add the ectra constraint types to DEoptim). If so, much of the rest of what I'll write here is irrelevant. I strongly suspect that there will be some constraint types that will still need to be 'adjusted' via a mapping method like the one laid out below, since a stochastic solver will hand us a vector that needs to be transformed at least in part to move into the feasible space. It's alsom not entirely clear to me that the methods presented in the paper can satisfy all our constraint types. I think our first step should be to test each constraint type, in some sort of hierarchy, starting with box constraints (almost all solvers support box constraints, of course), since some of the other transformations will violate the box constraints, and we'll need to transform back again. Each constraint can be evaluated as a logical expression against the weights vector. You can see code for doing something similar with time series data in the sigFormula function in quantstrat. It takes advantage of some base R functionality that can treat an R object (in this case the weights vector) as an environment or 'frame'. This allows the columns of the data to be addressed without any major manipulation, simply by column name (asset name in the weights vector, possibly after adding names back in). The code looks something like this: eval(parse(text=formula), data) So, 'data' is our weights vector, and 'formula' is an expression that can be evaluated as a formula by R. Evaluating this formula will give us TRUE or FALSE to denote whether the weights vector is in compliance or in violation of that constraint. Then, we'll need to transform the weight vector, if possible, to comply with that constraint. Specific Cases: I've implemented this transformation for box constraints in the random portfolios code. We don't need the evaluation I'll describe next for box constraints, because each single weight is handled separately. min_sum and max_sum leverage constraints can be evaluated without using the formula, since the formula is simple, and can be expressed in simple R code. The transformation can be accomplished by transforming the entire vector. There's code to do this in both the random portfolios code and in constrained_objective. It is probably preferable to do the transformation one weight at a time, as I do in the random portfolios code, to end closer to the edges of the feasible space, while continuing to take the box constraints into account. linear (in)equality constraints and group constraints can be evaluated generically via the formula method I've described above. Then individual weights can be transformed taking the value of the constraint (<,>,=) into account (along with the box constraints and leverage constraints). and so on... Challenges: - recovering the transformed vector from a optimization solver that doesn't directly support a mapping function. I've got some tricks for this using environments that we can revisit after we get the basic methodology working. -allowing for progressively relaxing constraints when the constraints are simply too restrictive. Perhaps Doug has some documentation on this as he's done it in the past, or perhaps we can simply deal with it in the penalty part of constrained_objective() Hopefully this was helpful. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Sat Jun 29 18:34:06 2013 From: martinrd at comcast.net (Doug Martin) Date: Sat, 29 Jun 2013 09:34:06 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CEE4E7.8010106@braverock.com> References: <51CEE4E7.8010106@braverock.com> Message-ID: <002401ce74e6$860b9850$9222c8f0$@comcast.net> I'll eventually catch up on this stuff (sounds like you should call it "exterior point methods"). Meanwhile, do either of you have an electronic version of the following paper by Burns? Burns, P., 2007, Random Portfolios for Performance Measurement, in Erricos John Kontoghiorghes & Cristian Gatu eds.: Optimization, Econometric and Financial Analysis (Springer). -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 6:45 AM To: PortfolioAnalytics Subject: [GSoC-PortA] mapping function Based on side conversations with Ross and Peter, I thought I should talk a little bit about next steps related to the mapping function. Apologies for the long email, I want to be complete, and I hope that some of this can make its way to the documentation. The purpose of the mapping function is to transform a weights vector that does not meet all the constraints into a weights vector that does meet the constraints, if one exists, hopefully with a minimum of transformation. In the random portfolios code, we've used a couple of techniques pioneered by Pat Burns. The philosophical idea is that your optimum portfolio is most likely to exist at the edges of the feasible space. At the first R/Finance conference, Pat used the analogy of a mountain lake, where the lake represents the feasible space. With a combination of lots of different constraints, the shore of the lake will not be smooth or regular. The lake (the feasible space) may not take up a large percentage of the terrain. If we randomly place a rock anywhere in the terrain, some of them will land in the lake, inside the feasible space, but most will land outside, on the slopes of the mountains that surround the lake. The goal should be to nudge these towards the shores of the lake (our feasible space). Having exhausted the analogy, let's talk details. A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224 It is possible that can use this method directly for random portfolios (and that we could add the ectra constraint types to DEoptim). If so, much of the rest of what I'll write here is irrelevant. I strongly suspect that there will be some constraint types that will still need to be 'adjusted' via a mapping method like the one laid out below, since a stochastic solver will hand us a vector that needs to be transformed at least in part to move into the feasible space. It's alsom not entirely clear to me that the methods presented in the paper can satisfy all our constraint types. I think our first step should be to test each constraint type, in some sort of hierarchy, starting with box constraints (almost all solvers support box constraints, of course), since some of the other transformations will violate the box constraints, and we'll need to transform back again. Each constraint can be evaluated as a logical expression against the weights vector. You can see code for doing something similar with time series data in the sigFormula function in quantstrat. It takes advantage of some base R functionality that can treat an R object (in this case the weights vector) as an environment or 'frame'. This allows the columns of the data to be addressed without any major manipulation, simply by column name (asset name in the weights vector, possibly after adding names back in). The code looks something like this: eval(parse(text=formula), data) So, 'data' is our weights vector, and 'formula' is an expression that can be evaluated as a formula by R. Evaluating this formula will give us TRUE or FALSE to denote whether the weights vector is in compliance or in violation of that constraint. Then, we'll need to transform the weight vector, if possible, to comply with that constraint. Specific Cases: I've implemented this transformation for box constraints in the random portfolios code. We don't need the evaluation I'll describe next for box constraints, because each single weight is handled separately. min_sum and max_sum leverage constraints can be evaluated without using the formula, since the formula is simple, and can be expressed in simple R code. The transformation can be accomplished by transforming the entire vector. There's code to do this in both the random portfolios code and in constrained_objective. It is probably preferable to do the transformation one weight at a time, as I do in the random portfolios code, to end closer to the edges of the feasible space, while continuing to take the box constraints into account. linear (in)equality constraints and group constraints can be evaluated generically via the formula method I've described above. Then individual weights can be transformed taking the value of the constraint (<,>,=) into account (along with the box constraints and leverage constraints). and so on... Challenges: - recovering the transformed vector from a optimization solver that doesn't directly support a mapping function. I've got some tricks for this using environments that we can revisit after we get the basic methodology working. -allowing for progressively relaxing constraints when the constraints are simply too restrictive. Perhaps Doug has some documentation on this as he's done it in the past, or perhaps we can simply deal with it in the penalty part of constrained_objective() Hopefully this was helpful. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Sat Jun 29 18:34:06 2013 From: martinrd at comcast.net (Doug Martin) Date: Sat, 29 Jun 2013 09:34:06 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CEE4E7.8010106@braverock.com> References: <51CEE4E7.8010106@braverock.com> Message-ID: <002401ce74e6$860b9850$9222c8f0$@comcast.net> I'll eventually catch up on this stuff (sounds like you should call it "exterior point methods"). Meanwhile, do either of you have an electronic version of the following paper by Burns? Burns, P., 2007, Random Portfolios for Performance Measurement, in Erricos John Kontoghiorghes & Cristian Gatu eds.: Optimization, Econometric and Financial Analysis (Springer). -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 6:45 AM To: PortfolioAnalytics Subject: [GSoC-PortA] mapping function Based on side conversations with Ross and Peter, I thought I should talk a little bit about next steps related to the mapping function. Apologies for the long email, I want to be complete, and I hope that some of this can make its way to the documentation. The purpose of the mapping function is to transform a weights vector that does not meet all the constraints into a weights vector that does meet the constraints, if one exists, hopefully with a minimum of transformation. In the random portfolios code, we've used a couple of techniques pioneered by Pat Burns. The philosophical idea is that your optimum portfolio is most likely to exist at the edges of the feasible space. At the first R/Finance conference, Pat used the analogy of a mountain lake, where the lake represents the feasible space. With a combination of lots of different constraints, the shore of the lake will not be smooth or regular. The lake (the feasible space) may not take up a large percentage of the terrain. If we randomly place a rock anywhere in the terrain, some of them will land in the lake, inside the feasible space, but most will land outside, on the slopes of the mountains that surround the lake. The goal should be to nudge these towards the shores of the lake (our feasible space). Having exhausted the analogy, let's talk details. A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224 It is possible that can use this method directly for random portfolios (and that we could add the ectra constraint types to DEoptim). If so, much of the rest of what I'll write here is irrelevant. I strongly suspect that there will be some constraint types that will still need to be 'adjusted' via a mapping method like the one laid out below, since a stochastic solver will hand us a vector that needs to be transformed at least in part to move into the feasible space. It's alsom not entirely clear to me that the methods presented in the paper can satisfy all our constraint types. I think our first step should be to test each constraint type, in some sort of hierarchy, starting with box constraints (almost all solvers support box constraints, of course), since some of the other transformations will violate the box constraints, and we'll need to transform back again. Each constraint can be evaluated as a logical expression against the weights vector. You can see code for doing something similar with time series data in the sigFormula function in quantstrat. It takes advantage of some base R functionality that can treat an R object (in this case the weights vector) as an environment or 'frame'. This allows the columns of the data to be addressed without any major manipulation, simply by column name (asset name in the weights vector, possibly after adding names back in). The code looks something like this: eval(parse(text=formula), data) So, 'data' is our weights vector, and 'formula' is an expression that can be evaluated as a formula by R. Evaluating this formula will give us TRUE or FALSE to denote whether the weights vector is in compliance or in violation of that constraint. Then, we'll need to transform the weight vector, if possible, to comply with that constraint. Specific Cases: I've implemented this transformation for box constraints in the random portfolios code. We don't need the evaluation I'll describe next for box constraints, because each single weight is handled separately. min_sum and max_sum leverage constraints can be evaluated without using the formula, since the formula is simple, and can be expressed in simple R code. The transformation can be accomplished by transforming the entire vector. There's code to do this in both the random portfolios code and in constrained_objective. It is probably preferable to do the transformation one weight at a time, as I do in the random portfolios code, to end closer to the edges of the feasible space, while continuing to take the box constraints into account. linear (in)equality constraints and group constraints can be evaluated generically via the formula method I've described above. Then individual weights can be transformed taking the value of the constraint (<,>,=) into account (along with the box constraints and leverage constraints). and so on... Challenges: - recovering the transformed vector from a optimization solver that doesn't directly support a mapping function. I've got some tricks for this using environments that we can revisit after we get the basic methodology working. -allowing for progressively relaxing constraints when the constraints are simply too restrictive. Perhaps Doug has some documentation on this as he's done it in the past, or perhaps we can simply deal with it in the penalty part of constrained_objective() Hopefully this was helpful. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sat Jun 29 21:14:26 2013 From: brian at braverock.com (Brian G. Peterson) Date: Sat, 29 Jun 2013 14:14:26 -0500 Subject: [GSoC-PortA] mapping function In-Reply-To: <002401ce74e6$860b9850$9222c8f0$@comcast.net> References: <51CEE4E7.8010106@braverock.com> <002401ce74e6$860b9850$9222c8f0$@comcast.net> Message-ID: <51CF3212.9010703@braverock.com> On 06/29/2013 11:34 AM, Doug Martin wrote: > I'll eventually catch up on this stuff (sounds like you should call it > "exterior point methods"). Meanwhile, do either of you have an > electronic version of the following paper by Burns? > > /Burns, P., 2007, Random Portfolios for Performance Measurement, in > Erricos John Kontoghiorghes & Cristian Gatu eds.: Optimization, > Econometric and Financial Analysis (Springer)./ > the working paper for that one is here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=630123 But the date on the book you cite is 2007, and I don't know what Pat changed, but the abstract and first page are nearly identical. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Sat Jun 29 21:36:04 2013 From: martinrd at comcast.net (Doug Martin) Date: Sat, 29 Jun 2013 12:36:04 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CF3212.9010703@braverock.com> References: <51CEE4E7.8010106@braverock.com> <002401ce74e6$860b9850$9222c8f0$@comcast.net> <51CF3212.9010703@braverock.com> Message-ID: <000301ce74ff$f1d4ca20$d57e5e60$@comcast.net> Thanks!! Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 12:14 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] mapping function On 06/29/2013 11:34 AM, Doug Martin wrote: > I'll eventually catch up on this stuff (sounds like you should call it > "exterior point methods"). Meanwhile, do either of you have an > electronic version of the following paper by Burns? > > /Burns, P., 2007, Random Portfolios for Performance Measurement, in > Erricos John Kontoghiorghes & Cristian Gatu eds.: Optimization, > Econometric and Financial Analysis (Springer)./ > the working paper for that one is here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=630123 But the date on the book you cite is 2007, and I don't know what Pat changed, but the abstract and first page are nearly identical. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From martinrd at comcast.net Sat Jun 29 21:36:04 2013 From: martinrd at comcast.net (Doug Martin) Date: Sat, 29 Jun 2013 12:36:04 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CF3212.9010703@braverock.com> References: <51CEE4E7.8010106@braverock.com> <002401ce74e6$860b9850$9222c8f0$@comcast.net> <51CF3212.9010703@braverock.com> Message-ID: <000301ce74ff$f1d4ca20$d57e5e60$@comcast.net> Thanks!! Doug -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 12:14 PM To: gsoc-porta at r-forge.wu-wien.ac.at Subject: Re: [GSoC-PortA] mapping function On 06/29/2013 11:34 AM, Doug Martin wrote: > I'll eventually catch up on this stuff (sounds like you should call it > "exterior point methods"). Meanwhile, do either of you have an > electronic version of the following paper by Burns? > > /Burns, P., 2007, Random Portfolios for Performance Measurement, in > Erricos John Kontoghiorghes & Cristian Gatu eds.: Optimization, > Econometric and Financial Analysis (Springer)./ > the working paper for that one is here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=630123 But the date on the book you cite is 2007, and I don't know what Pat changed, but the abstract and first page are nearly identical. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta From rossbennett34 at gmail.com Sat Jun 29 22:32:08 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Sat, 29 Jun 2013 15:32:08 -0500 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CEE4E7.8010106@braverock.com> References: <51CEE4E7.8010106@braverock.com> Message-ID: Brian, Thanks for sharing the paper and your ideas for the mapping function. One of the main point I got from the methodology described in the paper is that the sets of weights are omitted instead of transformed if they do not meet the constraints. - generate random portfolios - test each set of portfolio weights - if a constraint is violated, omit that set of weights - compute the objective function on each remaining set of weights - select the set of optimal weights Is this what you were getting at here? "A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/**papers.cfm?abstract_id=1680224 It is possible that can use this method directly for random portfolios." I could add a block of code at the end of random_portfolios that tests each set of weights against the constraints and only keeps the weights that satisfy the constraints. Thoughts? If we have to transform the weights, here are my thoughts looking at the specific cases for constraint types Constraint Type - Leverage (min_sum and max_sum) - This is done in randomize_portfolio by randomly permuting and increasing or decreasing an individual element (asset weight) until min_sum and max_sum constraints are satisfied while taking into account box constraints. These constraints are satisfied based on the way random portfolios are constructed. - This is done in constrained_objective by transforming the entire vector ( - if(sum(w)>max_sum) { w<-(max_sum/sum(w))*w } # normalize to max_sum - if(sum(w)wrote: > Based on side conversations with Ross and Peter, I thought I should talk a > little bit about next steps related to the mapping function. > > Apologies for the long email, I want to be complete, and I hope that some > of this can make its way to the documentation. > > The purpose of the mapping function is to transform a weights vector that > does not meet all the constraints into a weights vector that does meet the > constraints, if one exists, hopefully with a minimum of transformation. > > In the random portfolios code, we've used a couple of techniques pioneered > by Pat Burns. The philosophical idea is that your optimum portfolio is > most likely to exist at the edges of the feasible space. > > At the first R/Finance conference, Pat used the analogy of a mountain > lake, where the lake represents the feasible space. With a combination of > lots of different constraints, the shore of the lake will not be smooth or > regular. The lake (the feasible space) may not take up a large percentage > of the terrain. > > If we randomly place a rock anywhere in the terrain, some of them will > land in the lake, inside the feasible space, but most will land outside, on > the slopes of the mountains that surround the lake. The goal should be to > nudge these towards the shores of the lake (our feasible space). > > Having exhausted the analogy, let's talk details. > > A slightly more rigorous treatment of the problem is given here: > http://papers.ssrn.com/sol3/**papers.cfm?abstract_id=1680224 > It is possible that can use this method directly for random portfolios > (and that we could add the ectra constraint types to DEoptim). If so, much > of the rest of what I'll write here is irrelevant. I strongly suspect that > there will be some constraint types that will still need to be 'adjusted' > via a mapping method like the one laid out below, since a stochastic solver > will hand us a vector that needs to be transformed at least in part to move > into the feasible space. It's alsom not entirely clear to me that the > methods presented in the paper can satisfy all our constraint types. > > > I think our first step should be to test each constraint type, in some > sort of hierarchy, starting with box constraints (almost all solvers > support box constraints, of course), since some of the other > transformations will violate the box constraints, and we'll need to > transform back again. > > Each constraint can be evaluated as a logical expression against the > weights vector. You can see code for doing something similar with time > series data in the sigFormula function in quantstrat. It takes advantage of > some base R functionality that can treat an R object (in this case the > weights vector) as an environment or 'frame'. This allows the columns of > the data to be addressed without any major manipulation, simply by column > name (asset name in the weights vector, possibly after adding names back > in). > > The code looks something like this: > eval(parse(text=formula), data) > > So, 'data' is our weights vector, and 'formula' is an expression that can > be evaluated as a formula by R. Evaluating this formula will give us TRUE > or FALSE to denote whether the weights vector is in compliance or in > violation of that constraint. Then, we'll need to transform the weight > vector, if possible, to comply with that constraint. > > Specific Cases: > I've implemented this transformation for box constraints in the random > portfolios code. We don't need the evaluation I'll describe next for box > constraints, because each single weight is handled separately. > > min_sum and max_sum leverage constraints can be evaluated without using > the formula, since the formula is simple, and can be expressed in simple R > code. The transformation can be accomplished by transforming the entire > vector. There's code to do this in both the random portfolios code and in > constrained_objective. It is probably preferable to do the transformation > one weight at a time, as I do in the random portfolios code, to end closer > to the edges of the feasible space, while continuing to take the box > constraints into account. > > linear (in)equality constraints and group constraints can be evaluated > generically via the formula method I've described above. Then individual > weights can be transformed taking the value of the constraint (<,>,=) into > account (along with the box constraints and leverage constraints). > > and so on... > > Challenges: > - recovering the transformed vector from a optimization solver that > doesn't directly support a mapping function. I've got some tricks for this > using environments that we can revisit after we get the basic methodology > working. > > -allowing for progressively relaxing constraints when the constraints are > simply too restrictive. Perhaps Doug has some documentation on this as > he's done it in the past, or perhaps we can simply deal with it in the > penalty part of constrained_objective() > > Hopefully this was helpful. > > Regards, > > Brian > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sat Jun 29 23:13:36 2013 From: brian at braverock.com (Brian G. Peterson) Date: Sat, 29 Jun 2013 16:13:36 -0500 Subject: [GSoC-PortA] mapping function In-Reply-To: References: <51CEE4E7.8010106@braverock.com> Message-ID: <51CF4E00.20508@braverock.com> You're absolutely on the right track, except for this one part: * if a constraint is violated, omit that set of weights Depending on your constraints, your chance of constructing a valid portfolio may be very very slim. In random portfolios, we could try this, just keep drawing until you get ones that match, but this could take forever. In stochastic solvers, you typically only have support for box constraints, so again your chance of having truly feasible vectors is pretty small in many cases. This leads to the need for a transformation function, to bring a random/stochastic weights vector into compliance, or close, with all/most of your constraints. In DEoptim, we added the possibility of fnMap to be called directly on each generation's population, to 'fix' the random population and give it a full population. In PortfolioAnalytics, we can support an 'fnMap'-like transform directly in DEoptim and in random portfolios. We can support it *indirectly* for any arbitrary solver inside constrained_objective (as long as we can store the transformed weight vector to know what the objective was actually calculated on, which we can get to later). So i think we still need to write a fnMap transform function. some other notes: You're correct that volatility is usually dealt with as an objective. In fact, since it requires returns, I only think of this as an objective. Diversification I need to look at and see what you're calling 'diversification' to see if that requires the returns, or if it is just a measure of *weights* concentration, that word can mean either. Turnover can be handled either as an objective or as a constraint. Penalization is likely necessary in constrained_objective when we have to relax constraints somewhat to get to anything feasible, hopefully Doug can provide more guidance here, as the literature on relaxing constraints is pretty thin, to the best of my knowledge. Regards, Brian On 06/29/2013 03:32 PM, Ross Bennett wrote: > Brian, > > Thanks for sharing the paper and your ideas for the mapping function. > > One of the main point I got from the methodology described in the paper > is that the sets of weights are omitted instead of transformed if they > do not meet the constraints. > > * generate random portfolios > * test each set of portfolio weights > * if a constraint is violated, omit that set of weights > * compute the objective function on each remaining set of weights > * select the set of optimal weights > > Is this what you were getting at here? > "A slightly more rigorous treatment of the problem is given here: > http://papers.ssrn.com/sol3/__papers.cfm?abstract_id=1680224 > > It is possible that can use this method directly for random portfolios." > > I could add a block of code at the end of random_portfolios that tests > each set of weights against the constraints and only keeps the weights > that satisfy the constraints. Thoughts? > > If we have to transform the weights, here are my thoughts looking at the > specific cases for constraint types > Constraint Type > > * Leverage (min_sum and max_sum) > o This is done in randomize_portfolio by randomly permuting and > increasing or decreasing an individual element (asset weight) > until min_sum and max_sum constraints are satisfied while taking > into account box constraints. These constraints are satisfied > based on the way random portfolios are constructed. > o This is done in constrained_objective by transforming the entire > vector ( > + if(sum(w)>max_sum) { w<-(max_sum/sum(w))*w } # normalize to > max_sum > + if(sum(w) min_sum > o Implement by moving this into the mapping function... correct? > * Box (min and max) > o This is done in randomize_portfolio by construction > o This is done in constrained_objective by penalizing weights > outside box constraints. > o Implement by using logic from randomize_portfolio to transform > weights instead of penalizing them. Goal is to transform the > weights vector instead of penalize... correct? > * Group (cLO and cUP) > o One approach is to normalize the weights in each given group > that violate cLO or cUP so that the group weights sum to cLO or > cUP. This changes the sum of weights, so when the weights vector > is normalized the group constraints will likely be violated, but > it gets us close. See sandbox/testing_constrained_group.R > o Another approach is to add this to randomize_portfolio so the > group constraints as well as box and leverage are satisfied by > construction. Need to spend more time understanding code in > randomize_portfolio to see how feasible this is. > * turnover > o Could we include this in constrained_objective as a penalty? > * diversification > o Could we include this in constrained_objective as a penalty? > * volatility > o Could we include this in constrained_objective as a penalty? > * position_limit > o This may be able to be implemented in randomize_portfolio by > generating portfolios with the number of non-zero weights equal > to max.pos, then fill in weights of zero so the length of the > weights vector is equal to the number of assets, then scramble > the weights vector. The number of non-zero weights could also be > random so that the number of non-zero weights is not always > equal to max.pos. This could be implemented in the DEoptim > solver with the mapping function. This might be do-able in Rglpk > for max return and min ETL. Rglpk supports mixed integer types, > but solve.QP does not. May be able to use branch-and-bound > technique using solve.QP, but needs more research. > > > Regards, > Ross > > > On Sat, Jun 29, 2013 at 8:45 AM, Brian G. Peterson > wrote: > > Based on side conversations with Ross and Peter, I thought I should > talk a little bit about next steps related to the mapping function. > > Apologies for the long email, I want to be complete, and I hope that > some of this can make its way to the documentation. > > The purpose of the mapping function is to transform a weights vector > that does not meet all the constraints into a weights vector that > does meet the constraints, if one exists, hopefully with a minimum > of transformation. > > In the random portfolios code, we've used a couple of techniques > pioneered by Pat Burns. The philosophical idea is that your optimum > portfolio is most likely to exist at the edges of the feasible space. > > At the first R/Finance conference, Pat used the analogy of a > mountain lake, where the lake represents the feasible space. With a > combination of lots of different constraints, the shore of the lake > will not be smooth or regular. The lake (the feasible space) may > not take up a large percentage of the terrain. > > If we randomly place a rock anywhere in the terrain, some of them > will land in the lake, inside the feasible space, but most will land > outside, on the slopes of the mountains that surround the lake. The > goal should be to nudge these towards the shores of the lake (our > feasible space). > > Having exhausted the analogy, let's talk details. > > A slightly more rigorous treatment of the problem is given here: > http://papers.ssrn.com/sol3/__papers.cfm?abstract_id=1680224 > > It is possible that can use this method directly for random > portfolios (and that we could add the ectra constraint types to > DEoptim). If so, much of the rest of what I'll write here is > irrelevant. I strongly suspect that there will be some constraint > types that will still need to be 'adjusted' via a mapping method > like the one laid out below, since a stochastic solver will hand us > a vector that needs to be transformed at least in part to move into > the feasible space. It's alsom not entirely clear to me that the > methods presented in the paper can satisfy all our constraint types. > > > I think our first step should be to test each constraint type, in > some sort of hierarchy, starting with box constraints (almost all > solvers support box constraints, of course), since some of the other > transformations will violate the box constraints, and we'll need to > transform back again. > > Each constraint can be evaluated as a logical expression against the > weights vector. You can see code for doing something similar with > time series data in the sigFormula function in quantstrat. It takes > advantage of some base R functionality that can treat an R object > (in this case the weights vector) as an environment or 'frame'. This > allows the columns of the data to be addressed without any major > manipulation, simply by column name (asset name in the weights > vector, possibly after adding names back in). > > The code looks something like this: > eval(parse(text=formula), data) > > So, 'data' is our weights vector, and 'formula' is an expression > that can be evaluated as a formula by R. Evaluating this formula > will give us TRUE or FALSE to denote whether the weights vector is > in compliance or in violation of that constraint. Then, we'll need > to transform the weight vector, if possible, to comply with that > constraint. > > Specific Cases: > I've implemented this transformation for box constraints in the > random portfolios code. We don't need the evaluation I'll describe > next for box constraints, because each single weight is handled > separately. > > min_sum and max_sum leverage constraints can be evaluated without > using the formula, since the formula is simple, and can be expressed > in simple R code. The transformation can be accomplished by > transforming the entire vector. There's code to do this in both the > random portfolios code and in constrained_objective. It is probably > preferable to do the transformation one weight at a time, as I do in > the random portfolios code, to end closer to the edges of the > feasible space, while continuing to take the box constraints into > account. > > linear (in)equality constraints and group constraints can be > evaluated generically via the formula method I've described above. > Then individual weights can be transformed taking the value of the > constraint (<,>,=) into account (along with the box constraints and > leverage constraints). > > and so on... > > Challenges: > - recovering the transformed vector from a optimization solver that > doesn't directly support a mapping function. I've got some tricks > for this using environments that we can revisit after we get the > basic methodology working. > > -allowing for progressively relaxing constraints when the > constraints are simply too restrictive. Perhaps Doug has some > documentation on this as he's done it in the past, or perhaps we can > simply deal with it in the penalty part of constrained_objective() > > Hopefully this was helpful. > > Regards, > > Brian > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > _________________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-__project.org > > http://lists.r-forge.r-__project.org/cgi-bin/mailman/__listinfo/gsoc-porta > > > > > > _______________________________________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-project.org > http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta > -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From rossbennett34 at gmail.com Sun Jun 30 00:05:53 2013 From: rossbennett34 at gmail.com (Ross Bennett) Date: Sat, 29 Jun 2013 17:05:53 -0500 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CF4E00.20508@braverock.com> References: <51CEE4E7.8010106@braverock.com> <51CF4E00.20508@braverock.com> Message-ID: See comments/questions below. Ross On Sat, Jun 29, 2013 at 4:13 PM, Brian G. Peterson wrote: > You're absolutely on the right track, except for this one part: > * if a constraint is violated, omit that set of weights > > Depending on your constraints, your chance of constructing a valid > portfolio may be very very slim. > > In random portfolios, we could try this, just keep drawing until you get > ones that match, but this could take forever. > > In stochastic solvers, you typically only have support for box > constraints, so again your chance of having truly feasible vectors is > pretty small in many cases. > > This leads to the need for a transformation function, to bring a > random/stochastic weights vector into compliance, or close, with all/most > of your constraints. > > In DEoptim, we added the possibility of fnMap to be called directly on > each generation's population, to 'fix' the random population and give it a > full population. > > In PortfolioAnalytics, we can support an 'fnMap'-like transform directly > in DEoptim and in random portfolios. > [Ross] Would the while loops in randomize_portfolio have to be modified for additional constraints or is it simple enough to apply the fnMap function at the end of the function to modify the portfolio that is returned? > We can support it *indirectly* for any arbitrary solver inside > constrained_objective (as long as we can store the transformed weight > vector to know what the objective was actually calculated on, which we can > get to later). > > So i think we still need to write a fnMap transform function. > > some other notes: > You're correct that volatility is usually dealt with as an objective. In > fact, since it requires returns, I only think of this as an objective. > [Ross] I added target volatility as a constraint based on the email from Peter Carl, but planned to implement it like an objective in constrained_objective. Is this ok or should I remove volatility as a constraint type? > Diversification I need to look at and see what you're calling > 'diversification' to see if that requires the returns, or if it is just a > measure of *weights* concentration, that word can mean either. > [Ross] I am calculating diversification as just a measure of weights concentration. diversification <- function(weights){ return(1 - sum(weights^2)) } > Turnover can be handled either as an objective or as a constraint. > [Ross] Great, I added support for both. > > Penalization is likely necessary in constrained_objective when we have to > relax constraints somewhat to get to anything feasible, hopefully Doug can > provide more guidance here, as the literature on relaxing constraints is > pretty thin, to the best of my knowledge. > > Regards, > > Brian > > > On 06/29/2013 03:32 PM, Ross Bennett wrote: > >> Brian, >> >> Thanks for sharing the paper and your ideas for the mapping function. >> >> One of the main point I got from the methodology described in the paper >> is that the sets of weights are omitted instead of transformed if they >> do not meet the constraints. >> >> * generate random portfolios >> * test each set of portfolio weights >> * if a constraint is violated, omit that set of weights >> * compute the objective function on each remaining set of weights >> * select the set of optimal weights >> >> >> Is this what you were getting at here? >> "A slightly more rigorous treatment of the problem is given here: >> http://papers.ssrn.com/sol3/__**papers.cfm?abstract_id=1680224 >> >> >> **> >> It is possible that can use this method directly for random portfolios." >> >> I could add a block of code at the end of random_portfolios that tests >> each set of weights against the constraints and only keeps the weights >> that satisfy the constraints. Thoughts? >> >> If we have to transform the weights, here are my thoughts looking at the >> specific cases for constraint types >> Constraint Type >> >> * Leverage (min_sum and max_sum) >> o This is done in randomize_portfolio by randomly permuting and >> >> increasing or decreasing an individual element (asset weight) >> until min_sum and max_sum constraints are satisfied while taking >> into account box constraints. These constraints are satisfied >> based on the way random portfolios are constructed. >> o This is done in constrained_objective by transforming the entire >> vector ( >> + if(sum(w)>max_sum) { w<-(max_sum/sum(w))*w } # normalize to >> max_sum >> + if(sum(w)> min_sum >> o Implement by moving this into the mapping function... correct? >> * Box (min and max) >> o This is done in randomize_portfolio by construction >> o This is done in constrained_objective by penalizing weights >> outside box constraints. >> o Implement by using logic from randomize_portfolio to transform >> >> weights instead of penalizing them. Goal is to transform the >> weights vector instead of penalize... correct? >> * Group (cLO and cUP) >> o One approach is to normalize the weights in each given group >> >> that violate cLO or cUP so that the group weights sum to cLO or >> cUP. This changes the sum of weights, so when the weights vector >> is normalized the group constraints will likely be violated, but >> it gets us close. See sandbox/testing_constrained_**group.R >> o Another approach is to add this to randomize_portfolio so the >> >> group constraints as well as box and leverage are satisfied by >> construction. Need to spend more time understanding code in >> randomize_portfolio to see how feasible this is. >> * turnover >> o Could we include this in constrained_objective as a penalty? >> * diversification >> o Could we include this in constrained_objective as a penalty? >> * volatility >> o Could we include this in constrained_objective as a penalty? >> * position_limit >> o This may be able to be implemented in randomize_portfolio by >> >> generating portfolios with the number of non-zero weights equal >> to max.pos, then fill in weights of zero so the length of the >> weights vector is equal to the number of assets, then scramble >> the weights vector. The number of non-zero weights could also be >> random so that the number of non-zero weights is not always >> equal to max.pos. This could be implemented in the DEoptim >> solver with the mapping function. This might be do-able in Rglpk >> for max return and min ETL. Rglpk supports mixed integer types, >> but solve.QP does not. May be able to use branch-and-bound >> technique using solve.QP, but needs more research. >> >> >> Regards, >> Ross >> >> >> On Sat, Jun 29, 2013 at 8:45 AM, Brian G. Peterson > > wrote: >> >> Based on side conversations with Ross and Peter, I thought I should >> talk a little bit about next steps related to the mapping function. >> >> Apologies for the long email, I want to be complete, and I hope that >> some of this can make its way to the documentation. >> >> The purpose of the mapping function is to transform a weights vector >> that does not meet all the constraints into a weights vector that >> does meet the constraints, if one exists, hopefully with a minimum >> of transformation. >> >> In the random portfolios code, we've used a couple of techniques >> pioneered by Pat Burns. The philosophical idea is that your optimum >> portfolio is most likely to exist at the edges of the feasible space. >> >> At the first R/Finance conference, Pat used the analogy of a >> mountain lake, where the lake represents the feasible space. With a >> combination of lots of different constraints, the shore of the lake >> will not be smooth or regular. The lake (the feasible space) may >> not take up a large percentage of the terrain. >> >> If we randomly place a rock anywhere in the terrain, some of them >> will land in the lake, inside the feasible space, but most will land >> outside, on the slopes of the mountains that surround the lake. The >> goal should be to nudge these towards the shores of the lake (our >> feasible space). >> >> Having exhausted the analogy, let's talk details. >> >> A slightly more rigorous treatment of the problem is given here: >> http://papers.ssrn.com/sol3/__**papers.cfm?abstract_id=1680224 >> >> >> **> >> It is possible that can use this method directly for random >> portfolios (and that we could add the ectra constraint types to >> DEoptim). If so, much of the rest of what I'll write here is >> irrelevant. I strongly suspect that there will be some constraint >> types that will still need to be 'adjusted' via a mapping method >> like the one laid out below, since a stochastic solver will hand us >> a vector that needs to be transformed at least in part to move into >> the feasible space. It's alsom not entirely clear to me that the >> methods presented in the paper can satisfy all our constraint types. >> >> >> I think our first step should be to test each constraint type, in >> some sort of hierarchy, starting with box constraints (almost all >> solvers support box constraints, of course), since some of the other >> transformations will violate the box constraints, and we'll need to >> transform back again. >> >> Each constraint can be evaluated as a logical expression against the >> weights vector. You can see code for doing something similar with >> time series data in the sigFormula function in quantstrat. It takes >> advantage of some base R functionality that can treat an R object >> (in this case the weights vector) as an environment or 'frame'. This >> allows the columns of the data to be addressed without any major >> manipulation, simply by column name (asset name in the weights >> vector, possibly after adding names back in). >> >> The code looks something like this: >> eval(parse(text=formula), data) >> >> So, 'data' is our weights vector, and 'formula' is an expression >> that can be evaluated as a formula by R. Evaluating this formula >> will give us TRUE or FALSE to denote whether the weights vector is >> in compliance or in violation of that constraint. Then, we'll need >> to transform the weight vector, if possible, to comply with that >> constraint. >> >> Specific Cases: >> I've implemented this transformation for box constraints in the >> random portfolios code. We don't need the evaluation I'll describe >> next for box constraints, because each single weight is handled >> separately. >> >> min_sum and max_sum leverage constraints can be evaluated without >> using the formula, since the formula is simple, and can be expressed >> in simple R code. The transformation can be accomplished by >> transforming the entire vector. There's code to do this in both the >> random portfolios code and in constrained_objective. It is probably >> preferable to do the transformation one weight at a time, as I do in >> the random portfolios code, to end closer to the edges of the >> feasible space, while continuing to take the box constraints into >> account. >> >> linear (in)equality constraints and group constraints can be >> evaluated generically via the formula method I've described above. >> Then individual weights can be transformed taking the value of the >> constraint (<,>,=) into account (along with the box constraints and >> leverage constraints). >> >> and so on... >> >> Challenges: >> - recovering the transformed vector from a optimization solver that >> doesn't directly support a mapping function. I've got some tricks >> for this using environments that we can revisit after we get the >> basic methodology working. >> >> -allowing for progressively relaxing constraints when the >> constraints are simply too restrictive. Perhaps Doug has some >> documentation on this as he's done it in the past, or perhaps we can >> simply deal with it in the penalty part of constrained_objective() >> >> Hopefully this was helpful. >> >> Regards, >> >> Brian >> >> -- >> Brian G. Peterson >> http://braverock.com/brian/ >> Ph: 773-459-4973 >> IM: bgpbraverock >> ______________________________**___________________ >> GSoC-PortA mailing list >> GSoC-PortA at lists.r-forge.r-__**project.org >> >> > >> http://lists.r-forge.r-__**project.org/cgi-bin/mailman/__** >> listinfo/gsoc-porta >> > listinfo/gsoc-porta >> > >> >> >> >> >> >> ______________________________**_________________ >> GSoC-PortA mailing list >> GSoC-PortA at lists.r-forge.r-**project.org >> http://lists.r-forge.r-**project.org/cgi-bin/mailman/** >> listinfo/gsoc-porta >> >> > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > ______________________________**_________________ > GSoC-PortA mailing list > GSoC-PortA at lists.r-forge.r-**project.org > http://lists.r-forge.r-**project.org/cgi-bin/mailman/**listinfo/gsoc-porta > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sun Jun 30 03:12:45 2013 From: brian at braverock.com (Brian G. Peterson) Date: Sat, 29 Jun 2013 20:12:45 -0500 Subject: [GSoC-PortA] mapping function In-Reply-To: References: <51CEE4E7.8010106@braverock.com> <51CF4E00.20508@braverock.com> Message-ID: <51CF860D.20405@braverock.com> On 06/29/2013 05:05 PM, Ross Bennett wrote: > See comments/questions below. <...> > [Ross] Would the while loops in randomize_portfolio have to be modified > for additional constraints or is it simple enough to apply the fnMap > function at the end of the function to modify the portfolio that is > returned? Yes, we should be able to run it at the end of the function to modify the portfolio. Doug is correct that the type of transformations I'm suggesting are properly called 'exterior point' methods. There are a few papers on the topic, I'll try to review and see how many of them are relevant to what we need. <...> > [Ross] I added target volatility as a constraint based on the email from > Peter Carl, but planned to implement it like an objective in > constrained_objective. Is this ok or should I remove volatility as a > constraint type? I don't see volatility as a constraint, but rather as an objective, based on the distinction that I laid out before that a portfolio objective needs the returns of the assets to be calculated against. I think for consistency that we should restrict 'constraints' to things that may be decided only on the weight vector, without needing to run calculations on the returns. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock From martinrd at comcast.net Sun Jun 30 18:51:46 2013 From: martinrd at comcast.net (Doug Martin) Date: Sun, 30 Jun 2013 09:51:46 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CEE4E7.8010106@braverock.com> References: <51CEE4E7.8010106@braverock.com> Message-ID: <001801ce75b2$280c8260$78258720$@comcast.net> -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 6:45 AM To: PortfolioAnalytics Subject: [GSoC-PortA] mapping function Based on side conversations with Ross and Peter, I thought I should talk a little bit about next steps related to the mapping function. Apologies for the long email, I want to be complete, and I hope that some of this can make its way to the documentation. The purpose of the mapping function is to transform a weights vector that does not meet all the constraints into a weights vector that does meet the constraints, if one exists, hopefully with a minimum of transformation. In the random portfolios code, we've used a couple of techniques pioneered by Pat Burns. The philosophical idea is that your optimum portfolio is most likely to exist at the edges of the feasible space. At the first R/Finance conference, Pat used the analogy of a mountain lake, where the lake represents the feasible space. With a combination of lots of different constraints, the shore of the lake will not be smooth or regular. The lake (the feasible space) may not take up a large percentage of the terrain. If we randomly place a rock anywhere in the terrain, some of them will land in the lake, inside the feasible space, but most will land outside, on the slopes of the mountains that surround the lake. The goal should be to nudge these towards the shores of the lake (our feasible space). Having exhausted the analogy, let's talk details. A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224 [Doug] This is the 2010 paper, which I just read much of. Very nice paper. I find Burns papers pretty useless except for pointing out nice statistical aspect and promoting PortfolioProbe. For example in the paper you sent, he does not say clearly what he is doing in generating the random portfolios and what he means by out-of-sample. As you once mentioned, I guess you got most details through direct conversation with him. Then I found the new Shaw 2011 paper at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. It has some nice additional material, e.g., Section 4.3 on lower bounds and Section 4.4 on bounded shorting, among other. I still need to finish reading this version. In general if you accept random portfolios that violate the constraints, what is a good way to: (a) Accept only those within a certain distance, with appropriately defined metric, of the feasible regions (you don't want to consider all infeasible solutions - I guess this the reason for min.sum, max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to take a solution nearest to a vertex? It is possible that can use this method directly for random portfolios (and that we could add the ectra constraint types to DEoptim). If so, much of the rest of what I'll write here is irrelevant. I strongly suspect that there will be some constraint types that will still need to be 'adjusted' via a mapping method like the one laid out below, since a stochastic solver will hand us a vector that needs to be transformed at least in part to move into the feasible space. It's alsom not entirely clear to me that the methods presented in the paper can satisfy all our constraint types. I think our first step should be to test each constraint type, in some sort of hierarchy, starting with box constraints (almost all solvers support box constraints, of course), since some of the other transformations will violate the box constraints, and we'll need to transform back again. Each constraint can be evaluated as a logical expression against the weights vector. You can see code for doing something similar with time series data in the sigFormula function in quantstrat. It takes advantage of some base R functionality that can treat an R object (in this case the weights vector) as an environment or 'frame'. This allows the columns of the data to be addressed without any major manipulation, simply by column name (asset name in the weights vector, possibly after adding names back in). The code looks something like this: eval(parse(text=formula), data) So, 'data' is our weights vector, and 'formula' is an expression that can be evaluated as a formula by R. Evaluating this formula will give us TRUE or FALSE to denote whether the weights vector is in compliance or in violation of that constraint. Then, we'll need to transform the weight vector, if possible, to comply with that constraint. Specific Cases: I've implemented this transformation for box constraints in the random portfolios code. We don't need the evaluation I'll describe next for box constraints, because each single weight is handled separately. min_sum and max_sum leverage constraints can be evaluated without using the formula, since the formula is simple, and can be expressed in simple R code. The transformation can be accomplished by transforming the entire vector. There's code to do this in both the random portfolios code and in constrained_objective. It is probably preferable to do the transformation one weight at a time, as I do in the random portfolios code, to end closer to the edges of the feasible space, while continuing to take the box constraints into account. linear (in)equality constraints and group constraints can be evaluated generically via the formula method I've described above. Then individual weights can be transformed taking the value of the constraint (<,>,=) into account (along with the box constraints and leverage constraints). and so on... Challenges: - recovering the transformed vector from a optimization solver that doesn't directly support a mapping function. I've got some tricks for this using environments that we can revisit after we get the basic methodology working. -allowing for progressively relaxing constraints when the constraints are simply too restrictive. Perhaps Doug has some documentation on this as he's done it in the past, or perhaps we can simply deal with it in the penalty part of constrained_objective() Hopefully this was helpful. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrd at comcast.net Sun Jun 30 18:51:46 2013 From: martinrd at comcast.net (Doug Martin) Date: Sun, 30 Jun 2013 09:51:46 -0700 Subject: [GSoC-PortA] mapping function In-Reply-To: <51CEE4E7.8010106@braverock.com> References: <51CEE4E7.8010106@braverock.com> Message-ID: <001801ce75b2$280c8260$78258720$@comcast.net> -----Original Message----- From: gsoc-porta-bounces at lists.r-forge.r-project.org [mailto:gsoc-porta-bounces at lists.r-forge.r-project.org] On Behalf Of Brian G. Peterson Sent: Saturday, June 29, 2013 6:45 AM To: PortfolioAnalytics Subject: [GSoC-PortA] mapping function Based on side conversations with Ross and Peter, I thought I should talk a little bit about next steps related to the mapping function. Apologies for the long email, I want to be complete, and I hope that some of this can make its way to the documentation. The purpose of the mapping function is to transform a weights vector that does not meet all the constraints into a weights vector that does meet the constraints, if one exists, hopefully with a minimum of transformation. In the random portfolios code, we've used a couple of techniques pioneered by Pat Burns. The philosophical idea is that your optimum portfolio is most likely to exist at the edges of the feasible space. At the first R/Finance conference, Pat used the analogy of a mountain lake, where the lake represents the feasible space. With a combination of lots of different constraints, the shore of the lake will not be smooth or regular. The lake (the feasible space) may not take up a large percentage of the terrain. If we randomly place a rock anywhere in the terrain, some of them will land in the lake, inside the feasible space, but most will land outside, on the slopes of the mountains that surround the lake. The goal should be to nudge these towards the shores of the lake (our feasible space). Having exhausted the analogy, let's talk details. A slightly more rigorous treatment of the problem is given here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1680224 [Doug] This is the 2010 paper, which I just read much of. Very nice paper. I find Burns papers pretty useless except for pointing out nice statistical aspect and promoting PortfolioProbe. For example in the paper you sent, he does not say clearly what he is doing in generating the random portfolios and what he means by out-of-sample. As you once mentioned, I guess you got most details through direct conversation with him. Then I found the new Shaw 2011 paper at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1856476. It has some nice additional material, e.g., Section 4.3 on lower bounds and Section 4.4 on bounded shorting, among other. I still need to finish reading this version. In general if you accept random portfolios that violate the constraints, what is a good way to: (a) Accept only those within a certain distance, with appropriately defined metric, of the feasible regions (you don't want to consider all infeasible solutions - I guess this the reason for min.sum, max.sum, etc.?), and (b) Assuming an appropriate metric, does one want to take a solution nearest to a vertex? It is possible that can use this method directly for random portfolios (and that we could add the ectra constraint types to DEoptim). If so, much of the rest of what I'll write here is irrelevant. I strongly suspect that there will be some constraint types that will still need to be 'adjusted' via a mapping method like the one laid out below, since a stochastic solver will hand us a vector that needs to be transformed at least in part to move into the feasible space. It's alsom not entirely clear to me that the methods presented in the paper can satisfy all our constraint types. I think our first step should be to test each constraint type, in some sort of hierarchy, starting with box constraints (almost all solvers support box constraints, of course), since some of the other transformations will violate the box constraints, and we'll need to transform back again. Each constraint can be evaluated as a logical expression against the weights vector. You can see code for doing something similar with time series data in the sigFormula function in quantstrat. It takes advantage of some base R functionality that can treat an R object (in this case the weights vector) as an environment or 'frame'. This allows the columns of the data to be addressed without any major manipulation, simply by column name (asset name in the weights vector, possibly after adding names back in). The code looks something like this: eval(parse(text=formula), data) So, 'data' is our weights vector, and 'formula' is an expression that can be evaluated as a formula by R. Evaluating this formula will give us TRUE or FALSE to denote whether the weights vector is in compliance or in violation of that constraint. Then, we'll need to transform the weight vector, if possible, to comply with that constraint. Specific Cases: I've implemented this transformation for box constraints in the random portfolios code. We don't need the evaluation I'll describe next for box constraints, because each single weight is handled separately. min_sum and max_sum leverage constraints can be evaluated without using the formula, since the formula is simple, and can be expressed in simple R code. The transformation can be accomplished by transforming the entire vector. There's code to do this in both the random portfolios code and in constrained_objective. It is probably preferable to do the transformation one weight at a time, as I do in the random portfolios code, to end closer to the edges of the feasible space, while continuing to take the box constraints into account. linear (in)equality constraints and group constraints can be evaluated generically via the formula method I've described above. Then individual weights can be transformed taking the value of the constraint (<,>,=) into account (along with the box constraints and leverage constraints). and so on... Challenges: - recovering the transformed vector from a optimization solver that doesn't directly support a mapping function. I've got some tricks for this using environments that we can revisit after we get the basic methodology working. -allowing for progressively relaxing constraints when the constraints are simply too restrictive. Perhaps Doug has some documentation on this as he's done it in the past, or perhaps we can simply deal with it in the penalty part of constrained_objective() Hopefully this was helpful. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ GSoC-PortA mailing list GSoC-PortA at lists.r-forge.r-project.org http://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/gsoc-porta -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at braverock.com Sun Jun 30 20:07:45 2013 From: brian at braverock.com (Brian G. Peterson) Date: Sun, 30 Jun 2013 13:07:45 -0500 Subject: [GSoC-PortA] Interface to specify portfolio object, add constraints, and add objectives In-Reply-To: <003001ce75b7$6190d630$24b28290$@comcast.net> References: <00a101ce708d$d0f415c0$72dc4140$@comcast.net> <004901ce74ef$efca9580$cf5fc080$@comcast.net> <003001ce75b7$6190d630$24b28290$@comcast.net> Message-ID: <51D073F1.9080305@braverock.com> On 06/30/2013 12:29 PM, Doug Martin wrote: > I could add a type in the switch statement to specify full_investment, > active, or dollar_neutral) > > add.constraint(portfolio, type="full_investment") > > Which way do you think is more intuitive and easier to use? > > */[Doug] Option 2 is preferred, as it is simpler and any user will know > what it means./* > > *//* I agree. This is more intuitive, at the cost of extra documentation and examples. (min_sum and max_sum will still be available, of course, and exact full investment may not happen based on relaxation of constraints in any event) > */P.S. Brian and Guy, I notice that the style ?word1_word2? seems to be > the convention. I have always preferred ?word1.word2? for/* > > */easier typing, but have the impression that the use of period instead > of underscore is frowned upon ? e.g., because of difficulties it/* > > */causes in some programming languages?/* > In R specifically, the period is used for S3 method dispatch. I also prefer periods, but they're frowned upon by most of the R style guides unless you're using them as methods for some specific thing. In practice, I'm not sure it matters. R's style is so jumbled that either works, as long as we're as consistent as we can be. (file this with the preference for camelCase advocated by some, but not all, of the R core). Cheers, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock