Up vs Down: Changing Correlations in Portfolio Optimization
James Barber, Geb Berry, Marco Ongaro, George Rupp

Introduction
Data
Initial Findings for 65-country Sample
Predictive Regressions and Index Returns
US Directional Prediction
Set-up of Experiment
Results
Efficient Frontier
Scenarios
Conclusions
Further Research
Forecasting of Variances and Covariance with ARCH

Excel Program:
Markowitz Optimization
Data Period and Reset Buttons
Other additions and changes

Download Excel Engine        Download Excel Output File

Introduction

In this work we will attempt to assess the validity of the following hypothesis that the correlation between markets changes as a function of the general trends of the moment in dominant economies:

Dominant economies like the US will have differing correlation to other markets depending on the direction of its movement (e.g. a market with low correlation to the US in a bull market will show a higher correlation in a market downturn), the effect being: in a domestic bull market, returns of a diversified portfolio are dampened, while in a domestic bear market, due to higher correlation, the diversified portfolio returns are amplified in bear market. Using this underlying premise, we built a model to perform active asset allocation in a portfolio of 5 countries (Belgium, Hong Kong, Japan, US, Venezuela) using a Markowitz optimization with differing correlation variables depending on the predicted direction of the US market. We found that this recognition of changing correlation resulted in an improvement of between 250 and 800 basis points depending on the level of risk and short selling permitted.
 

Data

Initial Findings for 65-country Sample

Our initial analysis clearly reflects a difference in correlation between country dollar returns when the US market moves upwards versus when it moves downwards. This initial research took a cross-section of the indices for 65 countries from the MSCI and IFC data for developed, emerging and frontier markets, and then calculated a correlation table for all 65 markets. We then compared the covariance of the 65 markets with the US to confirm there was a basis for our thesis. Initially we found that the covariance in down markets was greater than the covariance in positive markets 85% of the time. Further analysis indicated that the markets which did not support this initial thesis were markets for which we had less than two years of data. Reworking this analysis we found 96% of the situations the "negative" covariance was greater than the "positive" covariance, with Zimbabwe and Pakistan being the only exceptions to the rule. The covariance matrices are based on the last 60 periods of data, conditional on positive and negative markets.
 

Click on image to view full size

Predictive Regressions and Index Returns

Our model used five asset classes for the internationally diversified portfolio. These assets were forecasted by predictive linear regressions for the equity indexes for five countries (US, Japan, Hong Kong, Belgium, Venezuela) from which we made our baseline portfolio. The five countries were chosen to give worldwide exposure to several regions with different levels of economic development. Return information for these five indexes was based on monthly data taken from the MSCI world index and the MIFC database for emerging markets data. The regressions were built using this index data and additional variable data taken from Datastream for the 5-year period between 1992 to 1997.

The variables utilized were economic variables which were believed to be sensible leading indicators and had a acceptable t-statistics. Generally speaking, the levels of predictability of these regressions were not extremely high. It should be noted, however, that the emphasis of this exercise was not so much to forecast returns with the most accurate regression, but to show the improvement on returns when one recognizes the impact of different relationships between asset classes under different circumstances (i.e. changing correlation and the use of differing covariance matrices). An improvement in predictability, particularly for the lead market should only enhance the effects of the improved correlation rules. The regressions and the data used to create the regressions can be found on the main spreadsheet.
 

US Directional Prediction

The asset allocation program has utilized the domestic US market as the pivot for selecting the appropriate correlation between assets. The rationale for this was that as the US is a large component of the MSCI World Index it is a very efficient market and is probably the leading index in the world economy.

The accuracy of this prediction is perhaps the most important determinant in affecting the actual return of the portfolio over the test period. Our results indicated a hit ratio for the directional count of 75% from our regression prediction. This however is masked by the consistent positive movement of the US market over the past few years (since 1992 in this sample). We were able to predict a positive movement 97% of the time and disappointingly the negative returns only 14% of the time. This is understandable considering the historic mean return for the US market was 1.349% per month. We were thus not trying to predict below average performance but below zero performance, which assuming a normal distribution has a probability of 0.13.

The inability of our regression model to accurately predict the negative movements, specifically the substantial movements in August 1998 has not allowed the full benefits of our model to be realized. Adjusting this pivot point to account for the general positive direction of the US market would be a good direction for further study, and would undoubtedly result in a more refined and more accurate model. Nevertheless, given the regression models and directional predictors, the benefits appear to be substantial.
 

Set-up of Experiment

For the base case scenarios in each period we looked to optimize our portfolio weights for each asset by running a Markowitz mean-variance optimization based on the expected returns for each index from our predictive regressions and the correlation and covariance of data from a rolling 5-year period (60 months) immediately prior to the current period. This optimization was performed again at the end of each month for the period from 1992 to 1998, for a total of 72 months of which the last year was out of sample for the given regression data.

To test the hypothesis we used our predictive regression for the US index to forecast the direction of the US market. If the return was predicted to go positive rather than using the correlation for the previous 60 months, we would then optimize using the same expected returns as in the base case but using the correlation/covariance of the 5 assets in the last 60 months in which the US pivot market had seen a positive return. Similarly, if we predicted a negative return we used a correlation/covariance matrix based on the previous 60 months where the US had seen a negative return to determine our optimal weights.

To perform a Markowitz mean optimization the other two variables that are important in determining the eventual optimal weights are the level of risk (sigma) that is chosen and the decision whether or not to allow short sales at all, and whether or not to limit positions (long or short) in particular assets:

To take these differences into account, we ran the following scenarios which accounted for the base case scenario with a single comprehensive matrix, and the two matrix model, as well as two additional scenarios for a simple buy and hold portfolio optimized as of the end of the in-sample period (January 1998) using the 5-year historical return as the expected return. Additionally, we ran one scenario with positive and negative matrices where correlation/covariance was predicted using an ARCH regression of residuals and lagged residuals.
 
 
 
 
Scenario
Description
Short Sales
Sigma
S1
1 matrix
No
Variable US
S2
2 matrix
No
Variable US
S3
1 matrix
20/50
Variable US
S4
2 matrix
20/50
Variable US
S5
1 matrix
Yes no limit
Variable US
S6
2 matrix
Yes no limit
Variable US
S7
1 matrix
20/50
US Historic
S8
2 matrix
20/50
US Historic
S9
2 matrix ARCH
20/50
Variable US
S10
Buy and hold
N/A*
US Historic
  *The optimal strategy for scenario 10 had no short sales, or any long positions greater than 50% even when allowing for unlimited short sales. In each case above, the scenario was run by selecting the appropriate options and running the model through an Excel program that calculated the appropriate predictions, performed the optimization and recorded the weights for each period. These weights were then matched up with the actual observed data to determine the performance of each scenario.
 

Results

Efficient Frontier

The initial efficient frontier was derived given the historical variance and returns for the base set of assets. This was used to benchmark the various scenarios and reflect the various historical profiles of the various countries. The results are illustrated below.

Scenarios

The model then used the forecasted returns and the respective covariance matrices to allocate the weights to the various asset classes based on our mean-variance optimization Solver engine. The results of the scenarios are included in Table 1.

Table 1: Actual annualized means and standard deviations for the Assets and Scenarios based on data from January 1992 to December 1998
Asset
 Annual Mean
Annual Std. Deviation
Index
World Return
14.74%
12.23%
248.62
US Return
21.08%
12.31%
168.73
BE Return
21.86%
12.20%
247.59
HK Return
19.84%
32.86%
82.63
JP Return
-0.02%
23.60%
362.19
VE Return
0.86%
49.48%
42.47
Return S1
24.35%
13.42%
432.42
Return S2
26.90%
16.11%
485.86
Return S3
25.98%
12.41%
477.65
Return S4
30.28%
16.57%
580.48
Return S5
29.29%
12.96%
569.94
Return S6
35.37%
17.36%
751.59
Return S7
26.57%
17.77%
467.34
Return S8
34.85%
20.36%
705.21
Return S9
2.31%
11.13%
114.04
Fixed Weight
14.81%
23.17%
218.25
To compare the impact of using two rolling covariance matrices we cross-tabulated the results to reflect the differences more clearly in Table 2. Table 2: Performance summary of single and two matrix strategies  
Description of Scenario
Single matrix index
Two matrix index
Single matrix average return
Two matrix average return
Diff. in basis points
No short sales allowed, variable US volatility
432
486
24.35%
26.90%
255
Max 20% short sales and 50% long, variable US volatility
478
581
25.98%
30.28%
430
No limit on short selling, variable US volatility
570
752
29.29%
35.37%
609
Max 20% short sales and 50% long, historic US volatility
467
705
26.57%
34.85%
828
Average
530
The following graphs show a comparison of base and double matrix scenarios as compared to the buy and hold portfolio and the MSCI world index return over the same period. Enlarged graphs can be viewed by clicking on the relevant chart.

Chart 1: No short sales allowed, variable US volatility

Chart 2: Max 20% short sales and 50% long, variable US volatility

Chart 3: No limit on short selling, variable US volatility

Chart 4: Max 20% short sales and 50% long, historic US volatility
 

Conclusion

Although we achieved relatively poor levels in predictability and directional forecast from our US regression, especially for negative movements, the technique of using two differing covariance matrices yielded substantial improvements. On average across all scenarios this came out to a difference of 530 basis points per annum. The lowest returns and gains vis-à-vis the benchmark came unsurprisingly when short sales were not allowed where we showed an improvement of 260 basis points by adjusting the covariance matrix. Interestingly, our best results came from using fixed weight sigmas to optimize the portfolios.

Our results were computed from January 1992 to December 1997 which were in sample for the data from our predictive regressions, and from January 1998 to December 1998 out of sample. In all cases the correlation matrices were taken from backward looking data and can be considered out of sample.

Indeed the results from this simple study with a five-country portfolio over a limited time period are very interesting and show good support for our initial hypothesis. An interesting development of this analysis would certainly be to vary the time horizons and get a more accurate test in down market conditions, which was unfortunately not possible in this case due to the relative paucity of data for predictive regression of Venezuela. Future studies with a varying mix of countries for the portfolio, and particularly a more accurate predictor model for the pivot country would indeed prove very interesting, and we would conjecture result in even greater improvements from using a multi-matrix model. Furthermore, the Excel spreadsheet that provided the engine for our analysis can be easily modified to test these and other scenarios and can be found here.

 

Further research

Forecasting of Variances and Covariance with ARCH

We decided to push our investigation one step further and try to replace the historical variances and covariance used in our optimization program with a forecast of these values from an ARCH regression of residuals. From the forecasted returns and the observed values of the predicted regressions we calculated our residuals. These were then squared and multiplied across countries (see sheet Res_Res in the Excel Workbook, BE_BE represents the Belgian residuals squared, BE_HK, the Belgian residuals multiplied by the Hong Kong residuals, etc). We then split these residual across up markets (in the US) and down markets. Each of these samples were then regressed against themselves lagged by one period. To accomplish this we use a direct less squares formula with a rolling sample based on the last 60 data available. In this manner, the regressions are done automatically and were integrated in the program (see sheet Res_Reg). The results were then translated into a variance and covariance matrix for each state of the market: up or down (see sheet matrix). These matrices were linked to the main program and that selects the one to use as the input for the variance-covariance matrix used to allocate assets.

The results we obtained by this method were however quite unsatisfactory. The out of sample return was less than 50% of the world return. One of the main reasons that we found for this was the small sample available for the ARCH regressions, particularly for downward movements over the 5 years available (this would be true even if we use the full sample instead of a rolling sample). As the size of the sample was determined by the shortest series of return forecast of the five countries (Venezuela) and this sample is reduced even further when we split the returns into two smaller samples in order to differentiate the up and down markets. The down markets are the more critical ones in this case since the US markets are up most of the time. In this case, our sample for downward movements was reduced to 20 data points, which we feel did not provide an adequate basis for the correlation between indexes.

In conclusion, we are still convinced that an ARCH method making the distinction between up and down markets would be useful to improve the allocation of the assets. Nevertheless, its success is closely linked to the quality of a prediction model for the trend of the US market.