Survivorship bias and estimation of fund performance

If you run a fund, you are very happy about good performance and unhappy about poor returns. So are your investors. What happens if your fund performs poor for several years in a row? You loose your investors and finally close the fund. This fact is very important: After closing the fund, your fund data usually disappears from the market data providers (reuters, bloomberg, yahoo, etc.).

Take a look at the following example:

Fund A, B, C represent all available public funds in the hypothetical bigdata-biology-cloud-computing subsector (BBCC sector). In this example, fund B performs so poor that, it was closed in year 4. Now, performing a study on the BBCC sector, the analysts will find historical data for funds A and C, only. Consequently, they will report an average performance of 9% p.a. in the BBCC sector. However, a real investor investing in the BBCC sector distributing her investments equally across all available funds (A, B and C) would have gained 5% p.a. only. This is about half of the performance seen by the analysts. This kind of error happens easily and thus often.

Survivorship bias and historical backtesting

A similar effect happens often when people try to find a superior trading strategy in historical market data. Often you see the following algorithm for fitting parameters of a trading strategy:

1. Data = Historical Market Data
2. Choose values x_1 to x_n for parameters of trading strategy S
3. If performance( S(x_1,x_n), Data ) is maximum of all possible values of (x_1, x_n)

stop and return x_1 to x_n

else

goto 2.

end

Clearly, this strategy S(x_1,x_n) will perform great on the available historical data. But, this says nothing about the future performance of the strategy. And, usually, the strategy will not perform good in the future.

How can you do better? A better approach follows from division of the data into a training- and a test-set. The resulting algorithm would be:

1. Data_training = half of Historical Market Data
2. Data_test = other half of Historical Market Data
3. Choose values x_1 to x_n for parameters of trading strategy S
4. If performance( S(x_1,x_n), Data_training ) is maximum of all possible values of (x_1, x_n)

If performance( S(x_1,x_n), Data_test ) is good

stop and return “Performance of S is good with parameters: ” + x_1 to x_n

else

stop and return “Strategy S is a bad strategy”

end

else

goto 2.

end

In this case, the trading strategy S is optimized on a training set of the historical data and the performance of the test-set is good either. This still does not mean that the resulting strategy will perform well in the future. But at least it did perform well on unseen data. Now, this algorithm may return with “Strategy S is a bad strategy”, which will be unsatisfactory for most people. Most people will then develop more and more strategies S_1 … S_m  until the algorithm returns “Performance of S is good …”. But, is this really a good strategy? If you want an answer to this, make sure that you reserve some of the historical data for a validation and check your strategy on this unseen data. So, for optimizing a trading strategy you need at least three data sets: training set, test set and validation set. Again, this will say nothing about performance of your trading strategy in the future. But, it is a good indication if your strategy passes this tough strategy optimization.

Survivorship bias and fat tails

Another feature, which creates wrong views on the performance of financial assets or trading strategies are fat tails. There are several kind of trading strategies which seem to exploit arbitrage in the market. E.g. volatility arbitrage funds were popular a few years ago. They perform well on historical data and they deliver steady income with no risk. But, suddenly, something happens:

Source:  surlytrader.com

Suddenly, the performance of a few years is gone forever. And, this might also happen with strategies which are perceived risk-less like money market funds:

Money Market Funds can be risky, too (Source: Handelsblatt)

Conclusion

The survivorship bias significantly complicates the performance estimation of funds and trading strategies. And, even if you do everything right, your performance might be hit by a market shift as we saw for money market funds in 2008/2009. Finding free lunch in the financial market is hard.

Financial Modelling: Theory, Implementation and Practice with MATLAB Source (Wiley Finance) See at Amazon COM Amazon UK Amazon DE

Review

Form my point of view, the book is a very good reference for many models together with a reference implementation. The gap between theory and practical implementation is much smaller as in other books. But, the direct application of the algorithms in a real-life project is often not possible. The provided sources are rather a collection of algorithms and not a complete Matlab toolbox in that sense. For fulfilling this purpose, one would expect a framework with a consistent API across the algorithms and classes for managing and transforming market data inputs. E.g. a class for interest rates with transformation into discount factors or forward short-rates would help a lot. And a deeper class hierarchy for the different economic scenario generators would simplify the usage. But, this is left to the reader.

The explanations and derivations of the models are precise and clear. The presentation is good with minor deficits – e.g. sometimes the referenced figures appear several pages later. The part III of the book summarizes simple numerics and introduces basic concepts of Matlab. This seems somewhat redundant since there are much more focused books for numerics and Matlab programming.

In total, this book is a fantastic reference for everyone implementing state-of-the-art financial models.

Another great thing about this book is that the Matlab source codes are available for free download at http://www.mathworks.de/matlabcentral/fileexchange/authors/246981

The books outline:

Part I: Financial Markets and Popular Models

Chapter 1: Financial Derivatives – Data, Basics and Derivatives

Chapter 2: Diffusion Models

Chapter 3: Models with Jumps

Chapter 4: Multi-Dimensional Models

Part II: Numerical Methods and Recipes

Chapter 5: Option Pricing by Transformation and Direct Integration

Chapter 6: Advanced Topics Using Transform Techniques

Chapter 7: Monte Carlo Simulation and Applications

Chapter 8: Monte Carlo Simulation and Advanced Issues

Chapter 9: Calibration and Optimization

Chapter 10: Model Risk – Calibration, Pricing and Hedging

Part III: Implementation, Software Design and Mathematics

Chapter 11: Matlab – Basics

Chapter 12: Matlab Object Oriented Development

Chapter 13: Math Fundermentals

]]> http://computeraidedfinance.com/2013/02/07/financial-modelling-with-matlab-source-a-great-new-book/feed/ 0 Financial Modelling: Theory, Implementation and Practice with MATLAB Source (Wiley Finance) computeraidedfinance Book Cover http://computeraidedfinance.com/2013/01/03/quandl-a-wikipedia-for-numerical-data/ http://computeraidedfinance.com/2013/01/03/quandl-a-wikipedia-for-numerical-data/#comments Thu, 03 Jan 2013 10:48:24 +0000 tammerk http://computeraidedfinance.com/?p=1122 ]]> For data analysts like myself, one of the most useful pages on any blog resides on this website. The comprehensive list of data sites Andreas put together is indispensable because finding numerical data online, even though it is abundant, is so difficult and time consuming.

Quandl.com – a new player for finding market data

Any real world data user knows the pain of Googling for numerical data.  Many times I have spent tens of minutes and sometimes hours trying to find data on the web.  If I eventually find data, and often I do, I then spend time validating, formatting and cleaning the data.

I think this is a universal experience:  Any real-world data analyst knows, the blood, sweat and tears are not in the analysis of data, they’re in the acquisition of data.

One day, several years ago, after another long battle trying to find data, I realized that I, (and no doubt many others), needed someone to create the “Wikipedia of Numerical Data”.  At that moment, I became acutely aware of the difference between quantitative knowledge on the internet and qualitative knowledge.  The latter is beautifully and comprehensively organized by Wikipedia.  On the other hand, quantitative knowledge, (numerical data), is scattered across thousands of different web sites in hundreds of different formats and is generally difficult to find and difficult to use.

So I am now on a mission to build “Wikipedia for Numerical Data”.  I am doing this not by asking publishers to put their data on my site, but rather by building a smart index that knows where to get data when a user wants it.  My site, www.quandl.com, is something akin to a search engine for numerical data.

Use Quandl.com and google your market data

Quandl is open and free.  It is curated and maintained by a (thus far) small team of like minded individuals.  My goal is to recruit more help and make it so that there is a one easy to use site where you can get any numerical data you need.

If you are interested in learning more about our objectives or perhaps even getting involved, please visit our site where you will find my email address with little difficulty.

I always hear that cats work best in the internet. So, here is a great cats Christmas song:

Merry Christmas! And a happy new year!

Andreas

Me, as a dedicated Matlab user, I am very excited about the things coming: The MathWorks can build the best toolbox unifying the comfort of PCT and the speed of Jacket. Let’s see, what happens.

Optimal stopping is required to benefit from early exercise rights.

Background

Many types of traded financial options include early exercise rights. Examples are index options, which are often early exercisable. People call these put or call options American options in contrast to European options, which are only exercisable at maturity. This feature is also common in many warrant types, and many OTC contracts. Even more complex are e.g. convertible bonds, where also the issuer can force conversion.

The main argument for early exercise provisions is that the clients like to have choices. Therefore, people often argue that one should always structure financial instruments in a way that the holder can exercise early – the product would sell at a higher price. But, this is not always the case. E.g. the cost of the early exercise feature is not represented correctly in many life insurance contracts and in many mortgages. If the policy holders would start exercising their policies more rationally, the insurance providers might be in deep trouble.

Optimal Stopping

Looking at early exercise decisions, I would like to provide a more visual example: Find a good parking lot at a large supermarket. In many supermarkets, you find a parking like this:

Usual situation at a supermarket parking

You drive slowly towards the supermarket and at each empty spot (1) you have to decide to park (exercise) or to continue. Continuing the search evolves the risk that you do not find a better spot and you have to drive another full round. A simple solution to this problem is to just take the first empty spot. In a simple numerical example, we find that this strategy would give is the following distribution of distance to the market:

Histogram of the distribution of the distance to the supermarket using the simple strategy: Take the first empty spot. (10,000 simulations)

This example uses 10,000 Monte Carlo simulations for finding a parking spot. Since in this example, there is only a 10% chance that each spot is empty, we often find a very bad spot (e.g. more than 1,000 of 10,000 park at a distance of 20) and sometimes we find no spot at all (about 1.200 written as distance 40). Less than 200 find the best spot with distance 1.

If we would like to improve this result, we can create an optimization problem minimizing the expected distance to the supermarket. I did this writing a little ThetaML which you find in the Appendix below. Now, we find that the maximizing strategy is to pass the first four spots (distance 17 to 20) and then take the next empty spot. This leave us with no one parking at a distance of 17 or further. The probability of getting the best spot at distance 1 is higher – over 200 simulations end here. But, this comes at a cost: More than 1,800 simulations do not find a spot at all and have to go for another round:

Histogram of the distribution of the distance to the supermarket using the optimal strategy: Take the first empty spot with distance less than 17. (10,000 simulations)

A Better Solution

Ok, I invested a lot of knowledge and some programming into optimizing the exercise strategy at the car parking. Is this the best we can do?

There is actually a better solution by rearranging the parking lot:

Rearranged parking lot

Now, the simple strategy: Take the first empty spot yields to a much better result:

Histogram of the distribution of the distance to the supermarket using a better parking lot layout and the simple strategy: Take the first empty spot. (10,000 simulations)

More than 1,000 of the 10,000 simulations find the best spot at a distance of 1. Less than 200 find a bad spot at a distance of 20 and only 1,200 find no spot at all. This is much better than the previous “optimal” solution. And I do not have to take my laptop with me to find the optimal exercise strategy at each time shopping. I wish parking lot architects could see this…

Taking this idea to the financial options market, what can we do? Maybe, the investors do not want to buy American type early exercise options. They might prefer Lookbacks which guarantee a maximal payout. Think about it!

Appendix

ThetaML simulation code solving the optimal stopping problem.

model parkinglot
 import DriveToMarket
 import NumberOfLots
 import ProbabilityOfFree
 export LotIsFree

 loop NumberOfLots
   if DriveToMarket
     distance = NumberOfLots - @time
   else
     distance = @time + 1
   end

   if rand()>ProbabilityOfFree
     LotIsFree = 0
   else
     LotIsFree = distance
   end
   Theta 1
 end
 LotIsFree = 2 * NumberOfLots
end

model FindBestLot
 import NumberOfLots
 import LotIsFree
 export LotTaken
 export Distance

 Distance = E(Distance!)
 loop NumberOfLots
   if LotIsFree>0
     if LotIsFree < E(Distance!)
       Distance = LotIsFree
       LotTaken = @time
     end
   end
   Theta 1
 end
 LotTaken = @time
 Distance = LotIsFree
end

Changes in Interest-Rate and Credit Instrument Pricing

Pre-crisis, there was a risk-free rate on which the market agreed. This way, different banks could create interest-rate swap transactions using basically the same conditions with different counter parties. This also meant that one could price the different instruments with the same discount curve.

Now, this risk-free rate has disappeared. The market has created a new view on credit risk. This view requires that collateral is set aside for transactions which were seen as risk-free before the crisis. A more detailed credit default modelling is also required – which is non-trivial. It seems that there is no established market standard for pricing the instruments consistently, yet. Chris Kenyon and Roland Stamm propose such a framework in their new book:

Chris Kanyon and Roland Stamm: “Discounting, LIBOR, CVA and Funding: Interest Rate and Credit Pricing (Applied Quantitative Finance).” See at Amazon.co.uk  Amazon.de  Amazon.com

The presented framework is quite complex. It involves several additional parameter curves. For me, the effort seems to be too much for most cases. But, it is a great starting point for a new pricing framework to evolve.

A presentation with some of the insights is available at PRMIA:  Presentation: Discounting and Curve Construction Since the Beginning of Financial Crisis 2007

Conclusion

It is interesting to see that pre-crisis, the pricing challenge lay in the complexity of the products. Post-Crisis, the challenge is already pricing simple products.

Historical Volatility

Historical volatility measures the deviation of asset returns from their historical average. It is a sample estimate that has time unit and depends on the length of the historical sample period. Denote as the asset return at time , the historical volatility of for is calculated as

,

where is the mean of over the time period . To obtain the sample estimate , the number is replaced by its sample counterpart , and

,

where we have divided the sum of squared deviations by to account for the loss of degree of freedom from using the sample mean in the calculation. For sampled at daily and monthly time intervals, the annualized historical volatility is respectively and .

Historical volatility can be useful in cases like benchmarking historical performances, assessing the behavior of economic variables, calculating historical risk measures, and so on.

Since historical volatility is the number we get from looking into the past realized data, it may not be as useful when we try to predict the future behavior of asset returns. This is when and why we have the implied volatility, i.e. the volatility number implied from market prices.

Implied volatility

Denote , , a set of market prices of traded options on asset , the implied volatility is obtained by setting the market price to the theoretical price, , with the argument as the option strike price, the current asset price, is the option time-to-maturiy, the risk-free interest rate, and so on. Since the only unknown in is the volatility number , we can solve the inverse pricing function to obtain the implied volatility using the bisection or Newton‘s iterative method. The obtained implied volatilities contain information of the market’s assessment of future price movements, and thus is a forward-looking risk measure. Here, you can find a little solver for implied volatilities in VBA.

Example: German DAX

The following graphs show the series of VDAX-NEW implied volatility index compared to the DAX historical volatilities. The implied volatility index is a good predictor of the historical volatilities.

(click to enlarge)

We can see that the implied volatility given by VDAX New and the 21-day historical volatility had similar values during the time presented. Most of the time, the implied volatility is slightly higher than the historical volatility.

(Click to enlarge)

The graph above shows the VDAX shifted by 21 days and the 21 day historical volatility. This way, we can see how well VDAX performs as a predictor of the historical volatility. We see that except the unforeseen spikes, implied volatility can be a good predictor of (future) historical volatility. Now, a similar graph with 1 year time horizons:

(click to enlarge)

With a one year horizon, the relationship between historical volatility and implied volatility is not that obvious.

An in-depth study of the predictability of historical volatility by implied volatility can also be found here in Wilmott Magazine.

Conclusion

While the historical volatility describes the size of the variations of the past, the implied volatility describes the size of the variations of the futures, implied by the traded option prices.

Ribbon (toolstrip)

As we can see in the screen shot, MATLAB now features an MS Office style ribbon, or toolstrip like The MathWorks calls it. The cool thing is that these buttons are context sensitive and show e.g. plots if relevant for the data:

This way, it is a lot easier to identify the plot you want from the large gallery of plots:

I like this. I would wish a quick preview on hovering over the icon. But, I leave this for The MathWorks next revision.

Apps

Now, MATLAB has a new method for packaging and distributing Software: Apps. The apps have their own place in the toolstrip (ribbon) and this way, they are easily accessible:

The “Get More Apps” button is connected to “MATLAB File Exchange” where users contribute files and after a review from The MathWorks, the apps are published. It is not entirely clear what kind of review The MathWorks performs. But, I think that this way the Apps might be virus proof to some extend.

Apps are always UI centric MATLAB programs, usually created with the GUI builder from MATLAB: “guide”. It is sad, that “guide” still misses features like tree views or HTML export. But, it does what it should: It helps creating basic GUIs.

Currently, this app market contains 60 apps. This is not much compared to the 16.000 other files on file exchange or millions of apps for phones. But, this will change quickly since the new apps are very convenient.

GPU

There are significant speed-ups in MATLAB 8 (R2012b) resp. its Parallel Computing Toolbox (PCT). The GPU now passes my benchmark about 2x faster on both my GPUs (GeForce GTX 275 and GTX 550 Ti). The speed-up advantage of Accelereyes Jacket is melting. But, Jacket is still 2x faster than the new PCT.

Conclusion

The new MATLAB 8 is a very useful update. The introduction of a ribbon is late, but it feels great. The new apps deliver an easy way packaging 3rd party functionality and deploying it to other MATLAB users. Other minor improvements and a major speed improvement of the Parallel Computing Toolbox with GPUs complete this new version.

Installation

The installation on my Asus laptop was smooth and took about 30 minutes. I could not upgrade my windows 7 because I wanted to switch from 32 Bit to 64 Bit. So, I am left with a clean installation. Windows found most drivers without even telling me. But, a few laptop specific functions like my function key “fn” and “+ sound volume” do not work.

Compatibility

All my programs worked without large issues. The small things I noticed:

• Navigation is all different: Users are often confronted with a full screen without any choices or options. They are often available pointing into one of the screen edges.
• “Start” is gone from the desktop and thus it is sometimes hard to find the programs and tools you need.
• “.NET 3.5″ is  not installed – installing it manually was no problem (otherwise my MS Excel plugins did not work).
• Drag and drop does not work with the new VLC.

Features, Look and Feel

• The look and feel is all different. Windows 8 feels fresh and clear.
• New features like external data versioning of “My Documents” are great.
• The UI is designed for a touch interface and keyboard. Using the keyboard only feels awkward.
• Metro Apps, Office 2013 Apps, the Webstore integration and the question for a Microsoft account everywhere (e.g. starting Visual Studio 2012 with JavaScript) give you the feeling of being monitored.
• The permanent online style together with hidden technical details destroys the feeling I had in previous Windows versions that you know at least to some extend where your data is and what is done with it. Now, everything is in the cloud…

Conclusion

The new Windows 8 will change the way of working. My impression is the even quants will have to learn JavaScript for creating useful UIs within Windows and MS Excel. Collaboration using Sharepoint or Skydrive is a lot easier and loosing data seems to be a lot harder.

Even though Windows 8 requires learning many new UI concepts, I like it. Integrating all business workflows with Apps and standard web technology is the right way to go.