Numerical software and tools for the actuarial community. Talk given at ATRC 2012

Published on: **Mar 3, 2016**

- 1. Numerical software & tools for the actuarial community John Holden Jacques du Toit 11th September 2012 Actuarial Teachers and Researchers Conference University of LeicesterExperts in numerical algorithmsand HPC services
- 2. Agenda NAG Introduction Software providers to the Insurance Market Numerical computation – why bother Problems in numerical computation NAG’s Numerical Libraries and Toolboxes Computational problems in Actuarial Science Numerical Excellence in Finance 2
- 3. Numerical Algorithms Group - What We Do NAG provides mathematical and statistical algorithm libraries widely used in industry and academia Established in 1970 with offices in Oxford, Manchester, Chicago, Taipei, Tokyo Not-for-profit organisation committed to research & development Library code written and contributed by some of the world’s most renowned mathematicians and computer scientists NAG’s numerical code is embedded within many vendor libraries such as AMD and Intel Many collaborative projects – e.g. CSE Support to the UK’s largest supercomputer, HECToR Numerical Excellence in Finance 3
- 4. Portfolio Numerical Libraries Highly flexible for use in many computing languages, programming environments, hardware platforms and for high performance computing methods Connector Products for Excel, MATLAB, .NET, R and Java Giving users of the spreadsheets and mathematical software packages access to NAG’s library of highly optimized and often superior numerical routines NAG Fortran Compiler and GUI based Windows Compiler: Fortran Builder Visualization and graphics software Build data visualization applications with NAG’s IRIS Explorer Consultancy services Numerical Excellence in Finance 4
- 5. Software providers to the Insurance Market ACTUARIS .. AIR Worldwide .. Algorithmics Microsoft Aon Benfield The Numerical Algorithms Group (NAG) ARC AXIS Oracle Financial Services Barrie & Hibbert PolySytems BPS Resolver RMS BWise SAS Institute ClusterSeven SunGard Conducter Towers Watson Conning Trillium Software … Ultimate Risk Solutions … WySTAR Numerical Excellence in Finance 5
- 6. How is this software made? Do these software providers write all their own code? Do these software providers write all their own Numerical Code? Why not? Numerical Excellence in Finance 6
- 7. How is this software made? Do these software providers write all their own code? No Do these software providers write all their own Numerical Code? No Why not? Let’s take a look Numerical Excellence in Finance 7
- 8. Why bother? Numerical computation is difficult to do accurately Problems of Overflow / underflow How does the computation behave for large / small numbers? Condition How is it affected by small changes in the input? Stability How sensitive is the computation to rounding errors? Importance of error analysis information about error bounds on solution Numerical Excellence in Finance 8
- 9. An example: sample variance For a collection of observations {xi ,,ii 1...n} {xi 1...n} the mean is defined as 1 n x xi n i 1 and the variance as n 1 s 2 ( xi x ) n 1 i 1 2 Numerical Excellence in Finance 9
- 10. Example calculation For this collection of observations{c 1, c, c 1} the mean is x 1 (c 1 c c 1) c 3 and the variance is s 2 1 (( 1)2 0 12 ) 1 2 <Excel – variance demo> Numerical Excellence in Finance 10
- 11. What’s gone wrong? Instead of 1 n s 2 ( xi x ) n 1 i 1 2 Excel uses an (analytically identical) formula 1 n 1 n 2 s 2 xi xi 2 n 1 i1 n i1 (one pass) faster to calculate accuracy problems if variance is small compared to x Numerical Excellence in Finance 15
- 12. Software providers to the Insurance Market ACTUARIS .. AIR Worldwide .. Algorithmics Microsoft Aon Benfield The Numerical Algorithms Group (NAG) ARC AXIS Oracle Financial Services Barrie & Hibbert PolySytems BPS Resolver RMS BWise SAS Institute ClusterSeven SunGard Conducter Towers Watson Conning Trillium Software … Ultimate Risk Solutions … WySTAR Numerical Excellence in Finance 16
- 13. Numerical computation – DIY Vs NAG DIY implementations of numerical components have their place, but NOT in production code. Handwritten and “hand me down” type code might be easy to implement, but will… NOT be well tested NOT fast NOT stable NOT deliver good error handling NAG implementations in contrast are fast and Accurate Well tested Thoroughly documented Give “qualified error” messages e.g. tolerances of answers (which the user can choose to ignore, but avoids proceeding blindly) Numerical Excellence in Finance 17
- 14. Why People use NAG Libraries and Toolboxes? Global reputation for quality – accuracy, reliability and robustness… Extensively tested, supported and maintained code Reduces development time Allows concentration on your key areas Components Fit into your environment Simple interfaces to your favourite packages Regular performance improvements! Numerical Excellence in Finance 18
- 15. NAG provides the atomic bricks … for the domain specialists to build the walls, houses and fancy castles! Users know NAG Components are here today, tomorrow and beyond Functions are not removed when new ones added without sensible notice and advice NAG functions are well documented Lets take a look…. Numerical Excellence in Finance 19
- 16. NAG Library and Toolbox Contents Root Finding Dense Linear Algebra Summation of Series Sparse Linear Algebra Quadrature Correlation & Regression Ordinary Differential Analysis Equations Multivariate Methods Partial Differential Equations Analysis of Variance Numerical Differentiation Random Number Generators Integral Equations Univariate Estimation Mesh Generation Nonparametric Statistics Interpolation Smoothing in Statistics Curve and Surface Fitting Contingency Table Analysis Optimization Survival Analysis Approximations of Special Time Series Analysis Functions Operations Research Numerical Excellence in Finance 20
- 17. NAG Data Mining Components Data Cleaning Regression Data Imputation Regression Trees Outlier Detection Linear Regression Multi-layer Perceptron Neural Data Transformations Networks Scaling Data Principal Component Analysis Nearest Neighbours Radial Basis Function Models Cluster Analysis Association Rules k-means Clustering Utility Functions Hierarchical Clustering To support the main functions and help with prototyping Classification Classification Trees Generalised Linear Models Nearest Neighbours Numerical Excellence in Finance 21
- 18. NAG routines for GPUs Random Number Generators L’Ecuyer mrg32k3a and Mersenne Twister (with skip- ahead) mt19937 Uniform distribution Normal distribution Exponential distribution Support for multiple streams and sub-streams Sobol sequence for Quasi-Monte Carlo ( up to 50,000 dimensions) Scrambled sequencing for Sobol (Hickernell) Brownian Bridge Numerical Excellence in Finance 22
- 19. Traditional Uses of NAG Libraries NAG is used where non-trivial mathematics must be done quickly and accurately on computers Largest user groups (not in order) Academic researchers (typically Statistics, Applied Mathematics, Finance, Economics, Physics, Engineering) Engineers (fluid dynamics, large-scale PDE problems, simulations) Statisticians (data mining, model fitting, analysis of residuals, time series, … ) Quantitative analysts (asset modelling and risk analysis) Numerical Excellence in Finance 23
- 20. Use of NAG Software in Statistics Multivariate Methods (G02/G04) Nearest correlation matrix, generalised regression with various error distributions (with and without missing data), robust/ridge/partial least squares regression, mixed effects and quantile regression, … Nonparametric Statistics (G08) Hypothesis testing Survival Analysis (G12) Time Series Analysis (G13) SARIMA, VARMA, GARCH, with various modifications Random Number Generators (G05) Numerical Excellence in Finance 24
- 21. The NAG Library and Actuarial Statistics Survival models: Cox regression model (g12bac) Kaplan-Meier estimator (g12aac) Weibull, exponential and extreme values (via g01gcc) Risk analysis/ loss functions: Distributions: lognormal, gamma, beta etc both distribution functions (g01) & random number generation (g05). Other Time series (g05 and g13) Convolutions: FFTs (c06) Kernel density estimation Graduation: generalised linear models (g02g) Analysis of risk factors: generalised linear models (g02g) Numerical Excellence in Finance 25
- 22. Use of NAG Software in Finance Portfolio analysis / Index tracking / Risk management Optimization , linear algebra, copulas… Derivative pricing PDEs, RNGs, multivariate normal, … Fixed Income/ Asset management / Portfolio Immunization Operations research Data analysis Time series, GARCH, principal component analysis, data smoothing, … Monte Carlo simulation RNGs …… Numerical Excellence in Finance 26
- 23. Why Quantitative Analysts Love NAG? General Problem To build asset models and risk engines in a timely manner that are Robust Stable Quick Solution Use robust, well tested, fast numerical components This allows the “expensive” experts to concentrate on the modelling and interpretation avoiding distraction with low level numerical components Numerical Excellence in Finance 27
- 24. Problem 1: Simulation (Monte Carlo) Simulation is important for scenario generation Several different numerical components needed Random Number Generators Brownian bridge constructor Interpolation/Splines Principal Component Analysis Cholesky Decomposition Distributions (uniform, Normal, exponential gamma, Poisson, Student’s t, Weibull,..) .. Numerical Excellence in Finance 28
- 25. Problem 1: Simulation (Monte Carlo) Simulation is important for scenario generation NAG to the rescue (CPU or GPU) Several different numerical components needed Random Number Generators √ Brownian bridge constructor √ Interpolation/Splines √ Principal Component Analysis√ Cholesky Decomposition √ Distributions (uniform, Normal, exponential gamma, Poisson, Student’s t, Weibull,..)√ .. √ √ Numerical Excellence in Finance 29
- 26. Problem 2: Calibration Financial institutions all need to calibrate their models Several different numerical components needed Optimisation functions (e.g. constrained non-linear optimisers) Interpolation functions Spline functions .. Numerical Excellence in Finance 30
- 27. Problem 2: Calibration Financial institutions all need to calibrate their models NAG to the rescue Several different numerical components needed Optimisation functions (e.g. constrained non-linear optimisers) √ Interpolation functions (used intelligently*) √ Spline functions √ .. √ √ *interpolator must be used carefully –must know the properties to pick appropriate method Numerical Excellence in Finance 31
- 28. Problem 3: Historical VaR VaR methodology (important for identifying what variables might impact you most (eg Yen Vs USD)) Several different numerical components needed Time Series Correlation and Regressions Matrix functions Cholesky Decomp RNGs .. Numerical Excellence in Finance 32
- 29. Problem 3: Historical VaR VaR methodology (important for identifying what variables might impact you most (eg Yen Vs USD)) NAG to the rescue Several different numerical components needed Time Series √ Correlation and Regressions √ Matrix functions √ Cholesky Decomp √ RNGs .. √ √ Numerical Excellence in Finance 33
- 30. NAG fits into your favourite environments Supporting Wide Range of Operating systems… Windows, Linux, Solaris, Mac, … …and a number of interfaces C, C++, Excel, Fortran, LabVIEW, VB, Excel & VBA, MATLAB, C#, F#, VB.NET, Maple, CUDA, OpenCL, Mathematica Java, R, S-Plus, Python Scilab, Octave … … Numerical Excellence in Finance 34
- 31. NAG and Excel Our libraries are easily accessible from Excel: Calling NAG DLLs using VBA NAG provide VB Declaration Statements and Examples NAG provide “Add-ins” Calling NAG Library for .NET using VSTO Functions with Reverse Communication (useful for Solver replication for example) can be provided Create NAG XLLs Numerical Excellence in Finance 35
- 32. NAG Portfolio Optimization in Excel Numerical Excellence in Finance 36
- 33. Example – Kaplan-Meier survival probabilities Data Life tables for WHO Member States Global level of child and adult mortality http://www.who.int/healthinfo/statistics/mortality_life_ta bles/en/index.html How many people out 0f 100,000 die at birth, until 1YO, 5YO, etc. and how many people live 100 years or longer Numerical Excellence in Finance 37
- 34. Input … and … output … Numerical Excellence in Finance 38
- 35. … followed by Excel plot Numerical Excellence in Finance 39
- 36. How do you call the functions in Excel Numerical Excellence in Finance 40
- 37. How do you call the functions in Excel Enter the inputs from the spreadsheet Excel function wizard Numerical Excellence in Finance 41
- 38. What’s under the hood? NAG Library function called via VBA Numerical Excellence in Finance 42
- 39. NAG and .NETNAG solutions for .NET1. Call NAG C (or Fortran) DLL from C#2. NAG Library for .NET “a more natural solution” DLL with C# wrappers Integrated help Not yet the full Library, but most widely used chapters included. Very popular with .NET dev community inc. in Financial Services. Numerical Excellence in Finance 43
- 40. NAG Toolbox for MATLAB Contains essentially all NAG functionality not a subset Runs under Windows (32/64bit), Linux (32/64- bit) and Mac (64 bit). Comprehensive documentation (in MATLAB and pdf) Easy migration to production code in C/C++ or Fortran Can be used with MATLAB compiler Numerical Excellence in Finance 44
- 41. NAG Toolbox help MATLAB formatting NAG formatting Numerical Excellence in Finance 45 chapters (in PDF)
- 42. NAG Toolbox for MATLAB Offers complementary functionality to MATLAB Alternative to several specialist toolboxes “I really like the NAG Toolbox for MATLAB for the following reasons (among others): It can speed up MATLAB calculations – see my article on MATLAB’s interp1 function for example. Their support team is superb.” http://www.walkingrandomly.com/?p=160 Senior Developer “concerning the ‘nearest correlation’ algorithm. I have to say, it is very fast, it uses all the power of my pc and the result is very satisfactory.” Numerical Excellence in Finance 46
- 43. Computational problems in Actuarial Science Liability Modelling Asset Modelling Solvency II Nearest Correlation Matrix example Numerical Excellence in Finance 47
- 44. Liability Modelling “Traditional” actuarial science is focused predominantly on liability modelling Forecast cash flows directly linked to mortality/longevity Example: a pension scheme. Premiums received until retirement, pension paid until death, lump sum paid upon death Requires some modelling of market conditions (assumptions on inflation, gilt yields, index returns, … ) Fair to say often this modelling is not very sophisticated and is not very computationally demanding. Numerical Excellence in Finance 48
- 45. Asset Modelling Liability modelling very well understood Been doing it for more than two centuries, mostly get it right (sometimes get it wrong) Commercial packages to do this (Prophet, MoSes, …) Often heavily regulated (e.g. pensions) Asset modelling, in the actuarial context, perhaps less so Traditional view been to model average behaviour over long horizons – simplicity is sensible, since so many assumptions anyway Numerical Excellence in Finance 49
- 46. Solvency II There is a regulatory push to change this “Solvency II = Basel for insurers” Similar risk methodology as banks, being introduced for insurers Aim is to stress insurer’s balance sheet to various shocks, especially market shocks Requires more explicit modelling of assets Initial guidelines laid down by regulator – pretty simplistic However, as with Basel, insurers encouraged to develop own approaches (which would be less punitive) Horizons fairly short-dated Numerical Excellence in Finance 50
- 47. Solvency II Asset modelling is difficult! Ask a financial mathematician (or a quant) Technically demanding and computationally demanding Moreover, every market is unique Not just across asset classes, but different countries as well. No “one-size-fits-all” approach possible Each has own behaviour, own peculiarities NAG Library used extensively for building sophisticated, robust asset models and risk engines Numerical Excellence in Finance 51
- 48. NEAREST CORRELATION MATRIX 52
- 49. Example: Nearest Correlation Matrix Mathematically, a correlation matrix 𝐶 ∈ ℝ 𝑛×𝑛 is ... 1. Square 2. Symmetric with ones on diagonal 3. Is positive semi-definite: 𝑥 𝑇 𝐶𝑥 ≥ 0 for all 𝑥 ∈ ℝ 𝑛 How do we estimate correlations? Historical data Parametric methods such as Gaussian Copulas Try to back it out from options markets Typically 1 and 2 easy enough to ensure Ensuring positive semi-definite can be tricky Numerical Excellence in Finance 53
- 50. Example: Nearest Correlation Matrix Historical data Take time series for several observables and try to estimate correlation Gaussian copula Model for turning a set of marginals + a correlation matrix into a joint distribution. Was popular in credit modelling until 2008/9 proved it was (in many cases) wholly inadequate Infer from options markets Combine individual options and options on indexes to back out correlation structure Numerical Excellence in Finance 54
- 51. Example: Nearest Correlation Matrix In all these cases, need to work with correlation matrices estimated from “real world” data Real data is messy Given importance of correlation, what happens if estimate not mathematically correct? Numerical Excellence in Finance 55
- 52. Example: Nearest Correlation Matrix NAG Library can find the “nearest” correlation matrix to a given square matrix 𝐴 2 G02AA solves the problem min 𝐶 𝐴 − 𝐶 𝐹 in Frobenius norm G02AB incorporates weights min 𝐶 𝑊 1/2 𝐴− 𝐶 𝑊 1/2 2 𝐹 Weights useful when have more confidence in accuracy of observations for certain observables than for others Numerical Excellence in Finance 56
- 53. Example: Nearest Correlation Matrix The effect of W:A = 0.4218 0.6557 0.6787 0.6555 0.9157 0.3571 0.7577 0.1712 0.7922 0.8491 0.7431 0.7060 0.9595 0.9340 0.3922 0.0318W = diag([10,10,1,1])W*A*W = Whole rows/cols 42.1761 65.5741 6.7874 6.5548 weighted by 𝑤 𝑖 91.5736 35.7123 7.5774 1.7119 7.9221 8.4913 0.7431 0.7060 Elements weighted 9.5949 9.3399 0.3922 0.0318 by 𝑤 𝑖 ∗ 𝑤 𝑗 Numerical Excellence in Finance 57
- 54. Example: Nearest Correlation Matrix Can also do dimension reduction (G02AE) So-called factor models. Similar to PCA in regression Suppose have assets 𝑌1 , ⋯ , 𝑌 𝑛 , an 𝑛 dimensional source of noise 𝑊 and an 𝑛 × 𝑛 “correlation” matrix 𝐴 where Yt1 Wt1 Ft A Yt n Wt n For example, a simple multi-asset model: one factor (e.g. Brownian motion) for each asset, and 𝐴𝐴 𝑇 gives correlation between all factors Numerical Excellence in Finance 58
- 55. 3rd Example: Nearest Correlation Matrix Can use NAG Library to reduce the number of factors Find a n × 𝑘 matrix D (where 𝑘 < 𝑛) such that Yt1 Wt1 Ft D Yt n Wt k Crucially, 𝐷𝐷 𝑇 gives a correlation structure as close as possible to the original structure implied by 𝐴 Can be very useful to reduce complexity and computational cost of some models and applications Numerical Excellence in Finance 59
- 56. Example: Nearest Correlation Matrix Spreadsheet (if time) Numerical Excellence in Finance 60
- 57. NAG is a HPCFinance partnerhttp://www.hpcfinance.euThe network is recruiting for Early Stage Researchers (ESRs ~ PhD Students) Experienced Researchers (ERs in Post Docs) Numerical Excellence ~ Finance 61
- 58. NAG and Actuarial Science - Summary NAG is keen to collaborate in building actuarial models and risk engines Your requirements likely to be different from banks/hedge funds We want to make sure we have what you need Risk engines likely to involve a LOT of computation NAG has significant experience in HPC services, consulting and training We know how to do large scale computations efficiently This is non-trivial! Our expertise has been sought out and exploited by organisations such as (BP, HECToR, Microsoft, Oracle, Rolls Royce, …….) Numerical Excellence in Finance 62
- 59. Keep in touchMany of you are already licensed to use NAG software….Technical Support and Help support@nag.co.ukTo reach the speakersjohn.holden@nag.co.ukjacques@nag.co.ukNAGNews http://www.nag.co.uk/NAGNews/Index.asp Numerical Excellence in Finance 63