2015-15994. Self-Regulatory Organizations; The Options Clearing Corporation; Notice of No Objection to an Advance Notice Concerning Modifications To Backtesting Procedures in Order To Enhance Monitoring of Margin Coverage and Model Risk Exposure

June 24, 2015.

On November 13, 2014, The Options Clearing Corporation (“OCC”) filed with the Securities and Exchange Commission (“Commission”) advance notice SR-OCC-2014-810 (“Advance Notice”) pursuant to Section 806(e)(1) of the Payment, Clearing, and Settlement Supervision Act of 2010 (“Payment, Clearing and Settlement Supervision Act”) ^[1] and Rule 19b-4(n)(1)(i) under the Securities Exchange Act of 1934 (“Exchange Act”) to modify backtesting procedures to better identify and make improvements to its monitoring of its margin methodology and to enhance its ability to manage risk.^[2] The Advance Notice was published for comment in the Federal Register on December 11, 2014.^[3] On January 9, 2015, pursuant to section 806(e)(1)(D) of the Payment, Clearing and Settlement Supervision Act,^[4] the Commission required OCC to provide additional information concerning the Advance Notice.^[5] The Commission did not receive any comments on the Advance Notice. This publication serves as a notice of no objection to the Advance Notice.

I. Description of the Advance Notice

As described in OCC's Notice,^[6] the proposed change modifies OCC's backtesting procedures to enhance its monitoring of margin coverage and model risk exposure. Such monitoring will allow OCC to better identify and make improvements to its margin methodology and thus enhance OCC's ability to manage risk.^[7]

OCC implements backtesting procedures to test its methodology for determining the amount of margin to collect from clearing members and validate the assumptions and mechanisms inherent in its methodology and to make any necessary changes to the methodology. Each trading day, OCC estimates the risk exposure of accounts and uses this estimate as a basis for each account's margin charge. On the following business day, OCC's current backtesting procedures compare an account's observed profit and loss (“P&L”) with the prior day's estimated risk using a variety of analytical and statistical tools. These daily tests measure the performance of OCC's risk measures for each account, and, therefore, also measure the performance of OCC's underlying margin methodology. OCC's backtesting program enables OCC to assess performance of its margining systems and determine whether financial risks are adequately or inadequately captured by the quantitative models in use.

OCC has conducted daily backtesting of margin accounts since 2006. OCC employs the “traffic light” test published by the Basel Committee on Banking Supervision in 1996 (the “Traffic Light Test”).^[8] In conducting the Traffic Light Test, OCC determines the actual number of instances in which the realized loss on an account exceeded the margin, referred to as an “exceedance,” over an observation period of one year. The number of exceedances during the observation period is compared against the number of expected exceedances under the assumption that the exceedances are independent and identically distributed over time. When backtesting results reveal the potential opportunity for remediation of OCC's margin methodology, OCC undertakes a root cause analysis to determine the cause of any issues. Any significant shortcomings of OCC's methodology lead to OCC undertaking a model improvement project designed to correct the problems. After analyzing the exceedances, OCC provides monthly reports to OCC's Enterprise Risk Management Committee (“ERMC”), which include, among other things, pertinent conclusions based on results from the full set of backtests.

OCC analyzed its backtesting program and identified several enhancements to the program, as discussed in more detail below: (1) Enhancement of and increase in the number of statistical tests, (2) data set changes, (3) forecast horizon changes, and (4) root cause analysis changes.

1. Enhancement of and Increase in the Number of Statistical Tests

As proposed in the Notice, OCC will enhance an existing statistical test and add three new statistical tests. OCC proposed to enhance its existing Traffic Light Test so that it may be applied to exceedances across all of OCC's margin accounts. Given that exceedances are not independent across margin accounts, OCC will enhance this test to address the dependency of exceedances between accounts.

In addition to the enhanced Traffic Light Test, OCC will implement three other industry standard tests related to exceedances in order to provide a more comprehensive set of tests. First, OCC will add the Kupiec Test,^[9] which is a new proportion of failures test that compares the actual number of exceedances with the number that would be expected in light of the confidence level associated with the calculation of margin. For example, when calculating margin with a confidence level of 99%, the number of exceedances is expected to be 1% of the total observations (i.e., the P&Ls for all accounts for all days during the measurement period). If the actual number of exceedances is near the expected number, this is an indication that the calculated margin requirements are not inaccurate estimates of the accounts' estimated losses.

Second, OCC will add the Christoffersen Independence Test,^[10] which is a new statistical test that measures the extent to which exceedances are independent of each other. Specifically, if OCC's margin models are correctly assessing risk, the probability of an exceedance occurring at any two points in time should be the same as the probability of an exceedance occurring at either point in time, individually, without the exceedance occurring at the other point in time. Third, OCC will add the Probtile test, which compares the distribution of the daily observed P&L to the daily forecasted P&L distribution. If the distribution of these P&L ratios approximates a uniform random distribution, this is an indication that OCC's margin models are not providing inaccurate forecasts of potential losses in an account. Combined, these new statistical tests will provide OCC with additional pertinent information to evaluate the effectiveness of its models in determining margin coverage.

2. Data Set Changes

In addition to the changes to its backtesting program, as described above, OCC also will make two enhancements to the data sets being backtested to allow for testing against various assumed portfolio and market data scenarios, in addition to the performance of actual portfolios against actual, current market conditions. First, OCC will backtest hypothetical portfolios, allowing for the design and monitoring of portfolios that have magnified sensitivities to particular aspects of the models used in the margin computations. Backtesting against hypothetical portfolios will provide a more comprehensive insight into the adequacy of the underlying model assumptions under market conditions prevailing in the backtest observation periods.

Under the second data set enhancement, OCC will backtest current accounts against earlier observation periods. The market data observed over the observation period is used to generate the margin forecasts and P&L and observation periods will be chosen to reflect special market conditions. OCC believes this enhancement should be useful because even though margin coverage might be adequate in the current environment, margin coverage could be inadequate under stressed conditions, such as periods of high volatility. The ability to select specific observation periods will not limit the backtesting to the current environments but rather will highlight performance of margin coverage and model performance in market scenarios other than prevailing market conditions.

3. Forecast Horizons Changes

Currently, OCC conducts backtesting using a one-day time horizon, which means that it compares calculated margin with realized P&Lthat occur on the business day following the calculation. However, OCC's margin calculations assume that positions will be liquidated over a two-day period, resulting in the test comparing two-day margin numbers to a one-day P&L calculation. This difference requires OCC to make adjustments to its existing backtesting methodology in its testing to account for the difference between the two-day liquidation period used in its margin calculation and the one-day horizon used in the P&L calculation.

Pursuant to the proposal, OCC will revise its backtesting methodology to take into account losses over a two-day time horizon, which will match the two-day liquidation period used in the margin calculation without such adjustments. OCC will implement the necessary functionality into its backtesting system to conduct a two-day time horizon backtest, which will compare calculated margin against a two-day P&L calculation. OCC also will revise its backtesting methodology to compare one-day margin calculations against one-day P&L calculations, and will implement system functionality for such a test. All issues identified in any of these backtesting results will be reported to the ERMC. OCC believes that its adoption of the additional forecast horizons tests will allow it to have a more accurate view of the sufficiency of its margin methodology.

4. Root Cause Analysis Changes

Currently, OCC's backtesting staff conducts investigations, as necessary, in order to identify the root cause of exceedances. The investigation itself is a manual process that is dependent upon the facts and circumstances pertaining to a given exceedance. Pursuant to its proposal, OCC will now make system modifications that will provide OCC's backtesting staff with additional tools to facilitate such investigations. Specifically, OCC will add system functionality that should reveal attribution of losses due to underlying price movements and implied volatility movements. Further, these improvements will allow OCC to incorporate hypothetical accounts and positions into the tests and will allow OCC to identify risk factors that move above or below the projected values. These changes should improve OCC's ability to conduct investigations and root cause analyses that identify the root cause of exceedances by providing OCC with additional automated investigative tools which should, in turn, lead to improving OCC's backtesting methodology and its margin coverage.

II. Discussion and Commission Findings

Although Title VIII does not specify a standard of review for an advance notice, the Commission believes that the stated purpose of Title VIII is instructive.^[11] The stated purpose of Title VIII is to mitigate systemic risk in the financial system and promote financial stability by, among other things, promoting uniform risk management standards for systemically-important financial market utilities and strengthening the liquidity of systemically important financial market utilities.^[12]

Section 805(a)(2) of the Payment, Clearing and Settlement Supervision Act ^[13] authorizes the Commission to prescribe risk management standards for the payment, clearing, and settlement activities of designated clearing entities and financial institutions engaged in designated activities for which it is the supervisory agency or the appropriate financial regulator. Section 805(b) of the Payment, Clearing and Settlement Supervision Act ^[14] states that the objectives and principles for the risk management standards prescribed under Section 805(a) shall be to:

• Promote robust risk management;
• promote safety and soundness;
• reduce systemic risks; and
• support the stability of the broader financial system.

The Commission has adopted risk management standards under Section 805(a)(2) of the Payment, Clearing and Settlement Supervision Act (“Clearing Agency Standards”).^[15] The Clearing Agency Standards became effective on January 2, 2013, and require registered clearing agencies that perform central counterparty (“CCP”) services to establish, implement, maintain, and enforce written policies and procedures that are reasonably designed to meet certain minimum requirements for their operations and risk management practices on an ongoing basis.^[16] As such, it is appropriate for the Commission to review advance notices against these Clearing Agency Standards, and the objectives and principles of these risk management standards as described in Section 805(b) of the Payment, Clearing and Settlement Supervision Act.^[17]

The Commission believes that the proposal in this Advance Notice is designed to further the objectives and principles of Section 805(b) of the Payment, Clearing and Settlement Supervision Act.^[18] The Commission believes that the additional backtesting improvements should promote robust risk management by providing OCC with additional tools to test the performance of its margin methodology in a more comprehensive manner and better evaluate the effectiveness of its models in determining model coverage. First, the enhancement to OCC's existing Traffic Light Test and the adoption of the three new statistical tests should provide a more comprehensive set of tests for it to use to evaluate its margin models. Second, the enhancement of the data sets to be backtested should provide OCC with additional informative data on the performance of margin coverage and model performance in market scenarios other than prevailing market conditions. Third, revising the backtesting methodology to take into account losses over a two-day time horizon, should allow OCC to have a more accurate view of the sufficiency of its margin methodology. Finally, system modifications that should reveal attribution of losses due to underlying price movements and implied volatility movements should provide OCC with additional, automated investigative tools to conduct analysis into the root causes of exceedances.

In addition, the Commission believes that the proposal in this Advance Notice is consistent with Clearing Agency Standards, in particular, Rule 17Ad-22(b)(4) under the Exchange Act,^[19] which, in relevant part, requires registered clearing agencies that perform central counterparty services establish, implement, maintain, and enforce written policies and procedures reasonably designed to provide for an annual model validation consisting of evaluating the performance of the clearing agency's margin models and the related parameters and assumptions associated with such models. The Commission believes that this proposal is consistent with Exchange Act Rule 17Ad-22(b)(4) ^[20] because it provides OCC with the ability to employ improved statistical tests to better evaluate the performance of its margin models and thus improving its ability to validate such models.

III. Conclusion

It is therefore noticed, pursuant to Section 806(e)(1)(I) of the Payment, Clearing and Settlement Supervision Act,^[21] that the Commission does not object to advance notice proposal (SR-OCC-2014-810) and that OCC is authorized to implement the proposal.

By the Commission.

Robert W. Errett,

Deputy Secretary.

Footnotes

1.  12 U.S.C. 5465(e)(1). The Financial Stability Oversight Council designated OCC a systemically important financial market utility on July 18, 2012. See Financial Stability Oversight Council 2012 Annual Report, Appendix A, http://www.treasury.gov/​initiatives/​fsoc/​Documents/​2012%20Annual%20Report.pdf. Therefore, OCC is required to comply with the Clearing Supervision Act and file advance notices with the Commission. See 12 U.S.C. 5465(e).

2.  17 CFR 240.19b-4(n)(1)(i).

3.  See Securities Exchange Act Release No. 73749 (December 5, 2014), 79 FR 73673 (December 11, 2014) (SR-OCC-2014-810) (“Notice”).

4.  12 U.S.C. 5465(e)(1)(D).

5.  The Commission received a response from OCC with the additional information for consideration on April 29, 2015, which, pursuant to Sections 806(e)(1)(E) and (G) of the Payment, Clearing and Settlement Supervision Act, initiated a new 60 day period of review. See 12 U.S.C. 5465(e)(1)(E) and 12 U.S.C. 5465(e)(1)(G).

6.  See supra note 3.

7.  If OCC determines that the results of these modified backtesting procedures require changes to its margin model, OCC may be required to file an advance notice to effect those changes. See id.

8.  See “Supervisory Framework for the Use of `Backtesting' in Conjunction with Internal Model Approach to Market Risk Capital Requirement.” Located at http://www.bis.org/​publ/​bcbs22.htm.

9.  See, Kupiec, P. “Techniques for Verifying the Accuracy of Risk Management Models,” Journal of Derivatives, v3, P73-84 (1995).

10.  See, Christoffersen, Peter, “Evaluating Interval Forecasts.” International Economic Review, 39 (4), 841-862 (1998).

11.  See 12 U.S.C. 5461(b).

12.  Id.

13.  12 U.S.C. 5464(a)(2).

14.  12 U.S.C. 5464(b).

15.  17 CFR 240.17Ad-22.

16.  The Clearing Agency Standards are substantially similar to the risk management standards established by the Board of Governors of the Federal Reserve System governing the operations of designated financial market utilities that are not clearing entities and financial institutions engaged in designated activities for which the Commission or the Commodity Futures Trading Commission is the Supervisory Agency. See Financial Market Utilities, 77 FR 45907 (August 2, 2012).

17.  12 U.S.C. 5464(b).

18.  12 U.S.C. 5464(b).

19.  17 CFR 240.17Ad-22(b)(3).

20.  Id.

21.  12 U.S.C. 5465(e)(1)(I).