Economic principles and building blocks of a

Transcript:Economic principles and building blocks of a:
Economic principles and building blocks of a regulatory benchmarking framework Cognitive Analytics Management Conference 2018 Dr Srini Parthasarathy Thursday, 29 November 2018 Strictly confidential Strictly confidential 2 Five themes Framework considerations Model development Methodology Model validation Calibration of results 3 Strictly confidential legal considerations, and regulatory duties and objectives provide an overarching framework for benchmarking work. For example, some frameworks require that benchmarking should: explore ‘state of the art’ methods in an ideal world with perfect data to then decide on which are superior in a real world with imperfect data focus on strengthening incentives for the industry rather than company-specific efficiency requirements achieve better delineation between efficient and inefficient operators different approaches may produce different outcomes that need to be integrated into the broader regulatory and legal framework. For instance, some approaches: do not provide estimates of components of efficiency (static/dynamic efficiency) are more robust at estimating (large) inefficiency gaps and providing challenging targets focus on certain notions of efficiency (e.g. scale efficiency to examine savings from industry consolidation) key constraints are typically industry structure/maturity, data availability and time/resource availability, but: always feasible to undertake (some) benchmarking with the understanding that contexts/tools evolve framework must not be overly prescriptive to allow for deviations where justified framework must (periodically) be examined by a body of experts to ensure it remains ‘fit for purpose’ Framework considerations Legal framework, regulatory objectives, industry maturity 4 Strictly confidential emphasis on economics and engineering knowledge—applications typically deal with limited data (however large the number of regulated entities in a jurisdiction). So economics and engineering knowledge should trump statistical tests transparent engagement/consultation with the industry and other stakeholders should be encouraged (and this goes both ways). This helps with trust in the system, mitigating information asymmetry and can improve the overall quality of the assessment robust and consistent application of statistical methods needs to take into account the efficiency context. For example, hypothesis testing using standard regression methods is generally invalid in the presence of inefficiency. Also, a ‘pick ‘n’ mix’ approach to statistical tests needs to be robustly justified. Given assumptions underpinning statistical tests, they should be seen as indicative than conclusive no perfect models and approaches, and there should be genuine openness to alternative considerations and assumptions (the onus is on the developers to ensure alternatives are justifiable) balanced take on factors affecting industry-wide costs is needed so that the models are not biased