Eurofiling Workshop in Rome Katrin Heinze 5 May2014 The objective of this presentation is to provide an overview on the testing strategy of the SUBA project to explain the test plan ID: 810184
Download The PPT/PDF document "Testing of XBRL files for the supervis..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Testing ofXBRL files for the supervisory reporting
Eurofiling Workshop in Rome
Katrin Heinze
5
May2014
Slide2The objective of this presentation is to provide an overview on the testing strategy of the SUBA project
to explain the test plan
and sample test casesto inform about the next steps
Page
2
Eurofiling
Workshop in Rome, 5th of May 2014
Objectives
Slide31
2
3
Sample XBRL test cases
Next steps
SUBA
*
test strategy
Overview
Page
3
Eurofiling
Workshop in
Rome
, 5
th
of May 2014
* SUpervisory BAnking Data System
Agenda
Slide41
2
3
Sample XBRL test cases
Next steps
SUBA Test
S
trategy
Overview
Page
4
Agenda
Eurofiling
Workshop in Rome, 5th of May 2014
Slide5Functional Tests
Non-Functional Tests
Regression Tests
Based on
User requirements
Use cases
Experience of users
Technical requirements
Source code (unit tests)
…
Categorised following
Load tests (number of users)
Performance tests (processing time)
Volume test (quantity of data)
Stress test
Test of the (data) securityUsability tests…
Done after changes onComponentsXBRL taxonomyValidation rulesInterfacesEXDIRIADInfrastructure
…
Technical Test Execution Status
17.04.2014
SUBA Test Strategy
Eurofiling
Workshop in Rome, 5th of May 2014
Page
5
Slide6HP ALM usage within the SUBA project
Scope
Planning of releases and test cycles
Enable traceability between requirements, tests, defects and according code changes
Increase visibility with dashboards on analysis and KPIs
Achievements
Definition of test cases linked to the defined requirements
Execution of test cases based on defined test data
Defect tracking and documentation of bug fixes
Way forward
Increase the level of automation on test execution
Customise reports and analysis based on the project needs
Preparation of test cases for the next release
SUBA Test Strategy
Eurofiling
Workshop in Rome, 5th of May 2014
Page
6
Slide71. level: Categories
2. level: Use cases
3. level: Test cases
Conventions on numbering
Structure in
the Test Lab
Test plan structure
Eurofiling
Workshop in Rome, 5th of May 2014
Page
7
Slide8Tracking of the test progress in HP ALM
Eurofiling
Workshop in Rome, 5th of May 2014
Page
8
Slide91
2
3
Sample XBRL test cases
Next steps
SUBA T
est Strategy
Overview
Page
9
Agenda
Eurofiling
Workshop in Rome, 5th of May 2014
Slide10Positive functional test casesCreation of test data for each module of the COREP taxonomy
which is a valid XBRL instance document.
Valid according toXML specificationXBRL 2.1 specification
XBRL Dimension 1.0 specification
XBRL Formula 1.0 specification
EBA disabled validation rulesEBA Filing Rules for COREP and FINREP By using different file sizessmallmediumlargeSample XBRL test cases
Eurofiling Workshop in Rome, 5th of May 2014
Page
10
Slide11Negative functional test casesTests that ensure that the application shows a proper behaviour when invalid files are received.
Creation of test data where the system is expected to provide correct error
messages.Possible test cases : A reporting entity sends a file which
is not a well-formed XML document.
has not the correct format, i.e. jpg, txt,
xls etc.has not the expected encoding.includes neither facts nor contexts.misses a namespace in the xbrli root elementincludes letters where numbers are expected as fact valuemisses an explicit member in the scenario elementIncludes a fact with no value but also no nil attribute is providedRefers to a non-existing moduleRefers to two different modules.Sample XBRL test cases
Eurofiling Workshop in Rome, 5th of May 2014
Page
11
Slide121
2
3
Sample XBRL test cases
Next steps
SUBA T
est StrategyOverview
Page
12
Agenda
Eurofiling
Workshop in Rome, 5th of May 2014
Slide13Test scope
Test period:
13
th
to 15
th
of
May
Test activities:
Acquisition
NCA sends data to EXDI via
WebUI or standard adapter.EXDI system sends a delivery message.SUBA system sends an acknowledgement message to NCA with the XBRL validation result NCA receives the messages in the scheduled time periods.
Eurofiling Workshop in Rome, 5th of May 2014
Page
13Next stepsVoluntary functional tests with NCAs
Slide14Load testsIT infrastructure for establishing a supervisory function at the ECB14
Next steps
First estimation about the load expected for the first remittance date
Number of reported entities
128
First remittance date NCAs to EBA/ECB
31.07.2014
Modules
reported
Number of files (per institute)
Max. file size in MB
Max. number of observations
COREP consolidatedOwn Funds1115
177.536Large Exposures1264355.930NSFR197.510
Total
51249.66469.244.928
*) Estimations based on information of the Deutsche Bundesbank for one big German bank
Slide15Organisation and approach – Test case preparation
Control point
MetricTarget (by priority)
Average
High
MediumLowA% reviewed requirements100%100%100%
100%B% requirements test coverage75%100%100%50%C% tests run90%100%
100%
75%
% tests succeeded
100%
100%100%100%Defect resolution time (working days)335Defects reopened ratio5%
5%10%
Control point A
Control point B
Control point C
Eurofiling Workshop in Rome, 5th of May 2014
Page
15
Next steps
Slide16Key issues and challenges
Load tests
Defining scenarios for load tests on XBRL processing via automated test, simulating also peaks
Preparation for next releases
Defining test cases based on more precise user requirements
EBA testing
Participating in tests scheduled by EBA
Functional tests
Supporting
the voluntary functional tests with the NCAs, organized by WG SIM
Eurofiling
Workshop in Rome, 5th of May 2014
Page
16
Next steps