2 Agenda Technology changes Testing techniques Latest testing methodologies Sharing experience TOOLS AND METHODOLOGIES TOOLS AND METHODOLOGIES Faisal Ghazaleh ITUT QSDG Dubai November 2014 ID: 667154
Download Presentation The PPT/PDF document "ITU-TQSDG – DUBAI Methodologies and ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
ITU-TQSDG – DUBAI Methodologies and Tools for Auditing the Quality of ServiceSlide2
2
Agenda
Technology changes
Testing techniques
Latest testing methodologies
Sharing experience
[
TOOLS AND METHODOLOGIES] Slide3
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide4
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide5
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide6
Global Wearable Devices Traffic Impact
:51 Percent of Total Mobile Data Traffic Will Be 4G by 2018
Mobile Data Traffic and Offload Traffic, 2018
Data Hungry Application
D
istribution
Global Mobile Data Traffic Forecast by Region
MESSAGE: DATA TRAFFIC EXPLOSION
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide7
ANSWER
: UNDERSTAND THE IMPACT OF THE NEED TO FOCUS ON CUSTOMERS
Subscribers
focus on their perceived experience
with offered service (e.g. voice, mobile video, gaming
), and
operators need to cost efficiently manage and control complex 4G
networks
while coping with high traffic growth
The wireless operators’ problem at its core:
4G ecosystem deployments allow very high data
rate apps
efficiently delivered
to a broad range of
devices. This raises subscriber expectations for fixed-line-like service experience (where mobility is not an excuse). This is fundamentally driving the capacity (
QoS
) crunch inside the network. Therefore: operators face a continuous struggle to maintain high
QoE
with continual capacity constraints due to spectrum and cost limits.
Solution:
Customer experience centric network testing and monitoring, and network optimization process automation
Multi RAT
Multi core
IMS**
RCS*/
Mobile Cloud
*IP Multimedia
Subsystem
**Rich
Communication Suite
MESSAGE: User satisfaction vs. operators challenges
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide8
[
TOOLS AND METHODOLOGIES]
Advance offerings and complicated networks
Kee
p the testing methodology s
imple
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide9
[
TOOLS AND METHODOLOGIES]
Reproducing the
same environment
requires to test like a customer using same application on the same device in the geographical location of the customer.
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide10
Moving closer to the
QoE
Moving to the cost efficiency
Moving to the could
Where the T&M is heading
Metrics
Licenses
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide11
KQIs/KPIs consistency across the different operators.
Time to publish the results and make available online.
Accuracy of the results.
Frequency of the test and cost of the benchmarking/auditing.
Varity of the offered services like
VoLTE
, OTT,
VoHSPA
, and video-audio services RCS-e.
Regulator challenges
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide12
SHARING EXPERIENCE: UNFIED SOLUTION and kpis/qkis Slide13
Global practicies by telecom regulators
[
TOOLS AND METHODOLOGIES]
PSTN
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide14
Global practicies by telecom regulators
[
TOOLS AND METHODOLOGIES]
PSTN
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide15
ANSWER: TEST THE NETWORKS LIKE A REAL CUSTOMER IS USING
IT: A CUSTOMER CENTRIC TESTING SOLUTION (ODM)
Laptop-based drive test solution
maintains
control and coordination through an On-Device Server
Server resides inside the terminal, managing the supported services (VoLTE, IP Logging, Call Control etc)
Repeatable “control scripts” supported across multiple devices (repeatable testing)
Enables testing of end user terminal QoS settings and IP stack characteristics
Provides framework for future services to be added easily and controlled by a single client (e.g. Blixt implementation)
UE
On-Device
Server
Service B
Call Control
Service A
VoLTE
Service M
IP Logging
TCP/IP (SIP, RTP, etc)
QoS setting per service
QoS
First to test
VoLTE-ViLTE
as a user
via on-device
VoLTE
client , test everything ON Device
MESSAGE: testing philosophy: to be highly
Innovative by designing measurement metrics
suited to the 4G/5G world: ODM solution
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide16
Use collected device data to benchmark your network performance against competitors.
Benchmark network coverage against competitors to ensure
customers receive top-quality service indoor and outdoor.
BENCHMARK LIKE A USER
Quickly see subscriber experience issues
using handset score that aggregates dropped and
blocked calls, handover failure, downlink/uplink
throughput analyses.
Aggregate handset data analytics. Clearly display the best
and worst performing handsets using six key metrics.
Faisal
Ghazaleh ITU-T QSDG, Dubai, November 2014Slide17
SHARING EXPERIENCE: customer agentsSlide18
Monitor a targeted group of end users based on the complains like a users in a certain location/with a certain mobile phone.
Regulators do not have access to the OSS data of the operators.
Interact with the end user right after the event.
Reaching the customer wherever they are across the country.
Regulator challenges
[
TOOLS AND METHODOLOGIES]
Faisal Ghazaleh
ITU-T QSDG, Dubai, November 2014Slide19
Engineering targeting – visualized through heat-maps
19
Faisal
Ghazaleh ITU-T QSDG, Dubai, November 2014
Global practicies by telecom regulatorsSlide20
20
Indoor / Outdoor
Faisal
Ghazaleh ITU-T QSDG, Dubai, November 2014Slide21
Single / Multi probe analysisTest drill down and replay
Cell info on map21
LIGHT drive testing
Faisal
Ghazaleh ITU-T QSDG, Dubai, November 2014Slide22
Wide range of service testing in both interactive mode as well as a remote controlled probe
22
Handset functionality
Faisal
Ghazaleh ITU-T QSDG, Dubai, November 2014Slide23
SHARING EXPERIENCE: innovatie new kqiSSlide24
Throughput values for DL ABM correlate extremely well to FTP
For ABM, DL and UL throughput and delay are obtained simultaneously
BLIXT™ ABM COMPARED TO FTP (DL EXAMPLE)
FTP intrusiveness= 100%
ABM intrusiveness* = 5%
95%
less resources
LEGEND
Blixt actual PHY (low)
FTP actual PHY (high)
Blixt app layer throughput
FTP app layer throughput
Blixt by Ascom © ASCOM 2014
24Slide25
ABM Accuracy AND LEVEL OF INTRUSIVENESS
Blixt by Ascom © ASCOM 2014
Measurements done in a perfect radio environment
Effective
zone
of ABM
Less
suitable
The accuracy displayed in the table on this slide is a comparison of
the
Blixt
™ result as compared to the
actual
available
bendwidth
in the network under test.
It is not a comparison with FTP!
FTP as a test method is likely to be significantly less accurate than
Blixt
™ under non-ideal conditions.
25Slide26
“Is it not better to use FTP since it is a standard application and better represents the user experience?”
Answer: FTP is clearly not an application commonly used by smartphone users. The objective of testing available bandwidth is to determine this characteristic of the network, and not how well FTP performs in the network on a specific device. This is essentially what the FTP legacy method measures.
“How do we know the results are accurate?”
Answer: Other customers have tested the technology and found it to be accurate, In reality, FTP is less likely to give an accurate result as it is more susceptible to be negatively affected by conditions of the test session not related to the available bandwidth.
“We are happy with our current test methodology, why would we change?”
Answer: With legacy technologies such as 3G, FTP and similar legacy methods may be perfectly adequate since the available bandwidth is modest.
With LTE bandwidth approaching or exceeding 100 Mbps, these legacy methods become at best highly impractical and at worst impossible to use, for a variety of reasons. Please see “
Speedtest
vs
Blixt
™” comparison.
Common Questions Answered
Blixt by Ascom © ASCOM 2014
26Slide27
CONCLUSIONS
PLEASE VISIT OUR WEBSITE:
http
://
www.ascom.com/nt/en/index-nt/about-us-network-testing/nt-about-us-resources.htm/
White papers:
VoLTE
, Video Streaming,
HetNets, Carrieir Aggregation and...watch the space:
eMBMS
testing to come soon
Informa
Webinar: Advanced testing with Ascom in LTE networks
Webinars:
VoLTE
Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014Slide28
QUESTIONS ?Slide29
Thank you
Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014Slide30
CONCLUSIONS
A FULL 24/7
QoE
/QOS CYCLE SOLUTION FOR MOBILE NETWORK AND SERVICES DESIGNED TO MEET CUSTOMERS’ SATISFACTION
Automated data correlation
and presentati
on
Built-in best practices scripted data analytics provide automated root cause analysis
Providing network and customer experience -centric diagnoses for voice (
VoLTE
, OTT,
VoHSPA
), and video-audio services (OTT, RCS-e/
Joyn
)
My competitors’ customers (Benchmark)
My customers, real field agents
My Customer Experience Data
Automated “what,” “why” and engineer- trusted “how” scenarios
Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014