Media stack deep dive Module Overview Overview of

Published  . 0 views
↓ Download
Media stack deep dive Module Overview Overview of
1 / 1
Media stack deep dive Module Overview Overview of - slide 1 of 43 Media stack deep dive Module Overview Overview of - slide 2 of 43 Media stack deep dive Module Overview Overview of - slide 3 of 43 Media stack deep dive Module Overview Overview of - slide 4 of 43 Media stack deep dive Module Overview Overview of - slide 5 of 43 Media stack deep dive Module Overview Overview of - slide 6 of 43 Media stack deep dive Module Overview Overview of - slide 7 of 43 Media stack deep dive Module Overview Overview of - slide 8 of 43 Media stack deep dive Module Overview Overview of - slide 9 of 43 Media stack deep dive Module Overview Overview of - slide 10 of 43 Media stack deep dive Module Overview Overview of - slide 11 of 43 Media stack deep dive Module Overview Overview of - slide 12 of 43 Media stack deep dive Module Overview Overview of - slide 13 of 43 Media stack deep dive Module Overview Overview of - slide 14 of 43 Media stack deep dive Module Overview Overview of - slide 15 of 43 Media stack deep dive Module Overview Overview of - slide 16 of 43 Media stack deep dive Module Overview Overview of - slide 17 of 43 Media stack deep dive Module Overview Overview of - slide 18 of 43 Media stack deep dive Module Overview Overview of - slide 19 of 43 Media stack deep dive Module Overview Overview of - slide 20 of 43 Media stack deep dive Module Overview Overview of - slide 21 of 43 Media stack deep dive Module Overview Overview of - slide 22 of 43 Media stack deep dive Module Overview Overview of - slide 23 of 43 Media stack deep dive Module Overview Overview of - slide 24 of 43 Media stack deep dive Module Overview Overview of - slide 25 of 43 Media stack deep dive Module Overview Overview of - slide 26 of 43 Media stack deep dive Module Overview Overview of - slide 27 of 43 Media stack deep dive Module Overview Overview of - slide 28 of 43 Media stack deep dive Module Overview Overview of - slide 29 of 43 Media stack deep dive Module Overview Overview of - slide 30 of 43 Media stack deep dive Module Overview Overview of - slide 31 of 43 Media stack deep dive Module Overview Overview of - slide 32 of 43 Media stack deep dive Module Overview Overview of - slide 33 of 43 Media stack deep dive Module Overview Overview of - slide 34 of 43 Media stack deep dive Module Overview Overview of - slide 35 of 43 Media stack deep dive Module Overview Overview of - slide 36 of 43 Media stack deep dive Module Overview Overview of - slide 37 of 43 Media stack deep dive Module Overview Overview of - slide 38 of 43 Media stack deep dive Module Overview Overview of - slide 39 of 43 Media stack deep dive Module Overview Overview of - slide 40 of 43 Media stack deep dive Module Overview Overview of - slide 41 of 43 Media stack deep dive Module Overview Overview of - slide 42 of 43 Media stack deep dive Module Overview Overview of - slide 43 of 43
Description: Media stack deep dive Module Overview Overview of Media Platform Audio Processing Components Media Improvements SDP Codec Details Mediation Server Bandwidth Estimation Which Factors Influence Media Quality? Audio and video coding Network

Related Topics

Download Presentation

"Media stack deep dive Module Overview Overview of" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

slide1. Media stack deep dive<br>
slide2. Module Overview Overview of Media Platform
Audio Processing Components
Media Improvements
SDP
Codec Details
Mediation Server
Bandwidth Estimation<br>
slide3. Which Factors Influence Media Quality? Audio and video coding

Network performance
Latency (delay and jitter)
Packet loss
Low and/or variable throughput Device
Low-resolution screen
Low processing capability
Environment
Noise
Lighting<br>
slide4. Mission
Empower people through high-quality and easy-to-use real-time audio/video technologies, any time, any place, and on any device
Designed to
Capture audio/video data and transmit it over IP networks for remote rendering with high quality
Mix and route audio/video to enable multiparty conferencing with high scalability/quality
Smart adaptive endpoint algorithms to adjust to varying network and device conditions
Provide high level of connectability through advanced NAT/Firewall traversal technologies Overview: Purpose of Media Platform<br>
slide5. Audio Processing Conceptual Architecture<br>
slide6. Audio Processing Pipeline on client Overview: Components<br>
slide7. Analog AGC: Analog Automatic Gain Control to make sure that the microphone signal level is appropriate for the Analog to Digital Conversion
AEC: Acoustic Echo Cancellation
NS: Noise Suppression, suppresses stationary background noise such as noises from the air conditioner or the computer hard fan etc.
VAD: Voice Activity Detection
SS: Silence Suppression. We encode only non-stationary signals, stationary noise is classified as silence by the SS, will not be transmitted to the far end.
Digital AGC: Automatic Gain Control in the digital domain to ensure send speech signal at a pre-determined level of -24dBoV. This will make sure that the far end receive sufficiently loud signal without clipping the speech.
CNP: Comfort Noise Packet. When silence suppression kicks in, a CNP will be sent at the end of a talk spurt to signal the decoder the end of the talk spurt and the energy level of the subsequent comfort noise used to avoid absolute silence. Overview: Components<br>
slide8. RTP: Real-time Transport Protocol that defines a standardized packet format for delivering audio and video over the internet.
SRTP: Secure RTP (encrypted RTP)
RTCP: Real-time Transport Control Protocol. Provides out-of-band control and statistics for an RTP flow.
Audio Healer: This block is used as post processing of decoded speech to compensate for packet loss or jitter.
Codecs: From code/decode, an algorithm to encode voice for digital transmission.
FEC: Forward Error Correction. Each RTP packet contains a redundant audio encoding of previous data. This is used to recover the audio if a previous packet was lost.
PLC: Packet Loss Concealment. This is digital signal processing to conceal lost or missing audio data by manipulating data that has been received so the listener won’t realize audio has been lost. Overview: Components (con’t)<br>
slide9. Main difference of jitter and loss
Average RTP inter frame jitter is much higher than that of wired networks (5 to 10)
Large burst loss happens from time to time while the average loss rate is reasonably low
Packet loss for WiFi and mobile networks could likely be caused by contention ratio. High network contention ratio will likely cause packet collisions in the wireless networks.

Network Planning, Monitoring, and Troubleshooting with Lync Server
http://www.microsoft.com/en-us/download/details.aspx?id=39084 Understanding WiFi and Mobile Networks<br>
slide10. When packets arrive early or later than 20 ms, it is called RTP inter frame jitter.
The jitter will cause the audio healer to drop the packets even when the packets finally arrive
This is equivalent to packet loss at the receive side. Average RTP Inter Frame Jitter<br>
slide11. Audio healer will delay triggering the FEC usage when first few burst losses happen but the longer term average loss rate is low.

Audio healer will not trigger FEC when burst losses (consecutive loss more than one frame) are larger than 3 frames

FEC is only applied up to distance of 3 frames. It is helpful when the packet loss is between 1 frame and 3 frames. When burst loss is larger than 3 frames, FEC won’t help much. FEC scheme for WiFi and Mobile Networks<br>
slide12. The additional jitter control logic in audio healer is needed to accommodate the large RTP inter frame jitter for WiFi and mobile networks.

The audio healer will increase jitter buffer size when large jitter is detected.
Increase jitter buffer size quickly when increase of RTP jitter detected
Decrease jitter buffer size slowly to avoid fluctuation of the jitter buffer size. Jitter Buffer control for WiFi / Mobile Networks<br>
slide13. Jitter Buffer<br>
slide14. Much of the processing in the audio pipeline introduces delay. Here are the major contributors: Latency Introduced by Audio Processing Capture device latency OS audio buffering Filterbank analysis AEC buffering Codec delay CPU processing Networking Codec delay CPU processing Networking Audio healing buffer CPU processing Networking Audio healing buffer Render buffering OS audio buffering Render device latency Capture side Media Server Render side Network latency Network latency Device OS Media
Platform HW OS/HW<br>
slide15. Audio Quality – Comfort Noise Comfort noise packet (CNP) support
CNP for Narrow Band and Wide Band according to RFC 3389
When silence suppression kicks in, a CNP will be sent at the end of a talk spurt to signal the decoder the end of the talk spurt and the energy level of the subsequent comfort noise used to avoid absolute silence.<br>
slide16. User facing diagnostics Events shown to user and logged in to QoE for audio quality issues Half duplex
High noise/low input speech level
Low near end to echo ratio (poor double talk quality)
Render glitch
Poor receive quality due to network and CPU
Poor send quality due to network
Echo caused by end point during conference call
Howling and multiple end point in the same room
Device (capture/render) not working
Render mute/zero volume
Client health (low CPU/power saving mode)<br>
slide17. First in industry with detailed audio quality requirements for GW performance
Fully automated testing combining Sage Equipment with media software
Automatic report/pass-fail GW Audio Quality Certification Team<br>
slide18. UC Audio Quality Certification Program Certification program for UC audio devices including
Headset/handset
Speakerphone
PC/laptop
Ensures devices work well with Lync and meet customer experience needs. Areas:
Acoustics (loudness, frequency response, noise and distortion, coupling, directivity, latency)
Driver, clock sync
Timestamp measurement
TIA and IEEE standard based and expanded for PC platform
Test tool:
UCDC by MSFT
HEAD acoustics<br>
slide19. Session Description Protocol<br>
slide20. Signaling & Media Every call has two parts PROXY UAC UAC PROXY Media Signalling<br>
slide21. Session Description Protocol SDP describes the Media Session in a call.
SDP is intended for multimedia management - session invitations, acceptance, modifications.
An SDP session description includes the following:
Session name and purpose
Time(s) the session is active
The media comprising the session
Information needed to receive those media (addresses, ports, formats, etc.)<br>
slide22. Session Description Protocol Describes multimedia sessions for announcement or initiation.
SIP INVITE from initiating endpoint
SIP “200 OK” or “183 Session Progress“ from other endpoint
Content-Type: application/sdp
Described in RFC 4566
Parameters are registered with IANA http://www.iana.org/assignments/sdp-parameters<br>
slide23. The Protocol SDP consist of number of textual lines in the format
<type> = <value>
Where type is a single case sensitive character. Value is structured text depending on the type.
A SDP session description may be divided into
Session-level descriptors
Media-level descriptors
The session-level part starts with a "v=" line and continues to the first media-level section.
Each media-level section starts with an "m=" line and continues to the next media-level section or end of the whole session description.
In general, session-level values are the default for all media unless overridden by an equivalent media-level value.<br>
slide24. The Protocol<br>
slide25. SDP Example<br>
slide26. Recent change (April 2015 Update) Enabled multiplexing of RTP and RTCP over a single UDP Port
Align with WebRTC RFC
Addresses some Firewall traversal issues
will not affect supported QoS setup<br>
slide27. UDP Multiplexing Sender
a=rtcp-mux and two a=candidate: lines
a=rtcp: line indicating the fallback port

Receiver
a=rtcp-mux and a single a=candidate: line<br>
slide28. SDP Examples<br>
slide29. Codecs<br>
slide30. Codecs: Primary Usage Skype for Business
SILK
Used for peer to peer calling (wideband)
RTA
Fallback for peer to peer calling, calls to PSTN
G.722
Used for conferencing with Lync client
G.722/2
Used for Lync Room System Conferencing
SIREN
Used when conferencing in Live Meeting or Office Communicator
G.711
Used between Mediation server and SIP/PSTN Gateway or PBX.
Used for PSTN participants in A/V conferences
Exchange UM
G.711, RTAudio, G.723<br>
slide31. SILK in Skype for Business Scenarios
With Skype for Business, SILK will be the default codec for all 1:1 communications
RTA is still available for back-compatibility
SILK not supported for single-core CPU
Clients Supported
Lync 2013 November Update (CU4) or higher, Skype for Business 2015
Mobile: Android, iPhone, iPad, Windows Phone
Not in the Server media stack today
Unsupported Clients
VDI Plug-in, Lync Web App, Attendant, Lync Phone Edition<br>
slide32. S4B to Skype Federated calls SILK WB with a target bit-rate of 36 kbps will be the default audio codec for federated skype calls.

Replaces G.722 @ 64 kbps

Lync 2013 CU4 (Nov 2013) and higher will use SILK. Older clients will continue using G.722.<br>
slide33. Codecs negotiated

SILK SDK available under royalty-free license for partners who wish to implement decoding tools SILK Codec<br>
slide34. SILK Codec: bit-rate and MOS Reduced bit-rate compared to G.722

Improved quality (MOS) MOS calculated using ITU-T P.863 POLQA<br>
slide35. Codecs: Bandwidth<br>
slide36. Codecs: Packetization Time Packetization time is duration of audio sent in each packet.
Increasing packetization increase delay, decreases network bandwidth 57Kbps 43 Kbps
+20 ms delay 38.3 Kbps
+40 ms delay Packet overhead numbers includes:
IP, UDP, RTP, SRTP headers/footers<br>
slide37. Codecs: Forward Error Correction Lync uses two forms of FEC to protect against packet loss:
Sending redundant audio data (RED payload type)
For RTAudio, additional redundant bits in audio payload (3kbps) to help decoder recover from loss
For RED packet:
Sender sends one previous audio payload
Either the n-1, n-2 or n-3 audio payload (distance)
Usage is dynamic and is based on healer needs
Uses payload type 97. Audio codec payload type is included inside RED header.<br>
slide38. Codecs: Redundant payload (FEC) Main
payload
t = 40ms RTP Header Redundant payload
t = 0ms Main payload
t = 60ms RTP Header Redundant payload
t = 20ms Main payload
t = 80ms RTP Header Redundant payload
t = 40ms Main payload
t = 100ms RTP Header Redundant payload
t = 60ms Sent @ t=40ms Sent @ t=60ms Sent @ t=80ms Sent @ t=100ms 2 packets lost Sender Main
payload
t = 20ms Redundant payload
t = 40ms Redundant payload
t = 60ms Main payload
t = 80ms Main payload
t = 100ms Main
payload
t = 20ms RTP Header Redundant payload
t = -20 ms Sent @ t=20ms Receiver Network FEC distance = 2, packetization = 20ms<br>
slide39. Bandwidth and Codec selection Media Platform is responsible for selecting which codec to use (from negotiated set)
Selection is based on codec priority and bandwidth
Bandwidth is obtained
Policy limit (CAC, in-band)
Bandwidth estimation
Selection determines:
Codec
Packetization time
Bit rate (if a VBR codec is selected)<br>
slide40. Codec Switching Since selection is from set of codecs negotiated in SDP, there is no Re-INVITE.
Switches:
Between codecs happens during periods of silence to avoid glitches (or after ~20-30s).
Between packetization time and bitrates happen on a per packet basis.
Codec switches are identified by a change in payload type in the RTP header.
QoE reports contain the last codec used for a call<br>
slide41. Mediation Server: G.711 to PSTN In OCS 2007: RTAudio NB used for PSTN calls
In OCS 2007 R2: Dynamic selection of G.711 for calls over LAN to Mediation Server for P2P call based on RTT
LAN Detection for OC-PSTN calls based on RTT (<20ms). Calls to PSTN will start with RTAudio NB while RTT and bandwidth checks are made (5-15s). Switches occurs during periods of silence.
In Lync 2010 RTM – CU3: G.711 selected from call start based on CAC configuration.
No RTT test. Does not begin with RTAudio.
In Lync 2010 CU4 / Lync 2013: G.711 selected from call start based on CAC configuration
WAN detection based on RTT (>20ms) will cause switch to RTAudio NB.<br>
slide42. Bandwidth Estimation Must estimate available bandwidth to make appropriate decisions for media
Wideband vs. narrowband, target bit rate, packetization, forward error correction, G.711 use
Performed by sending pairs of RTCP packets and measuring time to receive both.
Limited in ability to adjust quickly to large changes in bandwidth.<br>
slide44. Questions?<br>