WHITEPAPER
Infovista OTT video streaming testing
The generic framework comprises two testing steps:
• Scripting, which contains user interface (UI) actions and trigger points for events, to be sent to on-device measurement (ODM) to generate measurement events and KPIs per streaming application, but with shared ODM service and KPIs. For example, scripting tasks can be video search, ‘Streaming Video IP Service Access Time’ event generation. • IP Sniffing, working on commercial devices embedded in the generic framework for ODM IP recording, to provide KPI triggers and payload information for throughput calculations. The KPIs are defined by ETSI (ETSI TR 101.578, ETSI TS 102.250-2), and they are reported per session and refer to session establishment/session set-up handling, streaming, up/downloading, posting, messaging, as well as video quality during presentation (playback). In addition, they are common to all applications, and thus agnostic to the native OTT application. In the case of OTT video streaming, these KPIs can be used to determine the waiting time, session retainability and playback video quality perceived by the user (Figure 1). Examples of such ETSI-based KPIs are presented in Table 2.
With extensive experience in on-device testing, OTT voice and video QoE testing, and a highly qualified understanding of operators’ network performance and concerns, Infovista has developed a pragmatic video quality testing solution for OTT video streaming, based on the criteria mentioned in section “ A pragmatic testing solution ”. Generic framework for testing OTT video streaming applications As part of the generic testing strategy , Infovista developed an on-device generic framework for testing a variety of native OTT video streaming applications, such as YouTube, Netflix, TikTok, Facebook and others. Infovista’s generic framework for native OTT application testing ensures consistency and efficiency through: • Automated and fast testing using one script to collect field data for several OTT applications in a single drive test • A common set of KPIs for all tested native OTT applications, as defined by ETSI
• Generic definitions of triggering KPIs measurements for all tested native OTT applications, as defined by ETSI
TYPE
KPIS
QOE DIMENSION
Streaming video play start time, failure Streaming reproduction start delay, failure Streaming video IP service access time, failure ratio Streaming reproduction cut off ratio Streaming video interruption duration, percentage Streaming aggregated average session resolution Streaming video resolution changes Number of positive/negative resolution changes CDN download application throughput Application layer DL throughput
Perceived waiting time Perceived retainability
OTT video session quality
Perceived video quality during presentation/playback
OTT video quality
Video provider Player type Subscriber IP CDN transport protocol CDN media server IP address
OTT application configuration
n/a
Table 2. Examples of OTT video streaming KPIs.
8
Powered by FlippingBook