Technical Helpweb

Dialogic® PowerMedia™ Extended Media Server (PowerMedia XMS) - more articles

Tuning tips for servers and virtual machines to achieve low latency

Introduction

Latency is a key concern when designing and configuring a real-time communication environment. Although voice-based interactions can tolerate some level of delay, higher latency can impair communications and lead to lower user satisfaction. In the past, systems and applications that provided low latency services, or services that required precise timing, such as media processing, signal processing, etc, were typically executed on proprietary hardware and with proprietary real-time operating systems. In order to limit costs and design complexity, as well as increase the performance of traditional computing platforms such as the Intel-compatible PC, many services are being migrated to off-the-shelf hardware running Windows & Linux. The drive to reduce costs and manageability further has led to these services moving to virtual machines and into the Cloud.

Achieving low latency is still a difficult requirement when implementing services and applications. These applications can sometimes behave unpredictably in new environments for various reasons, some of which include:

- variable CPU clock speeds are used in order to save energy;
- the use of modern server architectures such as NUMA (Non-uniform memory access);
- virtualization of the hardware and clocking system - when running in a virtual machine, hardware resources are shared between all the virtual machines running on the physical server;
- complex network architectures that provide steady network streams equally to local peers and peers located on the other side of the world.
 

Recommendations

It is usually best to leave every system initially configured with factory default settings. Only if you observe a well-defined issue that is attributable to latency or a performance bottleneck, is it advisable to consider optimizing the environment. When doing so, bear in mind that optimization can be, and usually is, an iterative process.

Below are links to documents published by various vendors including Dialogic, which detail their recommendations to achieve low latency when using their products. This is by no means a comprehensive list, but will be useful to go through when tuning every aspect of the environment in order to make it as predictable and stable as possible, whilst achieving desirable latency results.

Note that Dialogic is not responsible for the content on any third-party websites listed below.

Dialogic^® PowerMedia™ Host Media Processing Software/Dialogic^® PowerMedia™ Extended Media Server
- Recommendations for PowerMedia HMP and PowerMedia XMS running on vSphere/ESXi, but applicable to any application requiring low latency on vSphere/ESXi - see Section 1.52 of the HMP Release Update for information on virtualization support and VMware ESXi 5.0 support.

- Guidelines and recommendations for tuning a system that has HMP software or a system that also contains Dialogic hardware - see Chapter 10 of the Configuration Guide

vSphere - Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs - White Paper

HP - Configuring and Tuning HP ProLiant Servers for Low-Latency Applications - White Paper 

Dell - Configuring Low-Latency Environments on Dell PowerEdge 12th Generation Servers - White Paper 
      - Configuring Low-Latency Environments on Dell PowerEdge Servers - White Paper

Network Tuning

- Use the latest network drivers available from the vendor of your network card
- Enable hardware assistance on network card where applicable
- Check network statistics for signs of dropped packets or bad packets. This can be done with netstat -s for IP statistics (Linux and Windows),  ethtool -S eth0 for Ethernet statistics on Linux and netstat -e for Ethernet statistics on Windows.
- Check network performance with open source tools such as iperf, netperf, pktgen, etc.

Recommendations for tuning network cards under Linux

Tuning 10Gb network cards on Linux - a basic introduction to concepts used to tune fast network cards - White Paper
Red Hat Performance Tuning Guide - see the section on Network Performance Enhancements

Recommendations for tuning network cards under Windows

High Performance Network Adapters and Drivers - MSDN
Networking Adapter Performance Guidelines - TechNet blog

Vendor-specific recommendations

Intel - Tuning Intel Ethernet Adapter throughput performance - Article and FAQ

Mellanox - Performance Tuning Guidelines for Mellanox Network Adapters - White Paper

Legal Note

This document discusses one or more open source products, systems and/or releases. Dialogic is not responsible for your decision to use open source in connection with Dialogic products (including without limitation those referred to herein), nor is Dialogic responsible for any present or future effects such usage might have, including without limitation effects on your products, your business, or your intellectual property rights