Configuring endpoints to use audio/video synchronization

You can use RTCP to communicate skew information to the host application and IP destination through RTP endpoints. To use RTCP in this way, enable RTP endpoints as follows:

Enable RTP receive endpoints to calculate audio and video skew values for incoming data streams and communicate these values to the host application.
Enable RTP video send endpoints to communicate video skew values to the IP destination.

You can enable RTP endpoints to provide skew information either during endpoint creation or after endpoint creation.

Note: A full-duplex RTCP session is only supported with full-duplex RTP endpoints. A simplex RTP endpoint pair (simplex send endpoint and simplex receive endpoint) does not support a full-duplex RTCP session.

Enabling RTP endpoints to detect and communicate skew values for incoming data streams

Note: This feature is for all audio and video full-duplex and simplex receive RTP endpoints. The example shown is for full-duplex endpoints.

During endpoint creation

When creating an endpoint, you can configure the startRtcp bit field in the RTCP_ENDPOINT_PARMS (or RTCP_V6_ENDPOINT_PARMS) structure so that an endpoint can detect and report skew offset values for incoming data streams to the host application. The endpoint communicates these values by sending MSPEVN_SKEW_OFFSET events to the application.

Use the following macros to configure the startRtcp bit field:

RTCP_ENABLE to enable RTCP for the endpoint.
RTCP_ENABLE_RCV_SKEW_CALC to enable the calculation of skew offsets for the endpoint.

You must configure both endpoints in an audio/video stream pair in order to obtain meaningful skew data. For information about using the returned information to calculate audio/video skew, see Calculating audio/video skew.

For more information about creating and configuring RTP endpoints, see RTPRTCP_ENDPOINT_PARMS or RTPRTCP_V6_ENDPOINT_PARMS, the MSPP Developer's Reference Manual, and the Fusion Developer's Manual.

After endpoint creation

After endpoint creation, you can use the MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command against an endpoint so that it can detect and report skew offset values for incoming data streams to the host application. This command uses the msp_ENDPOINT_RTPFDX_ENABLE_SKEW_CALC structure to indicate whether to enable or disable the skew offset calculation.

You must use the MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command against both endpoints in an audio/video stream pair in order to obtain meaningful audio/video skew data. For information, see Calculating audio/video skew.

The MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command can return the following events:

Event	Description
MSPEVN_SENDCOMMAND_DONE	Indicates that the MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command was successfully sent to the specified RTP endpoint on the CG board.
MSPEVN_SKEW_OFFSET	Unsolicited event indicating the timing offset (in ms) for the incoming data stream.

For information about creating and sending MSPP commands, see Creating and sending MSPP commands.

Calculating audio/video skew

The following table describes how an application can calculate the audio/video skew value based on information sent from both endpoints in an audio/video stream pair:

Step	Action
1	First endpoint in an audio/video stream pair calculates a skew offset value based on the RTP packet and RTCP sender report timing of an incoming stream.
2	CG board uses an unsolicited MSPEVN_SKEW_OFFSET event to report the skew offset value to the application. The MSPEVN_SKEW_OFFSET event contains the following structure: typedef struct { DWORD FilterId; DWORD Offset; //Offset value in ms } msp_RTP_SKEW_OFFSET;
3	Second endpoint in the audio/video stream pair calculates a skew offset value based on the RTP packet and RTCP sender report timing of an incoming stream.
4	CG board uses an unsolicited MSPEVN_SKEW_OFFSET event to report the skew offset value to the application.
5	Application subtracts the audio skew offset value from the video skew offset value. This yields the audio/video skew value in ms. A positive result means the video lags the audio, while a negative result means the audio lags the video.
6	Application can do either of the following: Correct for the incoming audio/video skew on the CG board (possibly by adjusting audio/video jitter buffer latencies). For information about setting the audio jitter buffer latency, see the MSPP Developer's Reference Manual. Use the 3G-324M Interface to signal the audio/video skew value to a receiving 3G-324M terminal through the use of the h324_h223SkewIndication function. For more information, see the 3G-324M Interface Developer's Manual.

Examples

The following example shows how to use the MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command so that the server video endpoint with the MSP handle ephd can detect video skew offset values on an incoming data stream:

msp_ENDPOINT_RTPFDX_ENABLE_SKEW_CALC  skewCalc;
skewCalc.enable = RTCP_ENABLE_RCV_SKEW_CALC(0);
//Endian Adjust
skewCalc.enable = H2NMS_DWORD(skewCalc.enable);
command = mspBuildCommand(msp_ENDPOINT_RTPFDX_VIDEO,
                          MSP_CMD_RTPFDX_CALC_SKEW_OFFSET);
ret = mspSendCommand(ephd, command, &skewCalc, sizeof(skewCalc));
expectedEvent = (MSPEVN_SENDCOMMAND_DONE |
                 MSP_CMD_RTPFDX_CALC_SKEW_OFFSET);
if (WaitForSpecificEvent(gw,  CtaQueueHd, expectedEvent, &event) != 0)
    printf("\n\tERROR: mspSendCommand failed to send valid completion event\n");

The following example shows how to use the MSP_CMD_RTPFDX_CALC_SKEW_OFFSET command so that the audio endpoint with the MSP handle ephd1 can detect audio skew offset values on an incoming data stream:

msp_ENDPOINT_RTPFDX_ENABLE_SKEW_CALC  skewCalc;
skewCalc.enable = RTCP_ENABLE_RCV_SKEW_CALC(0);
//Endian Adjust
skewCalc.enable = H2NMS_DWORD(skewCalc.enable);
command = mspBuildCommand(msp_ENDPOINT_RTPFDX),
                          MSP_CMD_RTPFDX_CALC_SKEW_OFFSET);
ret = mspSendCommand(ephd1, command, &skewCalc, sizeof(skewCalc));
expectedEvent = (MSPEVN_SENDCOMMAND_DONE |
                 MSP_CMD_RTPFDX_CALC_SKEW_OFFSET);
if (WaitForSpecificEvent(gw,  CtaQueueHd, expectedEvent, &event) != 0)
    printf("\n\tERROR: mspSendCommand failed to send valid completion event\n");

The following example shows how to calculate the audio/video skew value upon receipt of MSPEVN_SKEW_OFFSET events from the audio data stream and the video data stream:

switch (pevent->id)
{
   case MSPEVN_SKEW_OFFSET:
      msp_RTP_SKEW_OFFSET *pSkewEvent;
      pSkewEvent = (msp_RTP_SKEW_OFFSET*)(pevent->buffer);
      skewOffset = (int)NMS2H_DWORD(pSkewEvent->offset);
      if(VideoCtx[nGw].rtpEp.hd == pevent->objHd)
      {
         vidOffset = skewOffset;
         bVid = TRUE;
      }
      else
      {
         audOffset = skewOffset;
         bAud = TRUE;
      }
      if (bVid == TRUE && bAud == TRUE)
      {
         vidSkew = vidOffset – audOffset;
         printf(“\nVideo lags audio by %d ms”, vidSkew);
         bVid = bAud = FALSE;
      }
      break;
}

Enabling RTP endpoints to send video skew values to the IP destination

Note: This feature is for server full-duplex video RTP endpoints.

During endpoint creation

When creating a video endpoint, you can configure the startRtcp bit field in the RTCP_ENDPOINT_PARMS (or RTCP_V6_ENDPOINT_PARMS) structure so that the endpoint can send video skew values to the IP destination. The configured endpoint sends video skew values in RTCP Sender Reports.

Use the following macros to configure the startRtcp bit field:

RTCP_ENABLE to enable RTCP for the endpoint.
RTCP_SET_0_INTERVAL to determine how quickly the RCTP Sender Reports begin after the RTP stream begins.
RTCP_VIDEO_LEADS_AUDIO, to indicate that video leads audio (if true).

After endpoint creation

After endpoint creation, you can use the MSP_CMD_RTPFDX_VIDEO_SKEW_TIME command against the video endpoint so that the endpoint sends video skew values to the IP destination. The configured endpoint sends video skew values in RTCP Sender Reports. The MSP_CMD_RTPFDX_VIDEO_SKEW_TIME command uses the msp_ENDPOINT_RTPFDX_SET_VID_SKEW structure to set the skew value.

The MSP_CMD_RTPFDX_VIDEO_SKEW_TIME command can return the following event:

Event	Description
MSPEVN_SENDCOMMAND_DONE	Indicates that the MSP_CMD_RTPFDX_VIDEO_SKEW_TIME command was successfully sent to the specified endpoint.

For information about creating and sending MSPP commands, see Creating and sending MSPP commands.

Example

The following example shows how to use the MSP_CMD_RTPFDX_VIDEO_SKEW_TIME command against the video endpoint with the MSP handle ephd so that the endpoint can signal video skew to the IP destination:

msp_ENDPOINT_RTPFDX_SET_VID_SKEW  skewCmd;

skewCmd.vidSkew = 0;

if (skew < 0)
{
   //Video leads audio
   skewCmd.vidSkew = RTCP_VIDEO_LEADS_AUDIO(skewCmd.vidSkew);
   skew *= -1;  //Make skew a positive number
}
skewCmd.vidSkew = RTCP_VIDEO_SKEW(skewCmd.vidSkew, skew);
//Endian Adjust
skewCmd.vidSkew = H2NMS_DWORD(skewCmd.vidSkew);
command = mspBuildCommand(msp_ENDPOINT_RTPFDX_VIDEO,
                          MSP_CMD_RTPFDX_VIDEO_SKEW_TIME);
ret = mspSendCommand(ephd, command, &skewCmd, sizeof(skewCmd));
expectedEvent = (MSPEVN_SENDCOMMAND_DONE |
                 MSP_CMD_RTPFDX_VIDEO_SKEW_TIME);
if (WaitForSpecificEvent(gw,  CtaQueueHd, expectedEvent, &event) != 0)
    printf("\n\tERROR: mspSendCommand failed to send valid completion event\n");