Speaking From The Edge

Building Custom Activities Using the Core API

The Speech Server API is interesting to play around with. And understanding how Speech Server works behind the scenes is invaluable in debugging. But the real value of learning the API comes when you decide to build your own custom activities.

Speech Server ships with a number of activities (aka "Speech Dialog Components"). These cover most of the common scenarios. They allow you to quickly start building applications. But as with the default Windows Forms or ASP.NET components, they are just a starting point.

There are two different types of custom controls, the Voice Response Sequence Activity and the Voice Response Composite Activity. Sequences allow you to wrap existing activities in a reusable control. You work with them much like a standard workflow. Composite activities however are built entirely using the Core API.

I'm going to build a very simple composite control. This control will act much like the existing Recording Audio activity but without requiring a prompt.   

If you are interested in the complete project code you may download it here.

Creating a Voice Response Activity Library

To begin we'll create a new Voice Response Activity Library project called RecordWithoutPrompt.

During the creation process Visual Studio will ask you which Application Resources you wish to generate. We'll uncheck all of them; they are not necessary for our library.

When the project is created it adds a Sequence activity called VoiceResponseActivity1.cs by default. We'll remove this activity and add our own Voice Response Composite Activity called RecordWithoutPrompt.cs.

Core Methods

Opening the designer for our new class shows a bare-bones graphical view of the activity. From here we'll simply right-click and select "View Code" from the context menu. What you'll find is a very simple stub for the ExecuteCore method:

protected override void ExecuteCore(ActivityExecutionContext context)
{
    // Add Core API calls here.
    // base.ExecuteCore(context);
}

Every Composite Activity should contain at least two methods - ExecuteCore and CancelCore. The ExecuteCore method is the entry point for our activity. The CancelCore method is fired when the parent workflow is canceling the activity. The CancelCore method is not added by default so you'll need to add it yourself.

protected override void CancelCore(ActivityExecutionContext executionContext)
{
    base.CancelCore(executionContext);
}

Input Data Using Properties

We'll use a couple of public properties to pass information into your activity from the parent workflow - PlayBeep and EndSilenceTimeout. These will be used by our activity to control the recording behavior. Being C# the syntax for these properties are rather straightforward.

private bool _PlayBeep = true;
[DefaultValueAttribute(true)]
public bool PlayBeep
{
    get { return _PlayBeep; }
    set { _PlayBeep = value; }
}

private TimeSpan _EndSilenceTimeout = TimeSpan.Parse("00:00:03");
[DefaultValueAttribute(typeof(TimeSpan), "00:00:03")]
[TypeConverterAttribute("Microsoft.SpeechServer.Authoring.DialogDesigner.ShortTimeSpanConverter, Microsoft.SpeechServer.Authoring.DialogDesigner, Version=2.0.3400.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35")]
public TimeSpan EndSilenceTimeout
{
    get { return _EndSilenceTimeout; }
    set { _EndSilenceTimeout = value; }
} 

You'll notice that the EndSilenceTimeout uses the TypeConverterAttribute. This controls how the property is used in the designer. Rather than having to define the TimeSpan as a string ("00:00:03") the user can enter "3 sec".

Return Data Using Events

To return the recording results from our activity back to the parent workflow we'll create an event that we can bind to in the parent workflow.

public delegate void RecordingCompleteEventHandler(object sender, Microsoft.SpeechServer.RecordCompletedEventArgs e);
public event RecordingCompleteEventHandler RecordingComplete;
protected void OnRecordingComplete(Microsoft.SpeechServer.RecordCompletedEventArgs e)
{
    if (RecordingComplete != null)
        RecordingComplete(this, e);
}

Recording Audio via the Core API

We're going to kick off our audio recording inside the  ExecuteCore method. We'll do so using the API to control the TelephonySession.Recorder object.

First we assign the properties we collected earlier. This will control how long the system waits after the caller stops talking until it declares the recording complete. It will also tell the recording object wether or not to play a beep prior to starting the recording process.

Next we'll wire up the RecordCompleted handler. This will process the results of our recording and close our activity.

Finally we'll kick off the recording using the RecordAsync() method.

protected override void ExecuteCore(ActivityExecutionContext context)
{
    // Assign the properties for the activity
    Workflow.TelephonySession.Recorder.EndSilenceTimeout = EndSilenceTimeout;
    Workflow.TelephonySession.Recorder.PlayBeep = PlayBeep;

    // Wire up our RecordCompleted event handler
    Workflow.TelephonySession.Recorder.RecordCompleted += new EventHandler<Microsoft.SpeechServer.RecordCompletedEventArgs>(Recorder_RecordCompleted);            

    // Start the recording
    string tempFileName = System.IO.Path.GetTempPath() + Guid.NewGuid() + ".wav";
    Workflow.TelephonySession.Recorder.RecordAsync(tempFileName);
    
    

Completing The Activity

Once our recording is complete we'll want to fire off our RecordingComplete event and close the activity. The Close() method is vital as it tells the parent workflow that we're done. Failing to include it will cause Speech Server to hang inside this activity.

private void Recorder_RecordCompleted(object sender, Microsoft.SpeechServer.RecordCompletedEventArgs e)
{
    // Fire our custom RecordingComplete event
    OnRecordingComplete(e);

    // Close this activity. This tells the workflow to continue.
    this.Close(e.Error);
}