This version of this document is no longer maintained. For the latest documentation, see http://www.qnx.com/developers/docs.

Extending the Multimedia Framework

This chapter contains information about writing your own multimedia filters.

A filter is the basic building block of an application that uses the Multimedia library. There are many existing filters that handle a wide range of multimedia data formats. However, you may need to handle a new format, or want to use some hardware to process the data. In these cases, you'll need to write a new multimedia filter, or modify an existing one.

The QNX Multimedia library comes with many fully functional filters, but you may wish to write your own. This chapter shows you how to write filters, with two examples of filters for an application that handles MPEG data. Typically you won't need to create reader or writer filters, since the QNX-provided filters handle most input and output scenarios. It's more likely that you'll want to write a parser or decoder for formats that the standard filters don't handle.

The sample filters in this chapter are a parser for MPEG data, and a decoder for MPEG video data. Of course your application would also require (at least) a reader filter to read data from a stream (for example, an MPEG file), and a writer filter that writes the video data to an external device. You'd probably also want an MPEG audio decoder and audio writer.

How do I write a multimedia filter?

Let's look at the generic steps you need to take to write either a codec or parser filter, and compare the differences between them.

Whether you're writing a parser or codec filter, you need to implement some or all of the methods defined in these interfaces:

In this interface:	Implement the methods for:
`AODeConstructor`	Creating and destroying the filter
`MediaInput`	Managing input channels
`MediaOutput`	Managing output channels
`MediaControl`	Controlling processing
`MediaSeeker`	Seeking to a location in the media stream
`AOResourceAccess`	Exposing filter resources

In addition, the library needs a way to query the filters on how well they can handle a media stream. The parser filter provides this functionality by implementing the AOStreamInspector interface, which rates raw data, while the codec filter implements the AOFormatInsector interface, which rates parsed data.

The filters in this example use the default MediaBufferAllocator provided by the Multimedia library to create buffers. If you want to handle your own buffer allocation and management (for example, if you have some specific hardware requirements), you need to implement this interface as well.

Let's walk through the implementation of an MPEG system parser filter and codec filter and see how all of the pieces fit together. An MPEG has at least two channels, video and audio. This parser will parse the MPEG data stream into the audio and video components, and pass them on to decoder filters that handle MPEG audio and MPEG video. The decoder filter in this example decodes the parsed MPEG video data.

An interface is declared as a static array of function pointers. You'll notice in some of the examples below that some filters that don't implement every function. In these cases, they use pointers to convenience functions in the Multimedia convenience library.

Creating and destroying

The AODeConstructor interface defines the methods that create and destroy a filter. Both the parser and decoder filters need to implement these methods:

 static AODeConstructor media_filter =
  {
    Create,
    Destroy
  };

The Create() method should:

allocate our MmFilter_t filter data structure
allocate our MmFilterUser_t user private data structure, and assign it to MmFilter_t->user. This structure is declared as type media_filter_user, and can contain variables specific to the filter. It is available to any of the methods you implement.
return the MmFilter_t filter on success, 0 if an error occurs

The Destroy() method should:

free all resources allocated in Create()
return 0

Let's have a closer look at how we could implement these methods for the decoder.

struct media_filter_user
{
    // input channel variables
    const MediaOutput *mo;
    const MediaBufferAllocator *mba;
    MmChannel_t *mbac;

    // output channel variables
    MmFormat_t format;

    // codec variables and
    // other variables
...
};

...

static void *Create(const AOICtrl_t *interfaces)
{
  MmFilter_t *f;

  // allocate our MmFilter_t filter data structure
  if( !(f = (MmFilter_t*) calloc(1,sizeof(MmFilter_t))))
    return 0;

  // initialize and setup out identification string
  if( !(f->element.ID = strdup("MPEGVIDEO_Decoder")))
    return (MmFilter_t*) Destroy(f);

  // flag the structure as a filter
  f->element.type = MM_ELEMENT_FILTER;

  // allocate our MmFilterUser_t user private data structure
  if( !(f->user = (MmFilterUser_t*) calloc(1,sizeof(MmFilterUser_t))))
    return (MmFilter_t*) Destroy(f);

  // allocate and setup our input channel
  if( !(f->ichannels=(MmChannel_t*)calloc(1,sizeof(MmChannel_t))))
    return (MmFilter_t*) Destroy(f);
  if( !(f->ichannels[0].element.ID = strdup("MPEGVIn"))
    return (MmFilter_t*) Destroy(f);
  f->ichannels[0].element.type    = MM_ELEMENT_CHANNEL;
  f->ichannels[0].filter          = f;
  f->ichannels[0].direction       = MM_CHANNEL_INPUT;
  f->ichannels[0].format.mf.mtype = MEDIA_TYPE_COMPRESSED|
  MEDIA_TYPE_VIDEO;

  // allocate and setup our  output channel
  if( !(f->ochannels = (MmChannel_t*) calloc(1,sizeof(MmChannel_t))))
    return (MmFilter_t*) Destroy(f);
  if( !(f->ochannels[0].element.ID = strdup("RawVideoOut")))
    return (MmFilter_t*) Destroy(f);
  f->ochannels[0].element.type    = MM_ELEMENT_CHANNEL;
  f->ochannels[0].filter          = f;
  f->ochannels[0].direction       = MM_CHANNEL_OUTPUT;
  f->ochannels[0].format.mf.mtype = MEDIA_TYPE_VIDEO;

  /*
  ... mutex, condvar, etc.. initialization
  */

  //success  return the newly created filter
  return f;

}

This function frees the resources allocated in the Create() function.

static int32_t Destroy(void *obj)
{
  MmFilter_t *f = (MmFilter_t*) obj;
  // make sure we have a valid pointer
  if(!f)
    return -1;
  // free our filter private data structure
  if( f->user )
  free(f->user);
  //free our input channel
  if( f->ichannels )
  {
    if( f->ichannels[0].element.ID )
    free(f->ichannels[0].element.ID);
    free(f->ichannels);
  }
  // free our output channel
  if( f->ochannels )
  {
    if( f->ochannels[0].element.ID )
      free(f->ochannels[0].element.ID);
    free(f->ochannels);
  }
  //free our filter
  if( f->element.ID )
  free(f->element.ID);
  free(f);
  // return success
  return 0;
}

Inspecting the stream

The Multimedia library needs to be able to query both the parser and decoder for their ability to process some data. The two types of filter implement different interfaces to accomplish this task:

The parser implements the AOStreamInspector, which rates raw, unparsed data (an AOIStream_t). You should always use the sobj->streamer->Sniff() method to nondestructively sniff the start of the stream.
The decoder filter implements AOFormatInspector, which rates data that is already parsed (and described by an AODataFormat_t).

Both methods return a rating from 0 to 100 (where 100 is the best) of how well the data can be handled. Filters already implemented in the library generally return 80 when they can process the data.

We use the MmFOURCC() macro in the decoder example to check whether the format four character code matches MPEG 1 video. This macro takes the four characters that make up a fourcc code, and returns an int representation of the fourcc.

Parser

static AOStreamInspector stream_inspector =
{
  SniffData
};

static int32_t SniffData(AOIStream_t *sobj)
{
  // Sniff the data  and return our rating for the stream
  // (0 to 100).
}

Decoder

static AOFormatInspector format_inspector =
{
  RateFormat,
};

static int32_t RateFormat(const AODataFormat_t *fmt)
{
  /*
     In the case of the mpegvideo decoder filter, we check
     that we have a matching format fourcc and format type
   */
  if( fmt->fourcc == MmFOURCC('M','P','1','V') &&
      fmt->mtype == (MEDIA_TYPE_COMPRESSED|MEDIA_TYPE_VIDEO) )
      return 95;
  return 0;
}

Managing input channels

The MediaInput interface defines methods that the Multimedia library uses to query, reserve, and release a filter's input channels. These methods contain all the functionality the library needs to connect our input channel to the output channel of the previous filter in the graph.

static MediaInput media_input =
{
  IterateChannels,
  AcquireChannel,
  ReleaseChannel,
  RateFormat,
  SetFormat,
  SetMediaOutput,
  SetInputStream
};

Both the parser and decoder filters can use convenience functions supplied by the Multimedia library for IterateChannels(), AcquireChannel(), and ReleaseChannel(). Since the parser accepts raw data, it doesn't need to rate or set its input format. On the other hand, the decoder takes parsed data, and therefore must do both. Parsed data must have a format (a MmFormat_t) associated with it, and the decoder sets its input format so the library can tell what sort of data it can handle.

The parser's input channel is connected to streaming data, so it implements SetInputStream(). The decoder's input channel is connected to buffered data, so it implements SetMediaOutput(). This is what these methods should do:

IterateChannels: This method gives the Multimedia library access to all our input channels. Since both the parser and decoder have a single input channel, we can use the singleIterateInputChannels() convenience method from the library. It returns the input channel on the first call, and NULL thereafter.
AcquireChannel: This method flags our input channels as being in use. Since both the parser and decoder have a single input channel, we can use the singleAcquireInputChannel() convenience method from the library.
ReleaseChannel: This method flags our input channels as being released. Since both the parser and decoder have a single input channel, we can use the singleReleaseInputChannel() convenience method from the library.
RateFormat, SetFormat: These methods apply to filters that get their input from the buffered output of another filter. RateFormat() inspects the format of the input channel, and returns a rating of how well the filter can handle it. SetFormat() negotiates the format.; Since our MPEG system parser has direct access to an input stream, it doesn't need to implement these methods. We can use the noRateInputFormat() and noSetInputFormat() convenience methods from the library.; The decoder does connect to the buffered output of the parser, so it needs to implement these methods.
SetMediaOutput, SetInputStream: These methods connect the filter's input channels to the output channels of the previous filter in the chain. SetMediaOutput() applies to filters that connect to a buffered output, and SetInputStream() applies to filters that connect to streaming output. Our parser will implement SetInputStream() and use the noSetMediaOutput() convenience method. Our decoder will implement SetMediaOutput(), and use the noSetInputStream() convenience method.

Parser

static MediaInput media_input =
{
  singleIterateInputChannels,
  singleAcquireInputChannel,
  singleReleaseInputChannel,
  noRateInputFormat,
  noSetInputFormat,
  noSetMediaOutput,
  SetInputStream
};

Let's take a closer look at how we would implement each of these functions:

// From the mmconvienience library:
  MmChannel_t *singleIterateInputChannels(const MmFilter_t *f,int32_t
* const cookie);

// From the mmconvienience library:
int32_t singleAcquireInputChannel(MmChannel_t *c);

// From the mmconvienience library:
int32_t singleReleaseInputChannel(MmChannel_t *c);

// From the mmconvienience library:
int32_t noRateInputFormat(MmChannel_t *c,MmFormat_t *f,int32_t *
const cookie);

// From the mmconvienience library:
int32_t noSetInputFormat(MmChannel_t *c,const MmFormat_t *f);

// From the mmconvienience library:
int32_t noSetMediaOutput(MmChannel_t *c,const MediaOutput *m);

/*
    This method is where a filter that connect its input
    to a stream save its streamer object , and decode
    enough of the input stream to figure out what the
    output channels are.
 */
static int32_t SetInputStream(MmChannel_t *c,AOIStream_t *sobj)
{
 /*
   In the case of the mpegsystem parser filter:
    - Find out how many audio and video streams
      are inside the system stream. We'll just
      sniff the mpeg system stream and extract the
      systemheader data, using the streamer Sniff()
      method.
    - Allocate and setup our output channels (audio/video).
    - Set up our output formats (audio/video, fourcc,
      buffer size, number of buffers).
    - Flag the provided channel as in use (set the
      MM_CHANNEL_INPUTSET bit in the channel's flags).
    - Return 0 on success, -1 on error.
  */
}

Decoder

static MediaInput media_input =
{
  singleIterateInputChannels,
  singleAcquireInputChannel,
  singleReleaseInputChannel,
  RateInputFormat,
  SetInputFormat,
  SetMediaOutput,
  noSetInputStream
};

Let's take a closer look at how we would implement each of these functions:

// From the mmconvienience library:
MmChannel_t *singleIterateInputChannels(const MmFilter_t *f,int32_t
* const cookie);

// From the mmconvienience library:
int32_t singleAcquireInputChannel(MmChannel_t *c);

// From the mmconvienience library:
int32_t singleReleaseInputChannel(MmChannel_t *c);


/*
    This is where a filter that connects to the
    buffered output of an other filter gives a
    rating on its ability to handle the format.
    Since our mpegvideo decoder filter only
    handles one specific input format,
    we just check that the proposed format is correct.
 */
int32_t RateInputFormat(MmChannel_t *c,MmFormat_t *f,int32_t * const
cookie);
{
  if( (*cookie) != 0 )
    return 0;
  (*cookie)++;

  if( f->mf.fourcc != MmFOURCC('M','P','1','V') || f->mf.mtype
!= (MEDIA_TYPE_VIDEO|MEDIA_TYPE_COMPRESSED))
    return 0;
  return 100;
}


/*
    This is where a filter that connects its input
    to the buffered output of an other filter would
    set a negotiated input format.
    In the case of the mpegvideo decoder, we just
    save the negotiated format.
 */
int32_t SetInputFormat(MmChannel_t *c,const MmFormat_t *f)
{
 memcpy(&c->format,fo,sizeof(MmFormat_t));
 ...
}


/*
    This is where a filter that connects its input
    to the buffered output of an other filter  saves the
    other filter's MediaOutput interface.
    This is also a good place to initialize the decoder.
 */
int32_t SetMediaOutput(MmChannel_t *c,const MediaOutput *mo);
{
  MmFilter_t *f = c->filter;
  f->user->mo   = mo;
  c->flags     |= MM_CHANNEL_INPUTSET;
  /*
  ... initialize the decoder
  */
  return 0;
}

// From the mmconvienience library:
static int32_t noSetInputStream(MmChannel_t *c,AOIStream_t *sobj)

Managing output channels

Methods that connect your filter's output channels to another filter's input channels, either buffered or unbuffered, are defined in the MediaOutput interface.

static MediaOutput media_output =
{
    IterateChannels,
    AcquireChannel,
    ReleaseChannel,
    GetStreamer,
    IterateFormats,
    VerifyFormat,
    SetFormat,
    NextBuffer,
    ReleaseBuffer,
    DestroyBuffers
};

This is what these methods should do:

IterateChannels: This method gives the library access to all of the filter's output channels by returning each channel in turn, each time the method is called.; In the case of an MPEG system parser, we have two output channels (one audio and one video) so we need to implement this method.; Since our decoder uses a single channel, we can use the convenience method singleIterateOutputChannels().
AcquireChannel: This method flags our output channel as being in use. Both parser and decoder should implement this method.
ReleaseChannel: This method flags our output channel as being released. Both parser and decoder should implement this method.
GetStreamer: This method provides another filter with a streamed interface to one of our output channels. Neither our parser nor decoder has a streaming output (both are buffered), so we can use the noGetStreamer() convenience method.
IterateFormats: This method queries the filter for all possible output formats that a given channel has available. Both parser and decoder should implement this method.
VerifyFormat: This method lets the library give the filter a last chance to accept or reject a negotiated output format for a given channel. Our parser and decoder can simply use the convenience method acceptVerifyOutputFormats().
SetFormat: This method sets the channel's negotiated output format. Both parser and decoder should implement this method.
NextBuffer: This method is called by the next filter in the graph when it needs a buffer for the given time. This method should acquire the buffer (in this case, using the default MediaBufferAllocator), but it doesn't need to release the buffer. This is the responsibility of the filter that calls the NextBuffer() method. Both parser and decoder should implement this method.
ReleaseBuffer: When the next filter in the graph is finished with the buffer we gave it in NextBuffer(), it calls this function to release the buffer back into its pool.
DestroyBuffers: This method releases any buffers. It's called by the Multimedia library when an output channel is being released.

Parser

Let's take a closer look at how we can implement this interface:

static MediaOutput media_output =
{
  IterateOutputChannels,
  AcquireOutputChannel,
  ReleaseOutputChannel,
  noGetStreamer,
  IterateOutputFormats,
  acceptVerifyOutputFormats,
  SetOutputFormat,
  NextBuffer,
  ReleaseBuffer,
  DestroyBuffers
};

/*  This is where we give the mmedia library access
    to all of our output channels. In the case
    of an mpegsystem parser, we have one audio and one
    video output channel. So we iterate through and
    return each channel, then NULL afterward.
 */
static MmChannel_t *IterateOutputChannels(const MmFilter_t *f,int32_t
* const cookie);
{
  int32_t cnum=*cookie;
  if( cnum >= f->user->nstreams )
    return 0;
  (*cookie)++;
  return &f->ochannels[cnum];
}


/*
    This is where the library flags our
    output channel as being in use.
    Acquire the given output channel if its
    available and mark it as acquired.
    Return -1 on error 0 on success.
*/
static int32_t AcquireOutputChannel(MmChannel_t *c)
{
  if( c->flags&MM_CHANNEL_ACQUIRED )
    return -1;
  // mark as acquired
  c->flags |= MM_CHANNEL_ACQUIRED;
  return 0;
}


/* This is where the mmedia library flags our
   output channel as being released.
   Mark the given channel as no longer acquired.
 */
static int32_t ReleaseOutputChannel(MmChannel_t *c)
{
  c->flags &= ~(MM_CHANNEL_ACQUIRED|MM_CHANNEL_OUTPUTSET);
  return 0;
}


// From the mmconvenience library:
AOIStream_t *noGetStreamer(MmChannel_t *c);


/*  This is where the library can query a
    filter for all possible output formats that
    a given channel has available. Since our
    mpegsystem parser has 2 output channels (audio
    and video) and just one output format per channel
    we return our output channel format the
    first time this function is called and NULL afterward.
 */
static int32_t IterateOutputFormats(MmChannel_t *c,MmFormat_t *fmt,int32_t * const cookie)
{
  if( (*cookie)!=0 )
    return 0;
  (*cookie)++;
  memcpy(fmt,&c->format,sizeof(MmFormat_t));
  return 100;
}

// From the mmconvenience library:
int32_t acceptVerifyOutputFormats(MmChannel_t *c,const MmFormat_t
*f);

/*  This is where the library sets the
    channel negotiated output format, and the
    MediaBufferAllocator interface to use to
    acquire and release buffers.
    Return -1 on error, 0 on success.
 */
static int32_t SetOutputFormat( MmChannel_t *c, const MmFormat_t
*fo,
const MediaBufferAllocator *mba, MmChannel_t *mbac )
{
  if( !(c->flags&MM_CHANNEL_ACQUIRED) )
    return -1;
  if( c->flags&MM_CHANNEL_OUTPUTSET )
    return -1;
  if( memcmp(fo,&c->format,sizeof(MmFormat_t))
!= 0 )
    return -1;
  c->user->mbac = mbac;
  c->user->mba  = mba;
  // mark the channel's output set
  c->flags     |= M_CHANNEL_OUTPUTSET;
  return 0;
}


/*  This method is called by the next filter in the
    graph when it needs a buffer for the
    given time.  Return the filter playing status
    (MM_STATUS_PLAYING, MM_STATUS_STOP, MM_STATUS_EOF,...)
 */
static int32_t NextBuffer(MmChannel_t *c,MmTime_t t,MmBuffer_t **buffer)
{
  /*
  In the case of the mpegsystem parser filter:
    - if we are not MM_STATUS_PLAYING  return our status;
    - check channel stream id (audio/video )
    - acquire the next buffer with the channel's MediaBufferAllocator
    interface (mba->AcquireBuffer())
    - fill the buffer with data
    - return current playing status
    - the Library handles releasing the buffer
  */
}

/*
    When the next filter in the graph
    is finished with the buffer we gave it in
    NextBuffer(), it calls this function to release
    it back into its pool. In the case of the mpegsystem
    parser filter, call the ReleaseBuffer()
    function provided through the MediaBufferAllocator
    interface.
 */
static int32_t ReleaseBuffer(MmChannel_t *c,MmBuffer_t *b)
{
  return c->user->mba->ReleaseBuffer(c->user->mbac,b);
}

/*
    This function is called by the library when
    an output channel is being released.
    In the case of the mpegsystem parser filter,
    call the FreeBuffer() function
    provided through the MediaBufferAllocator interface.
 */
static int32_t DestroyBuffers(MmChannel_t *c)
{
  if( c->user->mba )
c->user->mba->FreeBuffers(c->user->mbac);
  return 0;
}

Decoder

Let's take a closer look at how we would implement this interface:

static MediaOutput media_output =
        {
            singleIterateOutputChannels,
            AcquireOutputChannel,
            ReleaseOutputChannel,
            noGetStreamer,
            IterateOutputFormats,
            acceptVerifyOutputFormats,
            SetOutputFormat,
            NextBuffer,
            ReleaseBuffer,
            DestroyBuffers
        };

// From the mmconvenience library:
static MmChannel_t *singleIterateOutputChannels(const MmFilter_t
*f,int32_t * const cookie);


/*
    This is where the library flags
    our output channel as being in use.
    Acquire the given output channel if its
    available and mark it as acquired.
    Return -1 on error 0 on success.
 */
static int32_t AcquireOutputChannel(MmChannel_t *c)
{
  MmFilter_t *f=c->filter;
  // Make sure the output channel isn't already acquired
  if( c->flags&MM_CHANNEL_ACQUIRED )
    return -1;
  /* Make sure our input channel has already
     been acquired and its input set
     since otherwise we won't know the dimensions,
     etc of our output channel
   */
  if( !(f->ichannels[0].flags&MM_CHANNEL_INPUTSET)
)
    return -1;
  // Flag the output channel as acquired
  c->flags |= MM_CHANNEL_ACQUIRED;
  return 0;
}


/*
    This is where the library flags our output
    channel as being released. Mark the given channel
    as no longer acquired.
 */
static int32_t ReleaseOutputChannel(MmChannel_t *c)
{
  c->flags &= ~(MM_CHANNEL_ACQUIRED|MM_CHANNEL_OUTPUTSET);
  return 0;
}


/*
    This is where a filter can give another filter a
    streamed interface to one of its output channel(s).
    Since our mpegvideo decoder uses buffered output channels
    we just return NULL;
    This function has been implemented for you in the
    mmconvenience library and has the following prototype:
*/
AOIStream_t *noGetStreamer(MmChannel_t *c);

/*
    This is where the mmedia library can query a filter
    for all possible output formats that a given channel
    has available. Our mpegvideo decoder has quite a
    few output formats for the video output channel.
    So we just go through all of them, returning one
    format at a time, and incrementing the iterator.
    Return 0 when we are done.
 */
static int32_t IterateOutputFormats(MmChannel_t *c,MmFormat_t *fmt,int32_t
* const cookie)
{
  // save our iterator
  int32_t cnum = *cookie;
  // if our iterator >= 5  we are done
  if( cnum >= 5 )
  return 0;

  // Initialize our proposed output format with a known value
  memset(fmt,0,sizeof(MmFormat_t));
  memcpy(&fmt->mf,&c->format.mf,sizeof(AODataFormat_t));
  fmt->mf.mtype       = MEDIA_TYPE_VIDEO;
  fmt->min_buffers    = 1;
  // fill out some specific output format value according to this
iteration
  switch( cnum )
  {
    case 0:
      fmt->mf.fourcc  = MmFOURCC('Y','U','Y','2'); // Pg_VIDEO_FORMAT_YUY2
      fmt->mf.u.video.depth =16;
      break;
    case 1:
      fmt->mf.fourcc  = MmFOURCC('R','G','B','2'); // Pg_IMAGE_DIRECT_8888:
      fmt->mf.u.video.depth = 32;
      break;
    case 2:
      fmt->mf.fourcc  = MmFOURCC('R','G','B','4'); // Pg_IMAGE_DIRECT_888:
      fmt->mf.u.video.depth = 24;
      break;
    case 3:
      fmt->mf.fourcc  = MmFOURCC('R','G','B','6'); // Pg_IMAGE_DIRECT_565:
      fmt->mf.u.video.depth = 16;
      break;
    case 4:
      fmt->mf.fourcc  = MmFOURCC('R','G','B','5'); // Pg_IMAGE_DIRECT_555:
      fmt->mf.u.video.depth = 16;
      break;
    default:
      break;
  }
  // Adjust our format min buffer size according to this iteration
  fmt->min_buffersize = fmt->mf.u.video.width
* fmt->mf.u.video.height * ((fmt->mf.u.video.depth+7)>>3);
  // Increment our iterator for next iteration
  (*cookie)++;
  return 100;
}

/*
    This is where the mmedia library gives the filter
    a last chance to accept or reject a negotiated
    output format for a given channel. Return 0 to
    reject the proposed output format, 100 to accept
    it. This method has been implemented for you in
    the mmconvenience library.
    This method has the following prototype:
 */
int32_t acceptVerifyOutputFormats(MmChannel_t *c,const MmFormat_t
*f);


/*
    This is where the library sets the channel
    negotiated output format, and the MediaBufferAllocator
    interface is used to acquire/release buffers.
    Return -1 on error, 0 on success.
 */
static int32_t SetOutputFormat( MmChannel_t *c, const MmFormat_t
*fo,
const MediaBufferAllocator *mba, MmChannel_t *mbac )
{
  if( c->flags&MM_CHANNEL_OUTPUTSET )
    return -1;
  /* Save the MediaBufferAllocator interface our
     output channels will use
     to allocate an output buffer and its
     associated channel.
   */
  c->user->mbac = mbac;
  c->user->mba  = mba;
  // save the negotiated output format
  memcpy(&c->format,fo,sizeof(MmFormat_t));
  // mark the channel's output set
  c->flags     |= M_CHANNEL_OUTPUTSET;
  return 0;
}


/*
    This function is called by the next filter
    in the graph when it needs a buffer for the
    given time. Return the filter playing status
    (MM_STATUS_PLAYING, MM_STATUS_STOP,MM_STATUS_EOF,...)
 */
static int32_t NextBuffer(MmChannel_t *c,MmTime_t t,MmBuffer_t **buffer)
{
  /*
  In the case of the mpegvideo decoder filter:
  - If we are not MM_STATUS_PLAYING  return  our status
  - Acquire a buffer through the channel's MediaBufferAllocator
  interface (mba->AcquireBuffer())
  - If we're at the EOF, release the buffer; otherwise:
  - Fill the buffer with data
  - Return current playing status
  */
}


/*
    When  the next filter in the graph
    is finished with the buffer we gave it in NextBuffer(),
    it calls this function to release it back into its pool.
    In the case of the mpegvideo decoder filter just call
    the ReleaseBuffer() function provided through the
    MediaBufferAllocator interface.
 */
static int32_t ReleaseBuffer(MmChannel_t *c,MmBuffer_t *b)
{
  return c->user->mba->ReleaseBuffer(c->user->mbac,b);
}

/*
    This function is called by the library when
    an output channel is being released.
    In the case of the mpegvideo decoder filter we call
    the FreeBuffer() function provided through the
    MediaBufferAllocator interface.
 */
static int32_t DestroyBuffers(MmChannel_t *c)
{
  if( c->user->mba )
    c->user->mba->FreeBuffers(c->user->mbac);
  return 0;
}

Controlling processing

The MediaControl interface defines the methods for starting, stopping, pausing, and resuming media filters.

static MediaControl media_control =
{
  Start,
  Stop,
  Pause,
  Resume,
  Status
};

These methods should do the following:

Start(): The Multimedia library calls this function to start a graph. Filters can use this function to initialize threads, and perform other one-time things they need to do.
Stop(): The Multimedia library calls this method to stop the graph. It should only be called when the library is getting ready to destroy the graph. The filters that receive this call should stop threads, and other one-time things.
Pause(): The Multimedia library calls this method to pause the graph temporarily. This method is called any time a user wants to temporarily stop the graph, or wants to seek into the graph.
Resume(): The Multimedia library calls this method to resume the graph. This is called when the user wants to resume playback after a temporary pause, or after seeking somewhere.
Status(): The Multimedia library calls this method to extract the filter's current status. The filter should return its current playing status.

Parser and Decoder

Both our parser and decoder would implement these methods:

        static int32_t Start(MmFilter_t *f,MmTime_t media_time)
        {
            // start the filter
        }

        /       static int32_t Stop(MmFilter_t *f)
        {
            // stop the filter
        }

        static int32_t Pause(MmFilter_t *f)
        {
            // pause the filter
        }

        static int32_t Resume(MmFilter_t *f,MmTime_t media_time,MmTime_t
real_time)
        {
            // resume the filter
        }

        static int32_t Status(MmFilter_t *f)
        {
            // return the playing status
        }

Seeking to a location in the stream

The MediaSeeker interface defines the seek method, and the filters that need to be informed when the graph is seeking to a new location.

static MediaSeeker media_seeker =
{
  Seek
};

The Multimedia library calls this function when the graph needs to seek to a certain location in the media. Theoretically, all the filter has to do is empty its buffers, and let the normal AOStreamer handle the rest.

Parser and Decoder

static int32_t Seek(MmFilter_t *f, MmTime_t media_time)
{
  // to do: seek to a new position
}

Providing access to a filter's resources

The AOResourceAccess interface defines the methods that expose any internal resources to the outside world for reading or writing.

static AOResourceAccess resource_access =
{
  GetResources,
  SetResource,
};

These methods should do the following:

GetResources()

The Multimedia library calls this method to query a filter for the availability of a particular resource, for example when an application calls MmGetResourceValue() or one of its convenience macros. This method has the following prototype:

static const AOResource_t *GetResources(void *handle);

SetResource()

The Multimedia library calls this method to set the resource res to the value data in a filter. It's called, for example, when an application calls MmSetResourceValue() or one of its convenience macros. This method has the following prototype:

static int32_t SetResource(void *handle,const char *res,const void
*data);

You need to implement this interface only for filters that need to expose their internal resources.

Parser

In this example we implement both these functions.

The Addon Interface library also defines the type AOResource_t that's used for internal resource storage and handling data.

typedef struct
{
  char *name;     // name of resource
  char *description;  // description of resource
  void *value;    // filled in later with the value
  void *info;     // typing info (ie range, list of items, etc)
  int32_t type;   // AOR_TYPE_* flags
} AOResource_t;

A simple AOResourceAccess example

Let's assume that our filter wants to expose the following resources:

The current position in the media stream:
MmTime_t position; // read-only resource
The duration of the media stream:
MmTime_t duration; // read-only resource
A Flag to put the filter in debug mode:
uint32_t debug; // read/write resource

In our filter's internal data structure we have:

struct media_filter_user
{
  /*
   other data
  */
  uint32_t debug;      // put the filter in debug mode
  MmTime_t position;   // position in the stream
  MmTime_t duration;   // duration of the stream
  AOResource_t *res;   // data structure needed to store or handle
resources
};

We need to define a AOResource_t record for each one of these resources:

static const MmTime_t timerange[]  = {0,86400000000,1}; // min,
max, default value
static const int32_t  debugrange[] = {0,10,0}; // min, max, default
value

static const AOResource_t resources[] =
{
  {"Duration","Duration",(void*)offsetof(struct
  media_filter_user,duration),&timerange,
    AOR_TYPE_LONGLONG|AOR_TYPE_READABLE },
  {"Position","Position",(void*)offsetof(struct
  media_filter_user,position),&timerange,
    AOR_TYPE_LONGLONG|AOR_TYPE_READABLE },
  {"Debug","Debug Output",(void*)offsetof(struct media_filter_user,debug),&
  debugrange,
AOR_TYPE_POINTER|AOR_TYPE_READABLE|AOR_TYPE_WRITABLE
},
  { 0 }
};

In the Create() method of the AODeConstructor interface, we need to allocate some memory and set up our resource pointers:

static void *Create(const AOICtrl_t *interfaces)
{
  MmFilter_t *f;
  AOResource_t *res;
  // create the filter object
  if( !(f = (MmFilter_t*) calloc(1,sizeof(MmFilter_t))) )
    return 0;
  // allocate the filter user data
  if( !(f->user = (MmFilterUser_t*) calloc(1,sizeof(MmFilterUser_t)))
)
    return (MmFilter_t*) Destroy(f);
// allocate our resource data structure
  if( !(f->user->res = (AOResource_t*) malloc(sizeof(resources))))
    return (MmFilter_t*) Destroy(f);
// initialize the resource data structure
  memcpy(f->user->res,&resources,sizeof(resources));
  res = f->user->res;
// adjust the resources pointers to the correct offset value
  while( res->name )
  {
    char *p    = (char*) f->user;
    res->value = (void*) (&p[(int32_t)res->value]);
    res++;
  }
  /*
   ....
  */
  return f;
}

And finally, the implementation of the GetResources() and SetResource() functions of the AOResourceAccess interface:

static const AOResource_t *GetResources(void *handle)
{
  MmElement_t *e = (MmElement_t*) handle;
  if( e && e->type==MM_ELEMENT_FILTER
)
  {
    MmFilter_t *f = (MmFilter_t*) handle;
// success
// return a pointer to our resource data structure
    return f->user->res;
}
  return 0;
}

static int32_t SetResource(void *handle,const char *name,const void
*data)
{
  MmElement_t *e = (MmElement_t*) handle;
  if( e && e->type == MM_ELEMENT_FILTER
)
  {
    MmFilter_t   *f   = (MmFilter_t*) handle;
    AOResource_t *res = f->user->res;
    while( res->name )
    {
      if( strcmp(res->name,name) == 0 )
      {
        // found it!
        if(strcmp(name,"Debug") == 0 )
        {
          //fprintf(stderr, "setting debug  to %d\n", (int32_t)data);
          f->user->debug = (int32_t) data;
        }
        return 0;
      }
      res++;
    }
  }
  return -1;
}

Making the Multimedia library aware of the filter

The Multimedia library uses the Addon Interface (AOI) library to load multimedia filters and perform multimedia format negotiations at runtime. This means that if you export the set of interfaces discussed above and drop your filter compiled as a DLL into the /lib/mmedia/dll directory, any multimedia application that uses the multimedia architecture will be able to use the services your filter provides without recompilation.

Parser

Our exported mpegsystem parser interface list:

#ifdef VARIANT_dll
 AOInterface_t interfaces[] =
 #else
AOInterface_t mpegs_parser_interfaces[] =
 #endif
 {
   { "Name",1,"mpegs_parser" },
   { "Version",MM_VERSION,0 },
   { "AODeConstructor",AODECONSTRUCTOR_VERSION,&media_filter
},
   { "MediaInput",MM_INPUT_VERSION,&media_input },
   { "MediaOutput",MM_OUTPUT_VERSION,&media_output },
   { "MediaSeeker",MM_SEEKER_VERSION,&media_seeker },
   { "MediaControl",MM_CONTROL_VERSION,&media_control },
   { "AOStreamInspector",AOSTREAMINSPECTOR_VERSION,&stream_inspector
},
   { "AOResourceAccess",AORESOURCEACCESS_VERSION,&resource_access
},
   { 0,0 }
 };

Decoder

Our exported mpegvideo decoder interface list:

#ifdef VARIANT_dll
 AOInterface_t interfaces[] =
 #else
 AOInterface_t mpegv_decoder_interfaces[] =
 #endif
 {
   { "Name",1,"mpegv_decoder" },
   { "Version",MM_VERSION,0 },
   { "AODeConstructor",AODECONSTRUCTOR_VERSION,&media_filter
},
   { "MediaInput",MM_INPUT_VERSION,&media_input },
   { "MediaOutput",MM_OUTPUT_VERSION,&media_output },
   { "MediaSeeker",MM_SEEKER_VERSION,&media_seeker },
   { "MediaControl",MM_CONTROL_VERSION,&media_control },
{ "AOFormatInspector",AOFORMATINSPECTOR_VERSION,&format_inspector
},
   { 0,0 }
 };