Multimedia Storage and Retrieval

There are 3 requirements for the support of multimedia databases that differ from the support of relational databases:

  1. VOLUME of data. For example, one compressed video frame typically exceeds 10 KB; consequently, 30 frames per second requires a volume of over 300 KB/s -- far more than that required by typical relational queries.
  2. CONTINUITY within each video or audio stream (i.e., interframe and intersegment time constraints)
  3. SYNCHRONIZATION of multiple streams of data (e.g., video, closed caption, left speaker's audio, right speaker's audio). We note that individual streams of data should not be viewed as separate clients. Furthermore, interleaving is impractical if the only difference is, say, closed captioning or foreign language subtitles for a video.

Resources to Consider
  • Network
    • bandwidth
    • communications protocols
    • buffering
  • Disk Bandwidth
    • amount of data that can be read per unit of time
  • CPU
    • may be needed for decompression; however, JPEG and MPEG hardware decoders exist
  • Disk buffers to facilitate I/O

Admission Control and Buffer Allocation
  • These policies are influenced by a number of factors:
    • The reference patterns of a query (e.g., pause, fast forward, rewind). We call this the "demand" side. From an operating systems perspective, the reference patterns in programs are typically too complicated to analyze and predict. In relational DBMS's, however, reference patterns are defined by the relational algebra (6 basic operations). Good RDBMS's can exploit reference patterns for performance gains. With multimedia databases, the reference patterns are simpler than those for relational databases since the types of queries tend to be quite limited (and hence it is easier to determine and calculate the demand side).
    • The number of buffers available at the time the query is serviced. We call this the "supply" side. Queries in multimedia databases are not initiated unless an adequate number of buffers exists.
    • The characteristics of all active queries in the system.

Continuous Media Streams

Each video or audio stream typically has a fixed display rate. The display rate is what the user sees and is concerned with. The consumption rate, on the other hand, is determined from the perspective of the disk(s). In an uncompressed stream, the display rate equals the consumption rate. In a compressed stream, there are 2 scenarios:

  1. with constant compression "r" (e.g., JPEG), display_rate = r * consumption_rate
  2. with variable compression (e.g., MPEG), display_rate > min_constant * consumption_rate
To satisfy the continuity requirements of a multimedia database system, the reading curve must not be below the consumption curve. In practice, it makes sense to read at least one buffer ahead since most implementations require a buffer to be completely filled before it can be read. Obviously, the scheduling of multiple streams requires more planning since reading rates, consumption rates, switching times (e.g., disk seek time), and disk utilization rates need to be considered.

Buffer sharing can yield significant efficiencies in multimedia systems. Buffer sharing can be facilitated by careful interleaving. For example, each stream requires its maximum number of buffers only right after completing its reading period. The total number of buffers required is not necessarily constant throughout a given time period for a number of reasons: consumption rates often differ, data streams are not contiguously placed on disk, and idle periods exist.

Non-contiguous streams can be handled better if the data is put into clusters. These clusters (subsets of the data) are stored contiguously. Clustering avoids time-consuming seek operations within a stream's reading period. This kind of buffer analysis needs to be carried out at design time rather than at execution time (unlike admission control, which is dynamic).

Although continuity requirements must be met, we are not forced to follow a fixed reading order. In fact, intelligent disk scheduling can be used to reduce the total seek time. For example, a strategy can be used to restrict the maximum inter-piece (or cluster) distance on disk. This distance is referred to as a scattering parameter. Of course, some disk strategies are impractical to implement, especially if updates are involved, since a complete disk reorganization may be required.

On-going or Future Work
  • Buffer allocation strategies for variable compression rates. See also Continuous Media Filesystems.
  • Statistical models of variable compressed streams
  • Analyzing display orders (e.g., news-on-demand systems in which users may not care about the order of most, or all, of the short video clips requested)
  • Content-based retrieval of videos (e.g., search for the goals scored in a hockey game). See also our web page on Image Databases.