Cấu trúc dữ liệu - Chương 1: Các khái niệm cơ bản MIRS Issues

1Nguyễn Thị Oanh Bộ môn HTTT – Viện CNTT & TT oanhnt@soict.hut.edu.vn Chương 1: Các khái niệm cơ bản MIRS Issues 2Architecture  Main operations – Insert new item – Retrieval 3Main operations 4Data Model  Data model determines : – How information is organized and stored – What types of queries are supported  Requirements: DM should be – extensible, new data type can be added – able to represent basic media type + temporal, spatial relationships – flexible so that items can be specified, queried, searched at different levels of abstraction – allow efficient storage and search 5A General Multimedia Data Model  OO-based  Multiple layers: – Object layer  Spatial relationship: window size + position for each item  Temporal specification: timeline-based: start time + duration – Media type layer:  Common media type  Features or attributes for each media type are specified  ex.: image: size, color histogram, main objects contained – Media format layer  Specifies the media formats   for proper encoding, analysis & presentation 6A General Multimedia Data Model 7Data Model: Remaining issues  Each layer: – Not completely designed – No common standard  Most MIRS: application-specific – Limited number of features – Limited number of data type Special data model for each application: - VIMSYS: image + video - a general video model - virage image schema structure 8Data Model: Example  VIMSYS (Visual Information Management System) Define events that can be queried User-defined entity: 1 concept (sunset ) or a physical entity (heart) Segmentation layer (temporal, spatial info, ..) + Feature layer: histogram, texture Data + transformation (compression, color space conversion, image enhancement, 9User interface  Requirements: – Insert database items easily – effectively and effeciently enter queries – Present query results to the user effectively and efficiently – Be user-friendly allow user to : – specify various types of input – compose multimedia objects – Specify attribute types to be extracted and indexed 10 User interface – Query support  Multimedia query: – Diverse – Fuzzy ==> tools: – Searching:  By keywords, parameters mapping problem: « red car »  By example need input tools: microphone, camera, – Browsing: start browsing with  A very large query  Based on the DB organization  Item randomly chosen – Query refinement: Feedback 11 User interface – Result presentation  Many design issues: – Present all media types + temporal, spatial relationships + QoS – How to extract and present essential information to browse for: long audio segment, long video, large image – Reponse time should be short (communication subsystem time + DB search time) – Felicitate relevance feedback and query refinement 12 Feature extraction  Determine the retrieval effectiveness  Requirements: – complete as possible to represent the contents of the information items – represented and stored compactly – The computation of distance between features should be efficient, otherwise the system response time would be too long 13 Feature extraction Levels of feature Example Handling Techniques Meta data Author name, date, title, DBMS Text annotation (captures abstract concepts) Content description, keywords: happy, sad, Information Retrieval Low-level (data patterns and statistics of a multimedia object, and possibly spatial and temporal relations between parts of the object) Audio: frequency distribution, Image: color distributions, texture,shapes, Content-based retrieval High-level (attempts to recognize and understand objects) recognize and interpret humans Content-based retrieval 14 Indexing  1 Object ~ many features  1 feature ~ many parameters  Indexing in MIRSs should – be hierarchical and – take place at multiple levels.  Application classification  Different levels of features  spatial and temporal relationships between objects 15 Similarity measurement  Similarity: computed on extracted features  Relevance of retrieval results: judged by human (subjective and context dependent) ? Computed similarity values should be conform to human judgement – Features used ? – Similarity measure used ? 16 QoS (Quality-of-Service)  MMData requires: – High bandwidth – large storage space and high transfer rate – delay and jitter bound – and temporal and spatial synchronization key components: – hosts (including clients and servers) under the control of the operating system – the storage manager – the transport or communications system 17 Tổng kết  Data model  User interface: query support + presentation  Feature extraction  Indexing  Similarity Measurement  Storage 18