Title Generiranje multimedijskih pobuda temeljeno na ontološkom afektivnom i semantičkom označavanju
Title (english) Generation of multimedia stimuli based on ontological affective and semantic annotation
Author Marko Horvat
Mentor Nikola Bogunović (mentor)
Mentor Krešimir Ćosić (komentor)
Committee member Nikola Bogunović (član povjerenstva)
Granter University of Zagreb Faculty of Electrical Engineering and Computing (Department of Control and Computer Engineering) Zagreb
Defense date and country 2013, Croatia
Scientific / art field, discipline and subdiscipline TECHNICAL SCIENCES Computing Program Engineering
Universal decimal classification (UDC ) 004 - Computer science and technology. Computing. Data processing
Abstract Inteligentno generiranje multimedijskih pobuda složen je problem iz područja računalnih znanosti s izraženim interdisciplinarnim aspektima te podrazumijeva postupak rangiranog dohvaćanja afektivno i semantički označenih multimedijskih datoteka koje se koriste za pobuđivanje emocionalnih stanja korištenjem inteligentnih algoritama. Cilj disertacije jest unaprijediti baze afektivno označenih multimedijskih dokumenata i njihovo korištenje, povećati učinkovitost dohvaćanja pobuda, i time pomoći istraživačima s kvalitetnijim postupcima generiranja pobuda koji omogućuju preciznije i točnije dohvaćanje traženih pobuda. Primjena ontoloških struktura omogućuje opisno ekspresivniji, funkcionalno jednostavniji i brži upis novih i dohvaćanje postojećih multimedijskih objekata iz baza pobuda, te time olakšava njihovu izradu i korištenje. U disertaciji predstavljeni su postupci inteligentnog generiranja multimedijskih pobuda koji se temelje na formalnoj logici i ontološkim strukturama. Definirane su mjere učinkovitosti inteligentnih generatora. Predložene su ontološki strukturirane baze afektivno označenih multimedijskih podataka te opisan postupak upisa multimedijskih pobuda u takve baze. Izrađena je nova ontologija STIMONT za opis više razine semantike, emocija, konteksta i fiziologije pobuda. Postupci inteligentnog generiranja multimedijskih pobuda vrednovani su kroz više istraživanja, sa sudionicima i kroz postupke dohvaćanja informacija na računalu, te uspoređeni s postojećim postupcima generiranja pobuda. Navedenim postupcima dokazan je cilj disertacije i ostvareni su svi predloženi izvorni doprinosi. U eksperimentu inteligentnog generiranja pobude su razvrstavane krivuljama izdizanja za maksimalnu preciznost ili osjetljivost. Pokazano je da uporaba ontoloških struktura rezultira s do 24,9 % većom preciznosti, 13,1 % većom osjetljivosti i 9,9 % većom točnosti. Postupci inteligentnog generiranja pobuda omogućuju veću učinkovitost u dohvaćanju informacija od postojećih postupaka dohvaćanja pobuda, pružaju korisniku manji udio pogrešno odbačenih traženih pobuda i manji udio prikazanih pobuda koje nisu tražene. Također, opisane ontološke strukture mogu funkcionalno integrirati međusobno heterogene i nekompatibilne baze pobuda koje sadržavaju velike količine multimedijskih podataka te olakšati njihovu primjenu.
Abstract (english) Intelligent generation of multimedia stimuli is a complex problem in the field of computer science with pronounced interdisciplinary aspects. It implies a procedure of ranked retrieval of affective and semantically annotated multimedia documents which are used for elicitation of emotional states using intelligent computer algorithms. These multimedia documents are stored in specifically structured digital libraries called emotionally annotated multimedia databases or just stimuli databases for short. The files may be described with models of different expressivity and formality. Algorithms and procedures based on formal logic and automated reasoning are used to achieve ranked stimuli retrieval. They retrieve only those files which are most similar to the search query. The result set is then sorted based on relevancy to the posed query. Relevancy measures may use any stimuli feature, but measures based on stimuli semantics and affect are the most common. Multimedia stimuli generation may be described as a binary classification problem where a stimulus set stored in database relevant to the stated query is ranked, retrieved and displayed to a user. The goal of dissertation is to improve databases of affective multimedia documents and foster their utilization, increase efficiency of stimuli retrieval, and thus help researchers with better stimuli generation procedures that allow more precise and accurate retrieval of queried stimuli. The application of ontological structures enables, from users’ standpoint, descriptively more expressive, functionally simpler and faster input of new objects and retrieval of existing multimedia objects from stimuli databases which streamlines their construction and use. In this dissertation intelligent multimedia stimuli generation procedures based on formal logic and ontological structures are presented. Measures of intelligent stimuli generator goodness or efficiency are defined. Ontologically structured stimuli databases are presented along with a procedure for multimedia stimuli insertion. A new ontology STIMONT is developed for high-level representation of stimuli semantics, affect, context and physiology which is based on OWL DL ontology language and description logic. The ontology uses and partially expands semantics of EmotionML computer language. Also, a script stimuli generator language is defined as a unique interface for communication and functional integration of stimuli generators with other computer systems over telecommunication networks. The presented procedures of intelligent multimedia stimuli generation are evaluated through several experiments and compared to typical contemporary stimuli generation procedures. Some experiments involved participants while others were based on information retrieval procedures executed on a computer. Four different investigations of intelligent stimuli generation were accomplished: an international survey, two extrinsic and one intrinsic research. In total 85 students from University of Zagreb, Faculty of Electrical Engineering and Computing and Faculty of Humanities and Social Sciences participated in these experiments. The experiments were carried out in a close cooperation with Department of Experimental Psychology at Faculty of Humanities and Social Sciences. The experiments' results proved the goal of the dissertation and accomplished all its proposed contributions. In the intelligent stimuli generation experiment the stimuli where classified with lift charts for maximum precision or sensitivity. In the first case the classification threshold was defined with the maximum value of lift ratio, and in the latter case with the minimum result ratio which contains at least 90% of true positive results. It was shown that in the intelligent multimedia stimuli generation the usage of ontological structures improved precision up to 24.9%, sensitivity 13.1%, and accuracy 9.9%. If it is necessary to optimize generator for any efficiency measure other than sensitivity then accuracy classification is preferred, otherwise it is best to optimize classification for sensitivity. It was shown that stimuli in existing databases are poorly semantically annotated which results in a higher proportion of false positive and false negative results. The intelligent stimuli generation would be more efficient if universal and domain rules could be used, as well as a more detailed procedure for annotation of stimuli which can express statements about stimuli objects, events, their properties and scene. The intelligent multimedia stimuli generation procedures enable higher information retrieval efficiency than the existing stimuli retrieval procedures and result in smaller ratios of false negative and false positive stimuli. Finally, the described ontological structures can functionally integrate mutually heterogeneous and incompatible stimuli databases that contain large amounts of multimedia data thus streamlining their use.
Keywords
formalno predstavljanje znanja
automatizirano rasuđivanje
opisna logika
ontologije
dohvaćanje informacija
semantička sličnost
baze afektivnih multimedijskih podataka
afektivno računarstvo
Keywords (english)
formal knowledge representation
automated reasoning
description logic
ontologies
information retrieval
semantic similarity
affective multimedia databases
affective computing
Language croatian
URN:NBN urn:nbn:hr:168:470381
Study programme Title: Electrical Engineering and Computing Study programme type: university Study level: postgraduate Academic / professional title: Doktor znanosti elektrotehnike i računarstva (Doktor znanosti elektrotehnike i računarstva)
Type of resource Text
Extent 162 str. ; 30 cm.
File origin Born digital
Access conditions Closed access
Terms of use
Created on 2019-04-19 13:54:44