prikaz prve stranice dokumenta Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza
  Download
PDF 4.16 MB
master's thesis
Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza
Zagreb: University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, 2022. urn:nbn:hr:235:801522
Bakula, Slaven
University of Zagreb
Faculty of Mechanical Engineering and Naval Architecture

Institutional repository: Repository of Faculty of Mechanical Engineering and Naval Architecture University of Zagreb

Cite this document

Bakula, S. (2022). Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza (Master's thesis). Zagreb: University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture. Retrieved from https://urn.nsk.hr/urn:nbn:hr:235:801522

Bakula, Slaven. "Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza." Master's thesis, University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, 2022. https://urn.nsk.hr/urn:nbn:hr:235:801522

Bakula, Slaven. "Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza." Master's thesis, University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, 2022. https://urn.nsk.hr/urn:nbn:hr:235:801522

Bakula, S. (2022). 'Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza', Master's thesis, University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, accessed 17 November 2024, https://urn.nsk.hr/urn:nbn:hr:235:801522

Bakula S. Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza [Master's thesis]. Zagreb: University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture; 2022 [cited 2024 November 17] Available at: https://urn.nsk.hr/urn:nbn:hr:235:801522

S. Bakula, "Mehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza", Master's thesis, University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, Zagreb, 2022. Available at: https://urn.nsk.hr/urn:nbn:hr:235:801522

Metadata
TitleMehanizam za usmjeravanje pogleda virtualnog agenta temeljem zvučnog i vizualnog ulaza
Title (english)A mechanism for directing the virtual agent's gaze based on sound and visual input
AuthorSlaven Bakula
MentorTomislav Stipančić (mentor)
GranterUniversity of Zagreb
Faculty of Mechanical Engineering and Naval Architecture
Zagreb
Defense date and country2022-07-21, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Mechanical Engineering
Abstract
U sklopu Laboratorija za projektiranje izradbenih i montažnih sustava implementiran je virtualni softverski agent PLEA koji koristi multi-modalni pristup prilikom zaključivanja o emocionalnom stanju osobe s kojom komunicira temeljen na zvučnim i vizualnim informacijama. Koristeći neverbalni oblik komunikacije, korisnici mogu tako ostvariti kontakt sa virtualnim agentom. Kako bi komunikacija virtualnog agenta bila što sličnija stvarnoj osobi potrebno je implementirati značajke svakodnevnog govora kao što je pogled u oči prilikom komuniciranja. U ovom radu opisano je stvaranje jednog takvog sustava koji može biti implementiran u postojeći PLEA virtualni agent. Rad opisuje kreiranje virtualnog agenta u Unreal Engine programu na udaljenom računalu. U programskom jeziku Python uz korištenje dostupnih biblioteka napisan je program za prepoznavanje lica te određivanje kuta njegovog položaja na području kamere. Preko socket komunikacije šalju se vrijednosti definirane pozicije u stvarnom vremenu i ostvaruje se reakcija virtualnog agenta. Uz vizualnu akviziciju, napravljen je i program koji koristi lokalizaciju izvora zvuka korištenjem LyraTD-MSC modula te se slanjem iznos kuta, također preko socket protokola, ostvaruje reakcija virtualnog agenta.
Abstract (english)
As part of the Laboratory for the Design of Manufactured and Assembled Systems, a virtual software agent PLEA has been implemented, which uses a multimodal approach when inferring the emotional state of the person with whom it communicates based on sound and visual information. Using a non-verbal form of communication, users can make contact with a virtual agent. In order for the virtual agent's communication to be as similar as possible to a real person, it is necessary to implement features of everyday speech such as eye contact when communicating. This paper describes the creation of one such system that can be implemented in the existing PLEA virtual agent. The paper describes the creation of a virtual agent in the Unreal Engine program on a remote computer. A program for face recognition and determination of the angle of its position in the camera area was written in the Python programming language with the use of available libraries. Through socket communication, the values of the defined positions are sent in real time and the reaction of the virtual agent is realized. In addition to visual acquisition, a program that uses the localization of the sound source was created using the LyraTD-MSC module, and by sending the value of the angle, also via the socket protocol, the reaction of the virtual agent is realized.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:235:801522
Study programmeTitle: Mechanical Engineering; specializations in: Design, Process and Energy Engineering, Production Engineering, Engineering Modelling and Computer Simulation, Marine Engineering, Industrial Engineering and Management, Materials Engineering, Mechatronics and Robotics, Autonomous Systems
Course: Mechatronics and Robotics
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Embargo expiration date: 2024-07-07
Terms of useLicence Attribution 4.0 International (CC BY 4.0) icon
Created on2022-07-07 07:53:53