prikaz prve stranice dokumenta Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika
No public access
undergraduate thesis
Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika
Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing, 2020. urn:nbn:hr:168:621197
Matošević, Patrik
University of Zagreb
Faculty of Electrical Engineering and Computing

Institutional repository: FER Repository

Cite this document

Matošević, P. (2020). Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika (Undergraduate thesis). Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing. Retrieved from https://urn.nsk.hr/urn:nbn:hr:168:621197

Matošević, Patrik. "Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2020. https://urn.nsk.hr/urn:nbn:hr:168:621197

Matošević, Patrik. "Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2020. https://urn.nsk.hr/urn:nbn:hr:168:621197

Matošević, P. (2020). 'Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika', Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, accessed 18 November 2024, https://urn.nsk.hr/urn:nbn:hr:168:621197

Matošević P. Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika [Undergraduate thesis]. Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing; 2020 [cited 2024 November 18] Available at: https://urn.nsk.hr/urn:nbn:hr:168:621197

P. Matošević, "Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika", Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, 2020. Available at: https://urn.nsk.hr/urn:nbn:hr:168:621197

Metadata
TitlePoopćivost tumačivih neuronskih modela obrade prirodnoga jezika
Title (english)Generalizability of Interpretable Neural Models for Natural Language Processing
AuthorPatrik Matošević
MentorJan Šnajder (mentor)
Committee memberJan Šnajder (predsjednik povjerenstva)
Committee memberDomagoj Alagić (član povjerenstva)
Committee memberMartin Tutek (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Electrical Engineering and Computing
Zagreb
Defense date and country2020-07-06, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Computing
Abstract
Duboki neuronski modeli korišteni u zadacima obrade prirodnog jezika s vremenom su postali sve složeniji. To je dovelo do problema nemogućnosti tumačenja načina rada tih modela, a uvođenjem mehanizma pozornosti ovaj je problem djelomično riješen. Mehanizam pozornosti mehanizam je odabira skrivenih stanja iz povratnih neuronskih mreža koji su važni za zadatak, a također često poboljšava generalizaciju modela. Zadatak ovog rada bio je testirati poboljšavaju li se generalizacijske performanse modela korištenje označenih obrazloženja u podacima. Razmatrana su tri modela: dva jednostavna bez mehanizma pažnje i jedan s mehanizmom pažnje. Modeli su trenirani na tri skupa podataka s referentnog skupa ERASER i njihova je izvedba analizirana pojedinačno na temelju toga jesu li se koristili podaci o obrazloženjima ili ne. Rezultati su pokazali da se podaci o obrazloženjima ne mogu koristiti da bi se poboljšali rezultati jednostavnijih modela baziranim na povratnim mrežama na načine koje smo uzeli u obzir. Međutim, uvođenjem mehanizma pažnje, podatke o obrazloženjima je moguće jednostavno i na prirodan način dodati u proces treniranja te je to poboljšalo performanse modela, ako je model bio prikladan za zadatak.
Abstract (english)
Deep neural models used in natural language processing tasks have become more complex over time. This led to a problem of the impossibility of interpreting the inner workings of the model, but this problem was partly solved after the introduction of the attention mechanism. The attention mechanism is a mechanism for selecting outputs from recurrent neural networks that are important for the task, and they often improve generalizability of the model. This thesis investigates whether incorporating information about human-annotated rationales improves the generalizability of the model. Three models are considered: two simple ones without attention mechanism, and one with the attention mechanism. Models were trained on three datasets from the ERASER benchmark and their performance was analyzed individually based on whether the rationale information was used or not. Results show that the rationale information cannot be used to improve the basic RNN models in the ways we considered. However, after adding the attention mechanism, the rationale information can be easily and naturally incorporated in the training process and, if the model is well suited for the task, this can improve the performance.
Keywords
Keywords (english)
Languageenglish
URN:NBNurn:nbn:hr:168:621197
Study programmeTitle: Computing
Study programme type: university
Study level: undergraduate
Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsClosed access
Terms of useLicence In copyright icon
Public note
Created on2023-01-14 09:30:24