prikaz prve stranice dokumenta Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja
No public access
undergraduate thesis
Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja
Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. urn:nbn:hr:168:981173
Šegvić, Jerko
University of Zagreb
Faculty of Electrical Engineering and Computing

Institutional repository: FER Repository

Cite this document

Šegvić, J. (2023). Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja (Undergraduate thesis). Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing. Retrieved from https://urn.nsk.hr/urn:nbn:hr:168:981173

Šegvić, Jerko. "Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. https://urn.nsk.hr/urn:nbn:hr:168:981173

Šegvić, Jerko. "Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. https://urn.nsk.hr/urn:nbn:hr:168:981173

Šegvić, J. (2023). 'Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja', Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, accessed 27 December 2024, https://urn.nsk.hr/urn:nbn:hr:168:981173

Šegvić J. Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja [Undergraduate thesis]. Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing; 2023 [cited 2024 December 27] Available at: https://urn.nsk.hr/urn:nbn:hr:168:981173

J. Šegvić, "Primjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja", Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, 2023. Available at: https://urn.nsk.hr/urn:nbn:hr:168:981173

Metadata
TitlePrimjena generativnoga jezičnog modela na zadatke odgovaranja na pitanja i jezičnog modeliranja
Title (english)Application of a generative language model to question answering and language modeling tasks
AuthorJerko Šegvić
MentorJan Šnajder (mentor)
Committee memberJan Šnajder (predsjednik povjerenstva)
Committee memberDavid Dukić (član povjerenstva)
Committee memberJosip Jukić (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Electrical Engineering and Computing
Zagreb
Defense date and country2023-07-05, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Computing
Abstract
Rad proučava mogućnosti generativnog jezičnog modela GPT-2. Detaljno je objašnjena ideja iza generativnih jezičnih modela i principi na kojima se temelje. Objašnjena je arhitektura modela i mehanizmi koje konkretan model koristi. Performanse modela mjerene su na tri različita skupa podataka bez ugađanja i s ugađanjem na podskupovima za treniranje. Skupovi podataka testiraju sposobnost modela za rješavanje jezičnih zadataka odgovaranja na pitanja i jezičnog modeliranja. Za svaki skup je objašnjena njegova struktura, naveden broj primjera na svakom podskupu te objašnjeno što je njime testirano. Da bi model uspješno savladao te zadatke, mora moći pratiti dulji kontekst i dobro razumjeti sadržaj teksta. Opisan je postupak treniranja i nevedeni su hiperparametri korišteni za treniranje na svakom skupu. Provedena je detaljna analiza rezultata prije i nakon ugađanja modela. Komentirani su nedostatci i predloženi koraci za daljnji razvoj.
Abstract (english)
This work studies the abilities of the generative language model GPT-2. The general idea behind generative language models and the principles on which they are based are explained in great detail. The architecture of GPT-2 and the mechanisms used by it are also explained. The performance of the model is measured on three different datasets in the zero-shot setup and with fine-tuning on the training subsets. The datasets test the model's ability to handle more complex language tasks. The structure of each dataset is explained in detail. For each dataset, the size of each subset is listed, and it is explained which tasks they are testing. To succeed on these tasks, the model must be able to follow the longer context and understand it well. The training procedure is described, and hyperparameters used to train each dataset are listed. A detailed analysis of the results before and after fine-tuning was carried out. The deficiencies of training procedures are commented. The steps for further development are also suggested.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:168:981173
Study programmeTitle: Computing
Study programme type: university
Study level: undergraduate
Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsClosed access
Terms of useLicence In copyright icon
Public note
Created on2023-10-11 10:38:13