prikaz prve stranice dokumenta Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost
No public access
master's thesis
Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost
Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. urn:nbn:hr:168:458254
Nekić, Filip
University of Zagreb
Faculty of Electrical Engineering and Computing

Institutional repository: FER Repository

Cite this document

Nekić, F. (2024). Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost (Master's thesis). Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing. Retrieved from https://urn.nsk.hr/urn:nbn:hr:168:458254

Nekić, Filip. "Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost." Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. https://urn.nsk.hr/urn:nbn:hr:168:458254

Nekić, Filip. "Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost." Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. https://urn.nsk.hr/urn:nbn:hr:168:458254

Nekić, F. (2024). 'Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost', Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, accessed 06 November 2024, https://urn.nsk.hr/urn:nbn:hr:168:458254

Nekić F. Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost [Master's thesis]. Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing; 2024 [cited 2024 November 06] Available at: https://urn.nsk.hr/urn:nbn:hr:168:458254

F. Nekić, "Analiza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost", Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, 2024. Available at: https://urn.nsk.hr/urn:nbn:hr:168:458254

Metadata
TitleAnaliza i hibridizacija jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost
Title (english)Analysis and hibridization of a simple transformer model for modular addition using mechanistic interpretabilit
AuthorFilip Nekić
MentorMario Brčić (mentor)
Committee memberMario Brčić (predsjednik povjerenstva)
Committee memberIvica Botički (član povjerenstva)
Committee memberDanijel Mlinarić (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Electrical Engineering and Computing
Zagreb
Defense date and country2024-08-30, Croatia
Abstract
Ovaj diplomski rad istražuje analizu i hibridizaciju jednostavnog modela transformera za modularno zbrajanje koristeći mehanističku interpretabilnost.Na početku opisujemo povijest, arhitekturu, način rada, glavne komponente te prednosti i nedostatke transformera. Zatim se bavimo mehanističkom interpretabilnošću, fokusirajući se na ključne obrasce i metode za razumijevanje ponašanja modela. Analiza otkriva da model koristi Fourierov algoritam za množenje, što nam daje uvid u unutrašnji rad modela. Nadalje, hibridizacijom određenih neuralnih komponenti postigli smo poboljšanje performansi modela. Ova saznanja doprinose boljem razumijevanju transformera i pokazuju kako ciljane modifikacije mogu optimizirati i performanse i interpretabilnost.
Abstract (english)
This thesis explores the analysis and hybridization of a simple transformer model for modular addition using mechanistic interpretability. Initially, we provide an overview of the transformer history, architecture, key components, types and advantages and disadvantages. We then talk about mechanistic interpretability, focusing on key patterns and methods for understanding the behavior of the model. Additionally, we briefly explain hybridization in machine learning. Our analysis reveals that the model employs Fourier's multiplication algorithm, offering insights into its internal workings. Furthermore, by hybridizing certain neural components, we achieved improved model performance. These findings enhance our understanding of transformers and demonstrate how targeted modifications can optimize both performance and interpretability.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:168:458254
Study programmeTitle: Computing
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsClosed access
Terms of useLicence In copyright icon
Public note
Created on2024-10-12 21:40:30