prikaz prve stranice dokumenta Implementacija i analiza arhitekture Mamba za neuronske jezične modele
No public access
master's thesis
Implementacija i analiza arhitekture Mamba za neuronske jezične modele
Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. urn:nbn:hr:168:591000
Krsnik, Lea
University of Zagreb
Faculty of Electrical Engineering and Computing

Institutional repository: FER Repository

Cite this document

Krsnik, L. (2024). Implementacija i analiza arhitekture Mamba za neuronske jezične modele (Master's thesis). Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing. Retrieved from https://urn.nsk.hr/urn:nbn:hr:168:591000

Krsnik, Lea. "Implementacija i analiza arhitekture Mamba za neuronske jezične modele." Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. https://urn.nsk.hr/urn:nbn:hr:168:591000

Krsnik, Lea. "Implementacija i analiza arhitekture Mamba za neuronske jezične modele." Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2024. https://urn.nsk.hr/urn:nbn:hr:168:591000

Krsnik, L. (2024). 'Implementacija i analiza arhitekture Mamba za neuronske jezične modele', Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, accessed 17 February 2025, https://urn.nsk.hr/urn:nbn:hr:168:591000

Krsnik L. Implementacija i analiza arhitekture Mamba za neuronske jezične modele [Master's thesis]. Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing; 2024 [cited 2025 February 17] Available at: https://urn.nsk.hr/urn:nbn:hr:168:591000

L. Krsnik, "Implementacija i analiza arhitekture Mamba za neuronske jezične modele", Master's thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, 2024. Available at: https://urn.nsk.hr/urn:nbn:hr:168:591000

Metadata
TitleImplementacija i analiza arhitekture Mamba za neuronske jezične modele
Title (english)Implementation and analysis of the Mamba architecture for neural language models
AuthorLea Krsnik
MentorJan Šnajder (mentor)
Committee memberJan Šnajder (predsjednik povjerenstva)
Committee memberSiniša Šegvić (član povjerenstva)
Committee memberMarko Čupić (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Electrical Engineering and Computing
Zagreb
Defense date and country2024-07-04, Croatia
Abstract
Duboko učenje proteklih godina ostvaruje iznimne rezultate, pogotovo u području obrade prirodnog jezika kroz uporabu velikih jezičnih modela. Fokus istraživačke zajednice većinski je bio usmjeren na arhitekturu Transformatora kao temelj velikih jezičnih modela. Međutim, nedavna inovacija, arhitektura Mamba, pokazala je potencijal za postizanje usporedivih rezultata s boljom vremenskom i prostornom složenošću. U ovom radu, detaljno su razmotreni matematički temelji Mambe te optimizacijske tehnike za izvedbu na grafičkim procesorima. Za potrebe rada, arhitektura Mambe implementirana je u programskom okviru PyTorch te je nad tom implementacijom provedena evaluacija na različitim zadatcima obrade prirodnog jezika.
Abstract (english)
In recent years, deep learning has achieved remarkable results, especially in the field of natural language processing through the use of large language models. The research community has primarily focused on the Transformer architecture as the foundation of large language models. However, a recent innovation, the Mamba architecture, has shown the potential to achieve comparable results with better temporal and spatial complexity. This thesis provides a detailed examination of the mathematical foundations of Mamba and optimization techniques for implementation on graphics processors. For the purposes of this thesis, the Mamba architecture was implemented in the PyTorch framework, and evaluations were conducted on various natural language processing tasks.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:168:591000
Study programmeTitle: Computing
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsClosed access
Terms of useLicence In copyright icon
Public note
Created on2024-10-12 21:14:09