prikaz prve stranice dokumenta Arhitekture za raspoznavanje slika slojevima pažnje
No public access
undergraduate thesis
Arhitekture za raspoznavanje slika slojevima pažnje
Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. urn:nbn:hr:168:738470
Gudelj, Darijan
University of Zagreb
Faculty of Electrical Engineering and Computing

Institutional repository: FER Repository

Cite this document

Gudelj, D. (2023). Arhitekture za raspoznavanje slika slojevima pažnje (Undergraduate thesis). Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing. Retrieved from https://urn.nsk.hr/urn:nbn:hr:168:738470

Gudelj, Darijan. "Arhitekture za raspoznavanje slika slojevima pažnje." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. https://urn.nsk.hr/urn:nbn:hr:168:738470

Gudelj, Darijan. "Arhitekture za raspoznavanje slika slojevima pažnje." Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, 2023. https://urn.nsk.hr/urn:nbn:hr:168:738470

Gudelj, D. (2023). 'Arhitekture za raspoznavanje slika slojevima pažnje', Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, accessed 16 July 2024, https://urn.nsk.hr/urn:nbn:hr:168:738470

Gudelj D. Arhitekture za raspoznavanje slika slojevima pažnje [Undergraduate thesis]. Zagreb: University of Zagreb, Faculty of Electrical Engineering and Computing; 2023 [cited 2024 July 16] Available at: https://urn.nsk.hr/urn:nbn:hr:168:738470

D. Gudelj, "Arhitekture za raspoznavanje slika slojevima pažnje", Undergraduate thesis, University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, 2023. Available at: https://urn.nsk.hr/urn:nbn:hr:168:738470

Metadata
TitleArhitekture za raspoznavanje slika slojevima pažnje
Title (english)Architectures for Image classification through attention layers
AuthorDarijan Gudelj
MentorSiniša Šegvić (mentor)
Committee memberSiniša Šegvić (predsjednik povjerenstva)
Committee memberJosip Šarić (član povjerenstva)
Committee memberIvan Grubišić (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Electrical Engineering and Computing
Zagreb
Defense date and country2023-07-06, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Computing
Abstract
Vizualni transformeri postali su standardna arhitektura za obradu vizualnih podataka. Obilježje transformerske arhitekture je rezidualni tok, koji olakšava optimizaciju modela, no restriktira izradu novih značajki jer ima fiksnu dimanzionalnost kroz cijeli model. Model treba odlučiti zadržati prethodno izračunate značajke ili dodati nove. U ovom radu se proučavaju četiri različite arhitektonske modifikacije vizualnog transformera s ciljem poboljšanja ponovne uporabe značajki i parametarske učinkovitosti. Inspiracija se crpi iz konvolucijskih modela, posebno DenseNet-a, koji je pokazao uspješno rješavanje sličnih problema. Provodimo eksperimente na CIFAR-10 skupu podataka, pri tom pokazujući da je jedna od predloženih varijanti, gusti vizualni transformer, pareto optimalnija od vizualnog transformera.
Abstract (english)
Vision transformers have become a standard architecture for processing visual data. The characteristic of transformer architecture is the residual flow, which facilitates model optimization but restricts the creation of new features due to fixed dimensionality throughout the model. The model needs to decide whether to keep the previously calculated features or add new ones. This paper investigates four different architectural modifications of the visual transformer with the aim of improving feature reuse and parameter efficiency. Inspiration is drawn from convolutional models, particularly DenseNet, which has successfully addressed similar problems. We conduct experiments on the CIFAR-10 dataset, demonstrating that one of the proposed variations, Dense Visual Transformer, is Pareto-optimal compared to the Vision Transformer.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:168:738470
Study programmeTitle: Computing
Study programme type: university
Study level: undergraduate
Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsClosed access
Terms of useLicence In copyright icon
Public note
Created on2023-07-13 10:05:43