prikaz prve stranice dokumenta Primjene teorije informacija na probleme dubokog učenja
  Download
PDF 1.37 MB
master's thesis
Primjene teorije informacija na probleme dubokog učenja
Zagreb: University of Zagreb, Faculty of Science, 2024. urn:nbn:hr:217:644603
Tot, Jakob
University of Zagreb
Faculty of Science
Department of Mathematics

Institutional repository: Repository of the Faculty of Science

Cite this document

Tot, J. (2024). Primjene teorije informacija na probleme dubokog učenja (Master's thesis). Zagreb: University of Zagreb, Faculty of Science. Retrieved from https://urn.nsk.hr/urn:nbn:hr:217:644603

Tot, Jakob. "Primjene teorije informacija na probleme dubokog učenja." Master's thesis, University of Zagreb, Faculty of Science, 2024. https://urn.nsk.hr/urn:nbn:hr:217:644603

Tot, Jakob. "Primjene teorije informacija na probleme dubokog učenja." Master's thesis, University of Zagreb, Faculty of Science, 2024. https://urn.nsk.hr/urn:nbn:hr:217:644603

Tot, J. (2024). 'Primjene teorije informacija na probleme dubokog učenja', Master's thesis, University of Zagreb, Faculty of Science, accessed 02 October 2024, https://urn.nsk.hr/urn:nbn:hr:217:644603

Tot J. Primjene teorije informacija na probleme dubokog učenja [Master's thesis]. Zagreb: University of Zagreb, Faculty of Science; 2024 [cited 2024 October 02] Available at: https://urn.nsk.hr/urn:nbn:hr:217:644603

J. Tot, "Primjene teorije informacija na probleme dubokog učenja", Master's thesis, University of Zagreb, Faculty of Science, Zagreb, 2024. Available at: https://urn.nsk.hr/urn:nbn:hr:217:644603

Metadata
TitlePrimjene teorije informacija na probleme dubokog učenja
Title (english)Applications of information theory on deep learning problems
AuthorJakob Tot
MentorMatej Mihelčić (mentor)
Committee memberMatej Mihelčić (predsjednik povjerenstva)
Committee memberHrvoje Planinić (član povjerenstva)
Committee memberMiljenko Huzak (član povjerenstva)
Committee memberMarko Horvat (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Science
(Department of Mathematics)
Zagreb
Defense date and country2024-03-06, Croatia
Scientific / art field,
discipline and subdiscipline		NATURAL SCIENCES
Mathematics
Abstract
U ovom radu dajemo pregled osnovnih pojmova i koncepata teorije informacija i dubokog učenja. Rad započinjemo definiranjem najvažnijih mjera teorije informacija u diskretnom slučaju. Nakon toga, obrađujemo dva najvažnija teorema čitavog područja: teorem o kodiranju izvora i teorem o kodiranju kanala. Nakon toga uvodimo mjere u neprekidnom slučaju i pokazujemo poveznicu s diskretnim slučajem. U drugom dijelu rada bavimo se dubokim učenjem. Dajemo kratki povijesni pregled počevši od prvih neuronskih modela. Ilustriramo ekvivalentnost nekih klasičnih modela strojnog učenja s modelima s jednim neuronom. Posebnu pažnju posvećujemo unaprijednim i konvolucijskim neuronskim mrežama te prolazimo kroz algoritam unazadne propagacije. Za kraj, uspoređujemo točnost različitih modela na problemima klasifikacije slika. Tu naglašavamo da kao funkciju gubitka neuronskih modela koristimo unakrsnu entropiju, veličinu iz teorije informacija
Abstract (english)
In this paper, we provide an overview of the basic concepts and principles of information theory and deep learning. We begin by defining the most important measures of information theory in the discrete case. After that, we discuss two of the most important theorems in the entire field: the source coding theorem and the channel coding theorem. We then introduce measures in the continuous case and demonstrate the connection with the discrete case. In the second part of the paper, we focus on deep learning. We provide a brief historical overview starting from the earliest neural models. We illustrate the equivalence of some classical machine learning models with single-neuron models. We pay special attention to feedforward and convolutional neural networks and go through the backpropagation algorithm. Finally, we compare the accuracy of different models on image classification problems. Here we emphasize that the loss function used by the neural models is crossentropy, a quantity from information theory.
Keywords
Languagecroatian
URN:NBNurn:nbn:hr:217:644603
Study programmeTitle: Mathematical Statistics
Study programme type: university
Study level: graduate
Academic / professional title: sveučilišni magistar matematike (sveučilišni magistar matematike)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Terms of useLicence In copyright icon
Created on2024-05-31 10:26:48