prikaz prve stranice dokumenta Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki
Access restricted to authenticated users
master's thesis
Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki
Split: University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, 2022. urn:nbn:hr:179:671948
Janketić, Luka
University of Split
Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture
Department of Electronics and Computing

Institutional repository: Repository of the Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture

Cite this document

Janketić, L. (2022). Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki (Master's thesis). Split: University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture. Retrieved from https://urn.nsk.hr/urn:nbn:hr:179:671948

Janketić, Luka. "Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki." Master's thesis, University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, 2022. https://urn.nsk.hr/urn:nbn:hr:179:671948

Janketić, Luka. "Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki." Master's thesis, University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, 2022. https://urn.nsk.hr/urn:nbn:hr:179:671948

Janketić, L. (2022). 'Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki', Master's thesis, University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, accessed 29 July 2024, https://urn.nsk.hr/urn:nbn:hr:179:671948

Janketić L. Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki [Master's thesis]. Split: University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture; 2022 [cited 2024 July 29] Available at: https://urn.nsk.hr/urn:nbn:hr:179:671948

L. Janketić, "Skupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki", Master's thesis, University of Split, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, Split, 2022. Available at: https://urn.nsk.hr/urn:nbn:hr:179:671948

Metadata
TitleSkupljanje, agregacija, indeksiranje te analiziranje podataka u mikroservisnoj arhitekturi temeljenoj na ELK stack-u i Apache Kafki
Title (english)Collecting, aggregating, indexing, and analyzing data in a microservice architecture based on the ELK stack and Apache Kafka
AuthorLuka Janketić
MentorToni Perković https://orcid.org/0000-0001-8826-4905 (mentor)
GranterUniversity of Split
Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture
(Department of Electronics and Computing)
Split
Defense date and country2022-09-21, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Computing
Abstract
Ovaj diplomski rad unutar okvira obraduje polja strujanja podataka, obrade i pohrane podataka i Elastic Stack sustav kao i Apache Kafka sustav. Teoretski dio rada je usredotočen na približavanje koncepta pohrane, obrade i analize podataka u podatkovnim cjevovodima i predstavljanje tehnologija koje spadaju pod Elastic Stack i broker aplikaciju Kafka. Praktični dio rada pokriva izradu jednog sustava koji se sastoji od brokera, podatkovnog cjevovoda te sustava za pohranu i analizu podataka. Ti sustavi su Elastic Stack i Apache Kafka. Nakon postavljanja lokalnih okolina tih sustava zatim se ručno generiraju podatci koji se zatim ubacuju u sustav i prate kroz svaki dio puta te na kraju analiziraju u nekoliko primjera. Kroz proces postavljanja sustava te propagiranje podataka kroz sustav i njihova pohrana i analiza razjašnjeni su problemi koji se često pojavljuju pri upravljanjem takvim sustavom i pri podizanju i postavljanju samoga sustava. Elastic Stack i Apache Kafka su postavljeni za rad na lokalnom spremniku na bilo kojem računalu uz pomoć Docker-a tako da se sustav može koristiti na više uređaja nezavisno jedan o drugome. Također postavke su namještene tako da se bilo koja dodatna usluga može direktno spojiti na sustav. Elastic Stack je skup programa Elasticsearch, Logstash, Kibana i njihovi dodatci. Koriste se za sustav koji obavlja funkcije podatkovnog cjevovoda (Logstash), mjesta za pohranu i obradu podataka (Elasticsearch) i alata za analizu i vizualizaciju podataka. Apache Kafka je platforma za strujanje podataka koji može biti i broker u komunikaciji porukama između Elastic Stacka i bilo kojeg vanjskog izvora. Također u slučaju pada Elastic Stack-a ili nekog njegovog dijela, podatci unutar se neće izgubiti jer Kafka radi i kao međuspremnik podataka koji su prošli kroz njega. Docker je skup proizvoda platforma-kao-usluga koji koriste virtualizaciju na razini OSa za isporuku softvera u paketima koji se nazivaju spremnici.
Abstract (english)
This thesis focuses on fields of data streaming, data processing, data storage and Elastic Stack. The theoretical part of the thesis is focused on approaching the concepts of data storage, processing and analysis in data pipelines and presenting the technology that falls under Elastic Stack and the Kafka broker applications. The practical part of the thesis covers the creation of a system consisting of a broker, a data pipeline, and a system for data storage and analysis. These systems are Elastic Stack and Apache Kafka. After setting up these systems on the local environment, data is generated manually, which is then entered into the system and tracked through each part of the system and finally analyzed trough several examples. Through the process of setting up the system and the propagation of data through the system and their storage and analysis, the problems that often appear when managing such a system and when setting up and setting up the system itself are clarified. Elastic Stack and Apache Kafka are set up to run on a local container on any computer regardless of the OS with the help of Docker so that the system can be used on multiple devices independently of each other. Settings are also set so that any additional service can be directly connected to the system. Elastic Stack is a collection of software that consists of Elasticsearch, Logstash, Kibana and their plugins. These applications are used for creating a system that performs the functions of a data pipeline (Logstash), a place to store and process data (Elasticsearch), and a tool for data analysis and visualization. Apache Kafka is a data streaming platform that can also be a broker in message communication between Elastic Stack and any external source. Also in case of failure of the Elastic Stack or any part of it, the data inside will not be lost because Kafka also works as a buffer for the data that passed through it. Docker is a set of platform-as-a service products that use OS-level virtualization to deliver software in packages called containers.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:179:671948
Study programmeTitle: Computing
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
Type of resourceText
File originBorn digital
Access conditionsAccess restricted to authenticated users
Terms of useLicence In copyright icon
Created on2023-05-18 08:14:12