Title Lokalizacija i robotsko rukovanje otpadom iz mora Title (english) Localization and robotic manipulation with marine litter Author Jan Jakovljević Mentor Marko Švaco (mentor)Mentor Ante Bakić (komentor)Committee member Željko Šitum (predsjednik povjerenstva)Committee member Marko Katić (član povjerenstva)Committee member Filip Šuligoj (član povjerenstva)Granter University of Zagreb Defense date and country 2022-07-18, Croatia Scientific / art field, TECHNICAL SCIENCES Abstract S obzirom na porast zagađenja svjetskih oceana i mora širom planete zemlje i velike fluktuacije otpada u oceanima i morima dolazi do izbacivanja otpada na obale. Izbačeni otpad na obalama potrebno je sakupljati, što je težak i zahtjevan posao. Kao jedna od ekološki prihvatljivih rješenja nameće se autonomna mobilna platforma s robotskom rukom za sakupljanje otpada. U ovom radu dan je naglasak na rukovanje otpadom pomoću robotske ruke. Robotska ruka, odnosno sustav za rukovanje otpadom razvijen u ovom radu kasnije se može implementirati na različite mobilne platforme. U razvoju sustava za manipulaciju otpadom koriste se robotska ruka xArm6 Južno Korejske tvrtke UFACTORY. Za detekciju i grubu prostornu lokalizaciju otpada upotrebljava se konvolucijska neuronska mreža YOLO v5 koja potrebne vizijske 2D i 3D podatke dobiva iz stereovizijskog sustava Intel RealSense D455. Cjelokupna upravljačka podrška sustava razvijena je u robotskom operativnom sustavu (eng. Robot Operating System - ROS) pomoću programskih jezika Python i C++. Dobivene podatke iz stereovizijskog sustava, osobito oblake točaka (eng. point clouds) potrebno je obraditi. Obrada 3D podataka u ovom radu napravljena je pomoću programske knjižnice Open3D. Softver i sva akvizicija podataka iz okoline zajedno s obradom i planiranjem svih kretnji robota napravljena je na računalu. Odabrano računalo na modulu (eng. system-on-module) je Jetson Xavier NX tvrtke NVIDIA. Na početku rada opisuje se eksperimentalni postav sustava, svaka njegova komponenta te se daje dublji uvid u problematiku rada. Poslije detaljnijeg opisa problematike i sustava opisuje se način integracije različitih algoritama u sustav kako bi se ostvarila detekcija, lokalizacija, segmentacija i klasifikacija otpada u prostoru te preklapanje klastera s geometrijskim primitivom. Geometrijski primitiv se kasnije dodaje u ROS MoveIt scenu gdje se izvršava manipulacija otpadom. Na kraju rada napravljena je analiza dobivenih rezultata i evaluacija eksperimentalnog postava sustava, nakon čega se iznosi odgovarajući zaključak.
Abstract (english) Due to the increase in pollution across the planet and the large fluctuations of waste in the oceans and seas, waste is dumped onto shores. Garbage accumulated on beaches must be collected, which is a complicated and painstaking job. An autonomous mobile platform with a robotic arm for waste collection is being introduced as one of the environmentally friendly solutions. In this paper, the emphasis is on waste manipulation using a robotic arm. The xArm6 robotic arm from the South Korean company UFACTORY is used to create the waste manipulation system. For the detection and rough spatial localization of waste, the YOLO v5 convolutional neural network is used, which obtains the necessary 2D and 3D visual data from the Intel RealSense D455 stereo vision system. The full management support of the system was developed in the robot operating system (Robot Operating System - ROS) using Python and C++ programming languages. The obtained data from the stereovision system, especially the point clouds, must be processed. The processing of 3D data in this paper was done using the open3D program library. The software and all the data acquisition from the environment and the processing and planning of all system movements are made on the NVIDIA microprocessor. The chosen microprocessor is the Jetson Xavier NX. At the beginning of the work, the experimental set-up of the system and each of its components is described, and a deeper insight into the problems of the work is given. After a more detailed description of the problem and the system, the method of integrating different algorithms into the system is described to achieve the detection, localization, segmentation and clustering of waste in space and overlapping clusters with a geometric primitive. The geometry primitive is later added to the ROS MoveIt scene where it is manipulated. In the end, the task is rounded off with the analysis of the obtained results and the evaluation of the experimental setup of the system, after which the corresponding conclusion is obtained.
Keywords
Robotska ruka
MoveIt
ROS
3D vizijski sustav
open3D
YOLO
detekcija objekata
stereovizijski sustav
Keywords (english)
Robotic arm
MoveIt
ROS
3D vision system
open3D
YOLO
object detection
stereo vision system
point cloud
Language croatian URN:NBN urn:nbn:hr:235:624919 Study programme Title: Mechanical Engineering; specializations in: Design, Process and Energy Engineering, Production Engineering, Engineering Modelling and Computer Simulation, Marine Engineering, Industrial Engineering and Management, Materials Engineering, Mechatronics and Robotics, Autonomous Systems Type of resource Text File origin Born digital Access conditions Open access Terms of use Created on 2022-07-21 09:26:44
