prikaz prve stranice dokumenta Upravljanje uređajem metodom prepoznavanja gesti prstiju
  Download
PDF 2.23 MB
master's thesis
Upravljanje uređajem metodom prepoznavanja gesti prstiju
2023. urn:nbn:hr:200:836206
Čubaković, Lovro
Josip Juraj Strossmayer University of Osijek
Faculty of Electrical Engineering, Computer Science and Information Technology Osijek
Department of Communications
Chair of Electronics and Microelectronics

Cite this document

Čubaković, L. (2023). Upravljanje uređajem metodom prepoznavanja gesti prstiju (Master's thesis). Osijek: Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek. Retrieved from https://urn.nsk.hr/urn:nbn:hr:200:836206

Čubaković, Lovro. "Upravljanje uređajem metodom prepoznavanja gesti prstiju." Master's thesis, Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, 2023. https://urn.nsk.hr/urn:nbn:hr:200:836206

Čubaković, Lovro. "Upravljanje uređajem metodom prepoznavanja gesti prstiju." Master's thesis, Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, 2023. https://urn.nsk.hr/urn:nbn:hr:200:836206

Čubaković, L. (2023). 'Upravljanje uređajem metodom prepoznavanja gesti prstiju', Master's thesis, Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, accessed 22 November 2024, https://urn.nsk.hr/urn:nbn:hr:200:836206

Čubaković L. Upravljanje uređajem metodom prepoznavanja gesti prstiju [Master's thesis]. Osijek: Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek; 2023 [cited 2024 November 22] Available at: https://urn.nsk.hr/urn:nbn:hr:200:836206

L. Čubaković, "Upravljanje uređajem metodom prepoznavanja gesti prstiju", Master's thesis, Josip Juraj Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Osijek, 2023. Available at: https://urn.nsk.hr/urn:nbn:hr:200:836206

Please login to the repository to save this object to your list.
Metadata
TitleUpravljanje uređajem metodom prepoznavanja gesti prstiju
Title (english)Controlling the device using the method of recognizing finger gestures
AuthorLovro Čubaković
MentorMarijan Herceg (mentor)
Committee memberMarijan Herceg (predsjednik povjerenstva)
Committee memberMario Vranješ (član povjerenstva)
Committee memberRatko Grbić (član povjerenstva)
GranterJosip Juraj Strossmayer University of Osijek
Faculty of Electrical Engineering, Computer Science and Information Technology Osijek
(Department of Communications)
(Chair of Electronics and Microelectronics)
Osijek
Defense date and country2023-09-26, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Electrical Engineering
Telecommunications and Informatics
Abstract
U ovome radu predložen je sustav za upravljanje Spotify aplikacijom gestama prstiju. Za prepoznavanje gesti prstiju korištena je dugoročna rekurentna konvolucijska mreža (LRCN) istrenirana na skupu 20BN-Jester video sekvenci. LRCN model kombinira konvolucijsku neuronsku mrežu (CNN) i rekurentnu neuronsku mrežu (RNN), u ovome slučaju dugotrajno kratko pamćenje (LSTM jedinice), u svrhu učinkovite analize prostornih i vremenskih značajki u sekvencama pokreta. Implementirano je prepoznavanje sljedećih gesti koje korisnici mogu izvoditi: „swipe gore“ za postavljanje glasnoće zvuka na 100%, „swipe dolje“ za postavljanje glasnoće zvuka na 50% , „swipe lijevo“ za prebacivanje na prethodnu pjesmu u playlisti, „swipe desno“ za prebacivanje na sljedeću pjesmu u playlisti, „like“ za dodavanje pjesme u omiljeni sadržaj, „dislike“ za odbacivanje pjesme iz omiljenog sadržaja, „zaustavi/pokreni“ za zaustaviti ili pokrenuti pjesmu, „pojačaj“ za pojačavanje glasnoće zvuka za 5% te gesta „smanji“ za smanjivanje glasnoće zvuka za 5%. Osim ovih 9 gesti implementirana je gesta „nema geste“ kako bi model naučio situacije u kojima je osoba statična pred kamerom. Točnost LRCN modela na kojem je primijenjena metoda prijenosa učenja iznosi 92,14%. Evaluacija LRCN modela te evaluacija rada aplikacije za kontrolu Spotify glazbenog servisa temelji se na matricama zabune. Matrica zabune testne aplikacije za kontrolu Spotfiy servisa stvorena je na temelju rezultata prepoznavanja gesti koje je izvelo deset nasumičnih korisnika aplikacije. Točnost pri prepoznavanja gesti u stvarnom vremenu testirana na deset korisnika na osobnom računalu iznosi 80,2%, dok na ugradbenoj platformi Raspberry Pi 3 točnost iznosi 73,6%.
Abstract (english)
In this work, a system for controlling the Spotify application using finger gestures is proposed. A Long Short-Term Memory Recurrent Convolutional Network (LRCN) trained on the 20BN-Jester video sequences dataset was employed for finger gesture recognition. The LRCN model combines Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), specifically Long Short-Term Memory (LSTM) units, for efficient analysis of spatial and temporal features in motion sequences. The implemented system recognizes the following gestures that users can perform: „swipe up“ to set the volume to 100%, „swipe down“ to set the volume to 50%, „swipe left“ to switch to the previous song in the playlist, „swipe right“ to switch to the next song in the playlist, „like“ to add a song to favorites, „dislike“ to remove a song from favorites, „pauseplay“ to pause or play the song, „volume up“ to increase the volume by 5% and „volume down“ to decrease the volume by 5%. In addition to these 9 gestures, a „no gesture“ gesture has been implemented to allow the model to learn situations in which a person remains static in front of the camera. The accuracy of the LRCN model with transfer learning applied is 92,14%. The evaluation of the LRCN model and the evaluation of the Spotify control application are based on confusion matrices. The confusion matrix for Spotify control application was created based on the gesture recognition results performed by ten random application users. Real-time gesture recognition accuracy, tested on ten users on a personal computer, is 80,2%, while on the Raspberry Pi 3 embedded platform, the accuracy is 73,6%.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:200:836206
Study programmeTitle: Graduate University Study Programme in Electrical Engineering; specializations in: Communications and Informatics, Power Engineering
Course: Communications and Informatics
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka elektrotehnike (magistar/magistra inženjer/inženjerka elektrotehnike)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Terms of useLicence In copyright icon
Created on2023-10-10 11:16:44