摄像头基于深度学习的篮球训练人工智能助手系统

RochesterInstituteofTechnology

RITDigitalInstitutionalRepository

Theses

12-2021

Camera-baseddeeplearningAIassistantsystemforbasketballtraining

GuangkunZeng

gz4641@

Followthisandadditionalworksat:

/theses

RecommendedCitation

Zeng,Guangkun,"Camera-baseddeeplearningAIassistantsystemforbasketballtraining"(2021).Thesis.RochesterInstituteofTechnology.Accessedfrom

ThisThesisisbroughttoyouforfreeandopenaccessbytheRITLibraries.Formoreinformation,pleasecontact

repository@

.

CAMERA-BASEDDEEPLEARNINGAIASSISTANTSYSTEMFORBASKETBALLTRAINING

RIT

Camera-baseddeeplearningAIassistantsystemforbasketballtraining

By

GuangkunZeng

AThesisSubmittedinPartialFulfillmentoftheRequirementsfortheDegreeofMasterofFineArtinVisualCommunicationDesign

School/DepartmentofDesignCollegeofArtandDesignRochesterInstituteofTechnology

Rochester,NYDecember,2021

ThesisApproval

Camera-baseddeeplearningAIassistantsystemforbasketballtraining

ThesisTitle

GuangkunZeng

ThesisAuthor

Submittedinpartialfulfillmentoftherequirementsforthe

degreeofMasterofFineArts

TheSchoolofDesign|VisualCommunicationDesign

RochesterInstituteofTechnology|Rochester,NewYork

MikeStrobert ChiefThesisAdvisor

Name Title

ElectronicSignature:UseAdobeAcrobat

AssociateThesisAdvisor

Name Title

ElectronicSignature:UseAdobeAcrobat

Name(Optionalforsomeprograms)

selectaTitle

Title(Optionalforsomeprograms)

ElectronicSignature:UseAdobeAcrobat(Optionalforsomeprograms)

selectaTitle

Name(Optionalforsomeprograms) Title(Optionalforsomeprograms)

ElectronicSignature:UseAdobeAcrobat(Optionalforsomeprograms)

CAMERA-BASEDDEEPLEARNINGAIASSISTANTSYSTEMFORBASKETBALLTRAINING

Abstract

TheYOLO,aComputerVisionAlgorithms,isbroughtouttoanalyzethebasketballplayer’sstatusasadataset.Itcanrecordtheplayers’behavioronthecourtincludingdribbling,shooting,andrunning.Inthisway,theappcouldcollectthefieldgoalyoumadeandmissed.First,youshouldusethisapptorecordavideoofyourshoottraining.Afterthat,theAIwouldanalyzeandbringsouta3dvirtualdiagraminterpretyourperformance.Thisdiagramwillshowthehotzoneandcoldzoneforyourfieldgoal.Also,thetrackofyourballwillbedisplayedonthevideosothatyoucanknowiftheangleofyourshootingistoolowortoohigh.Intheend,theAI-basedonmachinelearningwillgiveoutaplanaccordingtoyourperformanceonshooting.

Asatrainingmobileapplicationsupportedbycamera-basedactionrecognition,thetargetaudienceisthebasketballamateurplayerswhodon’thavetheresourcesasproplayersdo.Thisprojectwillbedesignedasanewtrainingexperienceandwillbedeliveredasapromovideothatshowshowtousetheapplicationandalsothescenariopeopleuse.

Keywords

MR,virtualenvironment,camera-based,trainingsystem,deep-learning,machinelearning,motioncapture,AI

CriticalAnalysisandSummary

Context

Manybasketballenthusiastswanttoupgradetheirskillstoafarbetterlevel,butunfortunately,noteveryonehasthefundstohireprofessionalcoachestotrainonthebestcourt.Andwhat’smore,manyamateurplayersdonotevenknowhowtopracticeinaproperwaybecausethevideotutorialisnotasintuitiveastherealcoachorastheprofessionalhumancoachis.What’smore,itisalmostimpossibleforthebasketballplayertoquantifyhowgoodhis/hershootingis.Butpeopledoneedverygoodshootingformandtechniquetoshootconsistently.Howmightweimproveplayers’trainingexperience?

Nowadays,MR(mixedreality)haschangedourlife.ResearchbyVisualCapitalistprojectsthattheXRmarketwillbeworth$209billionby2022,markinganeight-foldincreasefrom20181.Furthermore,63%ofshareholdersinXRtechnologycompaniesbelievethetechnologywillbemainstreamby2024.Atthesametime,MixedRealityisquitewithinthemiddlebutalsothelongertermoftheentertainmentindustry.2Thistypeoftechnologycouldprovideacustomizedanduniquetrainingexperienceevencomparedtothebestcourtandtrainerscoulddo.3

TheMixedRealityisthetechnologythatcouldprovideanimmersiveexperienceanddirectdatavisualization.IproposetobringoutaMixedRealitymobileapplicationthatcangenerate3dvirtualdiagramofthebasketballcourttoassistplayerstounderstandtheirshootingperformance.Atthesametime,theappwillcollecttheplayer’sdataandmakethetrainingplanaccordingtotheAI-basedanalysissystem.

Methodology

Toimplementthissystem,multipletechnologieswillbeintegrated.TheYOLO,aComputerVisionAlgorithmswillrecognizetheballtraceandcollectdataofyourtraining.4Themotioncapturewith3Dvirtualenvironmenttechnologywillinterprettheshootinggestureandmovement.Thenthemachinelearningwillgenerateaspecifictrainingplan.

Thephaseofinterviewanduserresearchwillnotproceedbecausethisdesignputsmoreemphasisonconceptualdesign.Thisdesignfocuseson3problemsthatplayersoftenmeet.First,it’shardforamateurplayerstoquantifytheirskilllevel.Second,noteveryplayercanmakeapreciseplanasprofessional

1Emrich,T.(2020,February25).20for2020:AugmentedRealityTrendsandHowTheyMayPlayOutThisYear[Weblogpost].Retrievedfrom

/@tomemrich/20-augmented-reality-trends-to-keep-

an-eye-on-in-2020-d2b0258edbb

2Terry,Q.(2019,July23).ARiselevatingtheplayingfieldforsportsbycreatingsmartertrainingmethods.Retrievedfrom/futuresin/ar-is-elevating-the-playing-field-for-sports-by-creating-smarter-training-methods-77db01a84d64

3Lee,David."OurFirstShot(s)."Medium(blog).July17,2018./nex-team/our-first-shot-s-272c67d0349d.

4Terry,Q.(2019,July23).ARiselevatingtheplayingfieldforsportsbycreatingsmartertrainingmethods.Retrievedfrom/futuresin/ar-is-elevating-the-playing-field-for-sports-by-creating-smarter-training-methods-77db01a84d64

coachescando.Third,playersarenotsurethepracticeisexecutedperfectly.Peoplealsosometimesforgothowmanyshotsthey’vemade.

Threesolutionshavebeengivenout:First,A3ddiagramofyourplayscanhelpuserstoquantifytheirperformance.ThenAIDeeplearningcangiveyouadviceasgoodasacoachdoesorevenbetter.Finally,Asmartphonecamera-basedreal-timesystemcanrecordyourplayswithnomistakes.

Astheresultoftheproblemsolutions,HoopLabidentified3specificdesigngoalsthatareusedtoprototype：

Friendlytouse

Createanexperiencethatuserscanenjoy.Userscancustomizetheir3davatarandpickupfavoriteclothandhaircutsforit.Theavatarwillberiggedandmotioncapturedbyusersasawaytounderstandyourshootingmovement.

Easytounderstand

A360degrees3Ddiagramwillbegeneratedaccordingtotheplayrecording.Itisabletoturnyourstillcamerarecordinginto3dversionenvironment.TheAIwillcalculatethedistancebetweenobjects(player-player,shooter-hoop,defender-ball)

Convenienttorecord

Theuseronlyneedsasmartphonecameratodothebasicfunction.Usingasmartwatchtounlockadvancedfeatures.

Afterthreeroundsofprototyping,thecorrespondinginterfacelo-fiwireframesaredesigned.Basedonthoseinteractionwireframes,theoutputoftheUIvisualwireframeisfinallycompleted.ApromovideothatcombinedUIelementsandapplicationfunctionsisalsobemade.

Agamedesigntheoryisalsobroughtinforthissystem.Userscanearncoinsbyfinishingthetasks.Thentheycanusegamecoinstounlockavatarsandnewadvancedtasks.Thecirculareconomyisformedinthisprocess.

Ithinktherewillbesustainableiterationstoevaluatemyproposal.Atphaseone,whichistheinitialstateofaproduct,alargenumberofA/Btestswillbegiventotheusers.AccordingtotheA/Btestresultsthatwecollect,itcanbedecidedwhichfeatureisbetterforusers.Inphasetwo,theproductstartstooperatesmoothly,thenplentyofdatacanbeanalyzedbymachinelearning.DuetoAI,wecandecidewhatnewfeaturesthatmaysatisfyusersshouldbebuiltfurther.

Conclusion

It’sdifficulttoquantifytheexperienceofthesystem.Ifthesystemcouldpresentacustomizedtrainingplanfortheuserandcreateavirtual3denvironmentofarealcourtthattheplayerdoesn’tfeelstrangeaboutit,thisdesignwillbeasatisfiedsolution.

However,withthelimitofreal-timerendertechnologyandhardwarehashingpower,itisstillimpossibletoimplementanapplicationlikethisinamobilephone.ThisprojectisanapproachthatrepresentsonepossibilityofwhatAIalgorithmscandoforsportsgames.

AppendixA:ExpandedThesisDefensePresentation

Initialdesign

Designintension