端对端深度神经网络架构在汽车自动驾驶车速及转向角预测的应用-2023-05-自动驾驶

electronics

Article

End-to-EndDeepNeuralNetworkArchitecturesforSpeedandSteeringWheelAnglePredictioninAutonomousDriving

PedroJ.Navarro1,*,LeanneMiller1,FranciscaRosique1,CarlosFernindez-Isla1andAlbertoGila-Navarro2

checkfor

updates

Citation:Navarro,P.J.;Miller,L.;Rosique,F.;Fernández-Isla,C.;

Gila-Navarro,A.End-to-EndDeepNeuralNetworkArchitecturesfor

SpeedandSteeringWheelAngle

PredictioninAutonomousDriving.

Electronics2021,10,1266.

https://

/10.3390/electronics10111266

AcademicEditors:DongSeogHan,KalyanaC.VeluvoluandTakeoFujii

Received:13April2021

Accepted:18May2021

Published:25May2021

Publisher’sNote:MDPIstaysneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffil-iations.

Copyright:©2021bytheauthors.LicenseeMDPI,Basel,Switzerland.ThisarticleisanopenaccessarticledistributedunderthetermsandconditionsoftheCreativeCommons

Attribution(CCBY)license(https://

/licenses/by/

4.0/).

1

2

*

Divisi6ndeSistemaseIngenier½aElectr6nica(DSIE),CampusMuralladelMar,s/n,UniversidadPolitécnicadeCartagena,30202Cartagena,Spain;ler@upct.es(L.M.);paqui.rosique@upct.es(F.R.);

carlos.fernandez@upct.es(C.F.-I.)

GenéticaMolecular,InstitutodeBiotecnolog½aVegetal,EdificioI+D+I,PlazadelHospitals/n,UniversidadPolitécnicadeCartagena,30202Cartagena,Spain;alberto.gilan@um.es

Correspondence:pedroj.navarro@upct.es;Tel.:+34-968-32-6546

Abstract:Thecomplexdecision-makingsystemsusedforautonomousvehiclesoradvanceddriver-assistancesystems(ADAS)arebeingreplacedbyend-to-end(e2e)architecturesbasedondeep-neural-networks(DNN).DNNscanlearncomplexdrivingactionsfromdatasetscontainingthousandsofimagesanddataobtainedfromthevehicleperceptionsystem.Thisworkpresentstheclassification,designandimplementationofsixe2earchitecturescapableofgeneratingthedrivingactionsofspeedandsteeringwheelangledirectlyonthevehiclecontrolelements.Theworkdetailsthedesignstagesandoptimizationprocessoftheconvolutionalnetworkstodevelopsixe2earchitectures.Inthemetricanalysisthearchitectureshavebeentestedwithdifferentdatasourcesfromthevehicle,suchasimages,XYZaccelerationsandXYZangularspeeds.Thebestresultswereobtainedwithamixeddatae2earchitecturethatusedfrontimagesfromthevehicleandangularspeedstopredictthespeedandsteeringwheelanglewithameanerrorof1.06%.Anexhaustiveoptimizationprocessoftheconvolutionalblockshasdemonstratedthatitispossibletodesignlightweighte2earchitectureswithhighperformancemoresuitableforthefinalimplementationinautonomousdriving.

Keywords:autonomousdriving;end-to-endarchitecture;speedandsteeringwheelangleprediction;DNNforregression

1.Introduction

Autonomousdrivingtechnologyhasadvancedgreatlyinrecentyears,butitisstillanongoingchallenge.Traditionally,intelligentdecisionmakingsystemsonboardautonomousvehicleshavebeencharacterizedbytheirenormouscomplexity[

1

]andarecomposedofmultiplesubsystems,includingaperceptionsystem,globalandlocalnavigationsystems,acontrolsystem,asurroundingsinterpretationsystem,etc.,[

2

].Thesesubsystemsarecombinedaimingtocoverthecomplicateddecisionsandtaskswhichthevehiclemustperformwhilstdriving.Toobtaintheobjectivesofthevehicle,thesesubsystemsuseawiderangeoftechniqueswhichinclude:cognitivesystems[

3

],agentsystems[

4

],fuzzysystems[

5

],neuralnetworks[

6

],evolutionaryalgorithms[

7

]orrule-basedmethods[

8]

.

Deeplearningtechniquesarebecomingincreasinglypopularandarenowavaluabletoolinawiderangeofindustries,includingtheautomotiveindustry,duetotheirpowerfulimagefeatureextraction.Thesetechniqueshaveallowedtheso-calledend-to-end(e2e)drivingapproachtoappear,simplifyingthetraditionalsubsystemsgreatlyandreducingthetasksofmodelingandcontrolofthevehicle[

9

](Figure

1).TheappearanceofDNNs

meanthatdecision-makingsystemsonboardautonomousvehiclescanreplacemanyofthesubsystemsmentionedpreviouslywithneuralblocks[

10

].Theseneuralblocks,properlyinterconnectedandtrainedwiththecorrectdataarecapableofobtainingperformancesgreaterthan95%forthepredictionofvehiclecontrolvariables[

11]

.Anadvantageofthesemodelsisthattheygenerallyrequirefeweronboardsensorsasthemainsourceof

Electronics2021,10,1266.

/10.3390/electronics10111266

/journal/electronics

Electronics2021,10,12662of

21

informationfedtotheDNNsusuallyconsistsofRGBimagesandkinematicdatafromaninertialmeasurementunit(IMU)[

12

].Thismakesend-to-enddrivingsystemsmoreeasilyaccessiblethanthetraditionalperceptionsubsystemswithsensorssuchasLIDARwhichareverycostly.

Figure1.Traditionaldrivingsystemscomparedtoend-to-enddrivingsystem.

Deeplearningmethodsforautonomousdrivinghavegainedpopularitywithadvance-mentsinhardware,suchasGPUs,andmorereadilyavailabledatasets,bothforend-to-enddrivingtechniques[

13

]andtheuseofdeeplearninginindividualsubsystems[

14

].Therehavebeenavarietyofdifferentapproachesforthedevelopmentofdrivingapplicationsusingendtoendlearningtechniques.Inonestudy,a98%accuracywasobtainedusingconvolutionalneuralnetworks(CNN)togeneratesteeringanglesfromimagesgeneratedbyafrontviewcamera[

15

].Inasimilarwork,asequenceofimagesfromapublicdatasetwasusedasinputtotheCNN,topredictwhetherthevehiclewasaccelerating,deceleratingormaintainingspeedaswellascalculatingthesteeringangle[

16]

.

AninterestingapproachdesignedaCNNtodevelopahuman-likeautonomousdrivingsystemwhichaimstoimitatehumanbehaviormeaningthevehiclecanbetteradapttorealroadconditions[

13

].Theauthorsused3DLIDARdataasinputtothemodelandgeneratedsteeringandspeedcommands,andinadrivingsimulationmanagedtodecreaseaccidentswiththeautonomoussystemten-foldcomparedwiththehumandriver.AdrivingsimulatorwasalsousedtotestaCNN-basedclosedloopfeedbacktocontrolthesteeringangleofthevehicle[

17

].TheauthorsdesignedtheirownCNN,DAVE-2SKY,usingtheCaffedeeplearningframeworkandtestedthesysteminalane-keepingsimulation.Theresultswerepromising,althoughproblemsoccurredifthedistancetothevehicleinfrontbecamelessthan9m.

Variouslongshort-termmemory(LSTM)modelshavealsobeenstudied.Aconvolu-tionalLSTMmodelwithbackpropagationwastrainedtoobtainthesteeringanglefromvideoframesusingtheUdacitydataset[

18]

.AnFCN-LSTMarchitecturewasusedtopredictdrivingactionsandmotionfromimagesobtainingalmost85%accuracy.Acon-volutionalLSTMmodelwasalsousedtopredictsteeringanglesfromastreamofimagesfromafrontfacingcamera[

19

],improvingontheresultsfrompreviousworks[

20]

.

Anotherapproachconsistsinaddingmoresensors.Inoneworkadatasetwasobtainedusingsurroundviewcamerasinadditiontothetypicalfrontviewcamera[

21

].ThedataobtainedbythecameraswasusedtopredictthespeedandsteeringangleusingexistingpretrainedCNNmodels.Theuseofsurroundviewcamerasimprovedtheresultsobtainedatlowspeeds(<20km/h),butatgreaterspeedstheimprovementwaslesssignificant.

Inthiswork,wepresentadetailedstudyimplementingsixend-to-endDNNarchitec-turesforthepredictionofthevehiclespeedandthesteeringwheelangle.Thearchitectureshavebeentrainedandtestedusing78,011imagesfromrealdrivingscenarios,whichwerecapturedbytheCloudIncubatorCar(CIC)autonomousvehicle[

2]

.

Electronics2021,10,12663of

21

2.MaterialsandMethods

DNNend-to-endarchitecturesrequirelargevolumesofdataforthemodelstocon-vergecorrectly.ThedataneededtocreateDNNmodelsforautonomousdrivingorADAScanbeobtainedfromthreedifferenttypesofsources:

1.Adhoctests.Toperformthistypeoftesting,largeresourcesarerequired,intheformofoneormorevehicles,expensiveperceptionsystems(e.g.,LIDAR)andpersonnelcapableoftheinstallation,integrationandcommissioningofsophisticatedsensorsanddatarecordingsystems.Inaddition,thedatamustbepost-processed,andthesynchronizationofthedifferentvehicleinformationsourcesisrequired.

2.Publicdatasets.Therearedatasetsdevelopedbybusinessesanduniversitiesforau-tonomousdrivingwheredataobtainedfromtheperceptionsystemsoftheirvehiclescanbeaccessed[

10

].Someofthesepresentdiversescenarioswithdifferentlightandmeteorologicalconditions[

22

].Table

1

showssomerecentpublicdatasetsincludingnumberofsamples,typesofimagesavailableandtypesofvehiclecontrolactionsstored.

3.Simulators.Giventhecomplexityofconductingrealtests,autonomousdrivingsimulatorshavebecomeoneofthemostwidelyusedalternatives.Thesimulationindustryrangesfromsimulationplatforms,vehicledynamicssimulationandsensorsimulationtoscenariosimulationandevenscenariolibraries.Atpresent,therearemanyoptions,includinggenericsolutionswhichmakeuseofgamesandphysicenginesforsimulation[

23

]androboticssimulators[

16]

.Recentlyonthemarketcompaniesthatdevelopsimulationproductsspecificallydesignedtosatisfytheneedsofautonomousdrivinghaveappeared.SomeofthesecompaniesincludeCognata,CARLA,METAMOTO,etc.

Table1.Publicdatasetsforautonomousdriving.

Ref./Year

Samples

ImageType

LIDAR

RADAR

IMU

ControlActions

UPCT2019

78,000

RGB,Depth

Yes

No

Yes

Steeringwheel,Speed

LyftL5[

24

]/2019

323,000

RGB

Yes

No

Yes

-

nuScenes[

25

]/2019

1,400,000

RGB

Yes

Yes

Yes

-

Pandaset[

22

]/2019

48,000

RGB

Yes

No

Yes

-

Waymo[

23

]/2019

1,000,000

RGB

Yes

No

Yes

-

Udacity[

16

]/2016

34,000

RGB

Yes

No

Yes

Steeringwheel

GAC[

26

]/2019

3,240,000

RGB

No

No

No

Steeringwheel,Speed

Inthisworkadhocdatahasbeenchosen.Toobtainthedata,acustomdatasetwascreated,astheresultofadhocdrivingtestsperformedusingtheCloudIncubatorCarautonomousvehicle(CICar)[

2

](seeFigure

2),anautonomousvehicleprototypebased

ontheadaptionofthecommercialelectricvehicle,RenaultTwizy.Thevehiclehasbeenconvenientlymodifiedandhousesacompleteperceptionsystemconsistingofa2DLIDAR,3DHDLIDAR,ToFcameras,aswellasalocalizationsystemwhichcontainsareal-timekineticunit(RTK)andinertialmeasurementunit(IMU,seeFigure

2

c)andautomationofthedrivingelementsofthevehicle(accelerator,brake,steeringandgearbox).Allofthisiscomplementedwiththebiometricdataofthedriverstakenduringthedrivingtests.

2.1.DrivingTests

Agroupof30driversofdifferentageandgenderwereselectedtoperformthedrivingtests,ofwhichfivewerediscardedduetosynchronizationproblems,recordingfailureorincompletedata.ThedrivingtestswerecarriedoutinCartagenaintheRegionofMurcia,Spain,followingapreviouslyselectedroutewithrealtraffic.

Thisrouteprovidesasignificantsetoftypicalurbandrivingscenarios:(a)junctionswithrightofwayandchangesofpriority;(b)incorporation,internalcirculationandexitingofaroundabout;(c)drivingalongaroadwithandparkingareas;(d)mergingtraffic

Electronics2021,10,12664of

21

situations.Inordertocontemplateagreatervarietyofenvironmentalconditions,eachdrivercompletedtheroutetwiceatdifferenttimesofday(morning,afternoonorevening).InFigure

3

asampleofsomeofthedatasetimagesisshown,wheresomeofthedifferentdrivingconditionscapturedduringthetestscanbeobserved.

Figure2.(a)CloudIncubatorCarautonomousvehicle(CiCar).(b)CiCarondataacquisitionmission.(c)VehiclemodelandIMUsensormeasurementdetails.

Figure3.Imagesfromdataset.(a)pedestriancrossing;(b)saturationoftheilluminationonroundabout;(c)carbraking;(d)complexshadowsontheroad.

2.2.VehicleConfiguration

Asmentionedpreviously,thedatawascollectedusingtheCICarprototypevehicleinmanualmode,drivenbyahumandriver.InTable

2

thevariablesanddataacquiredduringthedrivingtestsareshown,aswellastheinformationaboutthedevicesandsystemsusedtoobtainthedata.

Eachsensorworkswithitsownsamplerate,andinmostcasesthisisdifferentbetweendevices.Toachievethecorrectdatasynchronizationandreconstructthetemporalsequencewithprecision,stampingtimeshavebeengeneratedforeachsensorandthesehavebeensynchronizedatthestartandendoftherecording.Therefore,allthedevicesarecontrolledbythecontrolunitonboardthevehicle,providingaperfecttemporalandspatialsynchronizationofthedataobtainedbythedifferentsensors.Thedatafromeachtestisdownloadedandstoredinthecentralserveroncethedrivehasfinished.

Electronics2021,10,12665of

21

Table2.CICar.Sensordata.

Variable/Unit

Device/System

Frequency

Vehicleposition/(LLA)1

4Hz

acceleration/(m/s2)

GNSS-IMU

20Hz

angularspeed/(。/s)

20Hz

Steeringwheelangle/(。)Distance/(m)

Speed/(m/s)

compactRioControlunit

50Hz

50Hz

50Hz

Frontalimage/

RGBDCamera

25fps

Driverattentionimage

RGBCamera

25fps

SurroundingsCloudPoints

LIDARs,ToFcameras

10Hz

1LLA—latitude,longitude,andaltitude.

2.3.DeepLearningEnd-to-EndArchitecturesClassification

End-to-end(e2e)systemsbasedonDNNarchitecturesappliedtoautonomousdrivingcanmodelthecomplexrelationshipsextractedfromtheinformationobtainedfromthevehicleperceptionsystem.Thisisachievedusingdifferenttypesofneuralblocksgroupedintolayers(e.g.,convolutionallayers,fully-connectedlayers,recurrentlayers,etc.),withtheaimofgeneratingdirectcontrolactionsonthesteeringwheel,theacceleratorandthebrake.Theseactionsonthevehiclecontrolelementscanbecategorical,e.g.,increaseordecreasethespeed,ortheycangenerateasetpointonthecontroller,e.g.,turn13.6degreesorreach45km/h.

Themachinelearningalgorithmsthatareusedtomodeldrivingactionsbelongtothesetknownassupervisedlearning.Thesealgorithmsacquireknowledgefromadatasetofsamplespreviouslyacquiredduringdrivingtestswithapreviouslyconditionedvehicle[

2]

orfromdrivingsimulators[

27

].Thesedatasetsincludedatafromtheperceptionsystem,suchas:images(RGBoIR),LIDAR,RADAR,IMU,aswellastheactionsperformedbythedriveronthevehiclecontrolelements,suchasthesteeringwheel,theacceleratorandthebrake.

Thegenerationofdiscretevariablesbyamachinelearningalgorithmisknownasregressionandisawidelystudiedproblem[

28]

.RegressionmodelsforDNNusethegradientdescentfunctiontosearchfortheoptimalweightsthatminimizethelossfunction.Thelossfunctionsusedforthesemodelsdifferfromthoseusedintheclassificationmodels,withthemostusedbeingthemeanabsoluteerror,meansquareabsoluteerrorormeanabsolutepercentageerror,amongothers.

Thisworkproposesaclassificationofe2earchitecturesbasedonthetypeofdatareceivedbytheDNNfromthevehicleperceptionsystem.Thisisdonebyconsideringtheimageprovidedbythevisualperceptionsystemofthevehicleasthemaindatasourceforthee2earchitecture.Basedonthetypeofnetworkinput,thearchitectureshavebeenclassifiedintothreetypes:(1)singledatae2earchitecture(SiD-e2e),(2)mixeddatae2earchitecture(MiD-e2e)andsequentialdatae2earchitecture(SeD-e2e).

2.3.1.SiD-e2eArchitecture

Thistypeofarchitectureusesasingledatasourcefortheinputlayertogeneratethesetpointsdirectlyforthecontrolelementsofthevehicle.TheSiDarchitecturesusethevisualinformationprovidedbyoneormorecameraslocatedonthefrontandperipheryofthevehicletocomposeasingleimageofthevehiclesfieldofviewofthevehicleasavisualinputtothenetwork[

15,

29,

30

].BeforebeingprocessedbytheDNN,theimagesarereducedinsizeandnormalized.Subsequently,theimagesgothroughconvolutionallayersofdifferentkernelsize(kk)anddepth(d)whichallowtheimagefeaturesthatminimizethecostfunctiontobeextractedautomaticallyinsuccessivelayers.Aftertheconvolutionallayers,theresultingvectoristransformedintoonedimension(Flayer)andconnectedtoasetoffully-connectedlayers(FC)whichhavethedecision-makingcapacity.Lastly,theFClayersendinthenumberofneuronsequaltothenumberofvariablestobe

Electronics2021,10,12666of

21

predicted[

15,

28

].Figure

4

showsanexampleoftheSiDarchitecturewherethenormalizedimagefeedsagroupofconvolutionallayerswithdifferentkernelsizes,followedbyasetoffully-connectedlayersandafinaloutputlayer.

Figure4.Singledatae2earchitecture(SiD).

Thenumberofconvolutionallayers,theirsize,paddingandstride,aswellasthenumberofneuronsintheFClayersareadjustedempirically.Theseparametersarede-pendentonthetrainingdatasetandthesizeoftheinputimages.Thereareworkswherethearchitectureshavebeendesignedusingbanksofconvolutionalfiltersofincreasingsize[

30

]andthereareotherswherethedesignistheopposite[

31,

32

].Generallyspeaking,theconvolutionallayerswithasmallkernelsizeextractreducedspatialcharacteristics,suchastrafficsigns,trafficlightsorlaneseparationlines,whilethosewithagreaterkernelsizedetectlargerelementsintheimage,suchasvehicles,pedestriansortheroad[

31]

.

2.3.2.MiD-e2eArchitecture

Mixeddataarchitecturesallowdifferentdatasourcesfromthevehicle,suchasRADAR,longitudinalandlateralaccelerations,angularvelocities,mapsorGPStobemergedtogetherwiththevisualinformationfromthevehicle’scameras.TheinclusionofmoreinformationsourcesintheDNNaimsto:(1)improvetheperformanceofthemodel,(2)improvethepredictionofspecificcasesorabnormaldriving;and(3)increasethetolerancetofailuresproducedbythedatasources[

21,

29,

33

].AsshowninFigure

5,thistypeof

architecturecombinestheresultsoftheSiD-e2e,suchasthoseshowninthepreviousSection

2.3.1

,withasetofFClayerswhichallowsthemappingofthecharacteristicsfromothervehicledatasourcesonalayerthatconcatenatesalltheinformation.

Figure

5

showsafirstinputbranchwheretherelevantinformationisextractedfromtheimagewithasecondbranchthatextractsextrainformation,forexamplefromtheIMUorGPS.Theconcatenationlayerreceivesaspecifiednumberofinputsfrombothbranchesofthemodel.Thenumberofconnectionsfromeachbranchisusuallydeterminedusingempiricaltechniques.MiDarchitectureishabituallyusedindatafusionintheperceptionsystemsofautonomousvehiclesorADAS.

Electronics2021,10,12667of

21

Figure5.Mixeddatae2earchitecture(MiD).

2.3.3.SeD-e2eArchitecture

Drivingisataskwherethefutureactionsonthevehicle’scontrolelementsdependgreatlyonthepreviousactions,thereforethepredictionofthecontrolactionscanbemodeledasatimeseriesanalysis[

16,

26,

34

].Sequentialdatabasedarchitecturesaimtomodelthetemporalrelationshipsofthedatausingfeedbackneuralunits(seeFigure

6

),thesetypesofneuralnetworksareknownasrecurrentneuralnetworks(RNN)[

34

].BasicRNNscanlearntheshort-termdependenciesofthedatabuttheyhaveproblemswithcapturingthelong-termdependenciesduetovanishinggradientproblems[

35

].Tosolvethevanishinggradientproblems,moresophisticatedRNNarchitectureshaveappearedwhichuseactivationfunctionsbasedongatingunits.Thegatingunithasthecapacityofconditionallydecidingwhatinformationisremembered,forgottenorforpassingthroughtheunit.Thelongshort-termmemory(LSTM)[

36

]andGRU(gatedrecurrentunit)aretwoexamplesofthesekindsofRNNarchitectures[

37]

.

Figure6.Sequentialdatae2earchitecture(SeD).

RNN[

15

],LSTM(longshort-termmemory)[

16

]andGRU(gatedrecurrentunit)arethemostusedformodelingthetemporalrelationshipsinthefieldofe2earchitectures.TheuseofRNNine2earchitecturesrequiresthenetworkinputdatatobetransformedintotemporalsequencesintheformoftimesteps(ts).ThepartitioningoftheNinputsamples

Electronics2021,10,12668of

21

ofthenetworkwillgenerate(N-ts)temporalsequencesthatwillcorrespondtoanoutputvectorfromthenetworkaccordingtoEquation(1):

S.input=f[I1,..,Its],[I2,..,Its+1]...,[In1ts,..,IN-1]g,

output=fots+1,ots+2,............,oNg

(1)

Figure

7

showstheprocedurestogenerateN-tssequencesofsizetsfromadatasetcomposedofNimagesandNpairsofoutputvalues(v:speed,θ:steeringwheelangle).

Figure7.Compositionofsequentialimagesandoutputvaluesdata.

TocreateamodelfromtheSeD-e2earchitectures,thiswillbetrainedwithtemporalsequencesofsizets(I1toIts)andthenextoutputvectortopredict(vts+1,θts+1)asitisshownintheFigure

7.

2.4.ParemetersofDeepNeuralNetworkArchitectures

ThenumberofparameterswhichcomeintoplayduringthedesignprocessofaDNNisenormousandwecanseparatethemintothreetypes:

(1)Networkinputparameters.Theseparametersrefertothewaythenetworkinputvaluesarepresented.Fordataintheformofimages,theshapeparametersinclude:.Normalization.NormalizationmustbeperformedonthedatabeforetrainingtheDNN.Anadequatenormalizationcanimprovetheconvergenceandperformance

ofthenetwork.Equations(2)and(3)showthemostcommontechniques.

Scaled(0,1)=(xi-min)/(max-min)(

2)

Standarized(m=0,σ=1)=(xi-m)/(σ)(3)whereminandmax,arethemaximumandminimumvaluespresentinthedatasetX={x1,...,xN},withuandσbeingtheaverageandstandarddeviationofthedataset,respectively.Thereareothernormalizationtechniques,forexample,themeancanbesubstitutedforthemodeinEquation(3),forcasesinwhichthedatadistributiondoesnotalignbelowthemean.

.Resizing.Asageneralruleandespeciallyine2earchitecturesforautonomousdriving,theimagesizeisreducedbeforebeingprocessedbythenetwork.Themainreasonforthisistodecreasethenetworkprocessingtimeandtheresourcesinvolvedintheprediction.

.Colorspacetransformations.Itiscommontotransformtheinputimagetoacolorspaceotherthantheonesuppliedbythecameratoimproveperformance,forexampleHSI,LAB,etc.,[

10]

.

.Preprocessing.Whenthedataiscapturedfromdifferentsourcesordataset,thesetendtohavedisparatefeaturesfromthedeviseitselforfromthelightingof

Electronics2021,10,12669of

21

thescenewheretheimageswerecaptured,thereforehistogramequalizationorimageenhancementalgorithmsareusuallyappliedtonormalizetheappearanceoftheentiredataset.

.Dataaugmentation.Thistechniqueconsistsinincreasingthesizeoftheoriginaldatasetinordertoachievehigherlevelsofgeneralizationandtoimprovetheperformanceofthenetwork[

38]

.

(2)Architectureconfigurationparameters.Theseparametersconstitutethecomposition

ofonearchitectureoranother,andtheseinclude:

.Typeoflayer.Thearchitecturescanstackdifferentsetsoflayersineachbranch:FC,Convolutional,RNN,Concatenated,etc.

.Layersettings.Eachlayerhasspecificconfigurations,forexample,convolutionallayerscanbeconfiguredwithdifferenttypesoffilters,33,55,...,kk,theirdepthornumberoflayers.

.Layerdistribution.Thearchitecturescanconsistofasinglebranch,canbebranchedattheinput,orattheoutputorboth.

(3)Hyperparameters.Theseparametersarecharacteristicofthelearningprocessimplicit

inDNN,andamongthemare:

.Batchsize.Numberofsamplesthatwillbepassedthroughthenetworkbeforeupdatingtheweights.

.NumberofEpochs.Numberoftimesthelearningalgorithmwillworkwiththetrainingdataset.

.OptimizationAlgorithm.Themostwidelyusedisthegradientdescentalgorithm.However,variationshavebeendeveloped,suchasAdaGradwhichadjuststhelearningrate.AnothervariantisAdam,withadaptivemomentum.

.Learningrate.Theoptimizationalgorithmallowshowmuchtheweightsareupdatedattheendofeachbatchtobecontrolled.

.