车端AI实时分析系统-2024-06-智能网联

REAL-TIMEANALYTICSFORENDPOINTAIIN

AUTOMOTIVESYSTEMS

JUNE2024

KURTDEKOSKI

BUSINESSDEVELOPMENTANDPRODUCTMANAGEMENTFORAUTOMOTIVESWSEPBD

RENESASELECTRONICSCORPORATION

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

from

AplatformpoweringEdgeAI

onRenesasprocessors

MLModel

Development

Explainability+

HardwareAnalytics

Solutions+

ReferenceDesigns

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page2

REALITYAIPLATFORMCOMBINESADVANCEDSIGNALPROCESSINGANDMACHINELEARNINGONMCU/MPUEDGENODES

AdvancedSignalProcessing

Automaticallysearchesawiderangeofsignal-processingtransformstocreateacustom,optimizedfeaturetransform

ArtificialIntelligenceandAnomalyDetection

Automaticallygeneratesmachinelearningmodels,explanatoryvisualizationsand

hardwaredesignanalytics

MCU/MPUEdgeNodes

RunsonalmosteveryMCUandMPUcoreavailablefromRenesas,withnewones

addedconstantly.RealityAIalsosupportsRenesasmotorcontrolboards.

SCALABLEFROM16-BITto64-BITCORES

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page3

AI/MLTECHNOLOGY

ChangingIndustryNeedsDrivingAutomotive

ElectrificationAutonomous

Driving

Increaseddrivingsafety

Decreasedrelianceondriver

SoftwareDefinedVehicleDigitizationandAI

Software-firstdevelopmentstructure

Sensorfusion

BenefitsofAIandML

Reducedhumanerror,especiallyinprocessesthatinvolveanalyzingnumericalorsignaldata.AIwon’tmakeoversightsduetofatigueorerrorasahumanmight.

Increaseoperationalefficiencybyautomatingmorefunctionsandidentifyingandconsolidatingredundantprocesses.

RealityAIprovidesthesmallestmodelsinthemarketintermsofRAM,Flashandprocessingcycles

RealityAImodelsaresupportedforRenesasMCUs

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page4

AUTONOMOUSMEGATREND

INCREASEDLEVELOFAUTONOMY,REDUCEDLEVELOFDRIVERENGAGEMENT

INCREASEDLEVELOFFEATURESTOMIMICDRIVERPERCEPTION

AIsoftware:AIforself-drivingcarsusingsensor-fedmachinelearningalgorithms

AVhardware:Sensorsprovidethekeydatainputstothecar’sAIsystems

Level5Autonomy

FullAutomation

Thevehicleiscapableof

performingalldrivingfunctionsunderallconditions.Thedrivermayhavetheoptiontocontrolthevehicle.

Level4Autonomy

HighAutomation

Thevehicleiscapableof

performingalldrivingfunctionsundercertainconditions.Thedrivermayhavetheoptiontocontrolthevehicle.

Level2Autonomy

PartialAutomation

Vehiclehascombined

automatedfunctionslike

accelerationandsteering,butthedrivermustremain

engagedwiththedrivingtaskandmonitortheenvironmentatalltimes.

Level3Autonomy

ConditionalAutomation

Driverisanecessitybutisnot

requiredtomonitorthe

environment.Thedrivermustbereadytotakecontrolofthevehicleatalltimeswithnotice.

LevelsofvehicularautonomydefinedbytheSocietyofAutomotiveEngineers(SAE)

Page5

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

MostlyuseAIfordifficultproblemswheretraditionalmethodsaren’tworkingandtherearenoobvioussolutions

Instrumentationandhardwareissuesaddcomplexity,mostoptimizeforcost

Cannotdeploy“blackbox”solutions

REALITYAISOFTWAREANDSOLUTIONSADDRESSCOMMONENGINEERINGFRUSTRATIONSWITHAI/ML

Engineers’frustrationswithAI/ML

HowRealityAImakesitbetter

Builtspecificallyfornon-visualsensingbasedonadvancedsignalprocessingmathandedge

deploymentonRenesasMCUs

Analyticstosupporthardwaredesign(notjustalgorithmsandmodelbuilding)

Explainabilitybasedontime-frequencycharacteristicswithfulltransparency

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page6

REALITYAIPRODUCTPORTFOLIOOFFERSBOTHSOFTWAREANDSOLUTIONS

Software

RealityAI

Softwareformodelcreationandhardwareoptimizationon

RenesasMCUs,MPUsand

motorcontrolkits

RealityAITools®

Buildmodels,visualizeexplanations,optimizehardwarebuild

RealityCheckTMMotor

Forembeddedpredictivemaintenanceandcontrolfeedback

RealityCheckTMAD

Forcloud-basedanomalydetectioninfactoryandon-line

Completeframeworkforspecificusecases,includinghardware,firmware,softwareandML

referencedesigns

RealityAISolutions

RealityCheckTMHVAC

Completeframeworkforsmart,self-diagnosingHVAC

AutomotiveSWS

Completeframeworkforaudio-basedADASsensingforvehicles

RenesasCore

ArmCore

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page7

REALITYAIPRODUCTPORTFOLIOOVERVIEW

RealityCheckMotor

AnomalyDetection

SWS-EVD

GettingStartedPackage

墓RealtyAIToolse

RealityCheckAD

DemoVideos

SoftwareManual

ApplicationNotes

SignalClassification

RealityCheckHVAC

Regression

(continuousvalueprediction)

NativedeploymentavailablenowforallRenesasRA,RX,RZ,RL78,RH850,andR-Carfamiliesofdevices

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page8

REALITYAIADDRESSESTHEFULLAIOTDESIGNLIFECYCLE

Only5%oftypicalprojectcostsarespentonbuildingmodels

⃞RealityAlToolse

Softwarehelpswiththeother95%

SensorSelection

andBOM

Optimization

Algorithmically-DrivenFeatureDiscovery

墓RealityAI'Tools⃞software

DataReadiness

AIExploreTM(AutoML)

EdgeAI/TinyMLCodeOptimization

Instrumentation&hardware

development,datacollection,

validationandinternal

confidencebuilding

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page9

RENESAS/REALITYAIFAMILY

InputSignal

Application

•AutoML

(nocoding)

•Explainability

AIExploreTM(AutoML)

Application

Software

Stack

SensorSelection

andBOM

Optimization

Customer's

•Cost-optimizedspecifications

•Minimumsensorset

ML/AI

Generated

Code

DataReadiness

•Automated

•Consistency

•Quality

•Coverage

EdgeAI/TinyMLCodeOptimization

OutputApplicationTriggers

•Embeddedcode

generation–C,C++

•Easeofdeployment

•MATLABcompatibility

NativedeploymentavailablenowforallRenesasRA,RX,RZ,RL78,RH850,andR-Carfamiliesofdevices

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page10

REALITYAITOOLS®–APPLICATIONINPUTSIGNAL

Fullyexplainable.“Whentheresonancepeakat146hz

disappears,weknowthefanisblocked”

30+dBshiftin

100dBSiren@15m

100dBSiren@0m

overallnoiselevel

Image

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page11

“GETSTARTED”WITHREALITYAIPRODUCTPORTFOLIO

Mostcustomersstartwithoneofour“GetStarted”packagesthatcombinesoftwarewithenhancedonboardingandservices,beforeswitchingtoanannualsoftwaresubscription

StarterPoCPrototype

Idon’thavedataIhavedata

4-monthPrototypePoC

•8-weekTrialRunwithaccessextendedtofourmonths

•EnhancedsupportfromRealityAI,includingpre-processing,model

constructionandpost-processing

•Constructionoflaptop-basedorothernon-embeddedprototype/demo

6-monthEdgeAIPrototype

•4-monthPrototypePoC,withaccessextendedtosixmonths

•DeliveryofEdgeAImodelcompiledforCortexM-classarchitectureorLinux

•Assistancewithhardwareand

applicationintegration/debugging

8-weekTrialRun

•Requirementsanddataassessment

•Onboardingandsetup

•TwoiterationsrunbyRealityAIteam

•Usertrainingonowndataandmodels

•Uptofourweeksofaccessfor

evaluationandcontinueddevelopment

5-monthPoC

•8-week“Phase0”

•Monitoringofclientdatacollection

•FouriterationsrunbyRealityAIteam

•Usertrainingonowndataandmodels

•ValidationstatisticsforevaluationofPoCresults

7-monthWorkingPrototype

•8-week“Phase0”

•Monitoringofclientdatacollection

•5-monthPOC

•Constructionoflaptop-basedoranothernon-embeddedprototype/demo

8-week“Phase0”

•Requirementsanddataassessment

•Datacoverageandcollectionplan

•Instrumentation/hardwareplan

•FeasibilityPoCplan

•R&Droadmapwithmachinelearning

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page12

REALITYAIDEVELOPMENT

WITHRENESASAUTOMOTIVEMCUsandSoCs

RenesasCore

16-bitMCU32-bitMCU

POWER

SUPERLOWPOWER

EFFICIENCY

RenesasmeetsthevariedrequirementsofourcustomerswitharichlineupofautomotiveMCUsandSoCs

Arm®Core

32-bitMCU

SCALABLEWITHSoC

64-bitSoC

HIGHPERFORMANCE

Embedded

AI-enabled

Solution

RealityCheck

Motor

CBM

EVDriveMotor

SWS-EVD

RealityAITools®

RealityCheckAD

OD-GR

BMS

RealityCheck

HVAC

MassProduction

Sampling

Development

Concept

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page13

AUTOMOTIVE,CONSUMERANDINDUSTRIALIOTAPPLICATIONS

RealityAITinyMLmodelsrunoneveryRenesasMCU

AutomotiveSWS

Completeframeworkforaudio-basedADASsensingforvehicles

RealityCheckHVAC

Completeframeworkforsmart,self-diagnosingHVAC(nowengagingwithearlyadoptercustomers)

Diagnosethesourceof

Remotelivestockbehaviormonitoring

Airconditionersthatpredicttheirownmaintenanceneeds

Hearapproachingemergencyvehicles,carsandbicycles

powerqualityissuesbasedon3-phaseAC

Detectweather,terrain,roadconditionsand

micro-collisions

Maketiresthatreporttheirstateofwear

HomeApplianceelectric

motorsthatmonitorthehealthofthesystemstheydrive

Predicte-bikechaindeteriorationPumpsthatknowtheyaredry

orcavitating

RealityCheckMotor

Forembeddedpredictivemaintenanceandcontrolfeedback

Filtersthatknowhowlonguntiltheywillclog

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page14

TARGETUSECASE

SEEINGWITHSOUND

Environmentparticipantsemitsoundsthatcanoftenbeheardbeforetheyareseen

•Aroundcorners

•Obstructedviews

•Atadistance

•Vehicleblindspots

•Whereothersensorscannotsee

•Wheredriversarenotfocused

Usingsounds,theAItechnologyforSWS

providesearlywarningofemergencysirens,on-

comingvehiclesandothersoundsourcestoaugmentexistingautonamousdrivingandADASsystems

SWS-EVD

WhySWS–Contributingfactorsandcauses:

DistractedDriving

AutomotiveApplicationFeatures

EmergencyVehicleDetection+BlindSpotDetection

+OtherVehicles+HornHonks

ActiveandPassiveNoiseCancelation

+Pedestrians+Cyclists

AutonomousEmergencyBraking–Pedestrian/Cyclist

CyclistDooringPrevention

+PavementConditions/Terrain

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page15

TARGETUSECASE

SEEINGWITHSOUND

Vehicles30m–obstructedview/intersectionwithmoderatenoise

DemonstrationVideo

Blindspots

Sirens1km–normaloperatingconditionsandspeeds

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page16

TARGETUSECASE:CONDITIONMONITORING

ROADCONDITIONSANDTERRAIN

Targetingvariousnoisepatternsthatindicatedifferingroadsurfacesthataffectdrivinghabits

DryRoadSurface

WetRoadSurface(howmuch)

Snowy/IcyRoadSurface

GravelorSandRoadSurfaces

Cobblestonesurfaces

DevelopmentLevel=PoC

墓RealityAIToolse

Arm®Core

RenesasCore

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page17

TARGETUSECASE

MICRO-COLLISION

Detectionofsmallimpactsagainstthevehicle

GlassBreakage

Scrapes/Scratches

SmallDings/Dents

DevelopmentLevel=PoC

墓RealityAIToolse

Arm®Core

RenesasCore

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page18

TARGETUSECASE

AUDIOANDVOICECOMMANDS

Identifyoccupantsforindividualizedsoundzone

Clearcommunicationfrom

fronttorearoccupants

PresenceDetectionofeachoccupantandseatinglocation.

AllowsfortargetedSoundZoneforaudioperformance

Allowsforcontrolledin-cabincommunication

Anti-spoofing

Recordedverseactual

AI/MLtrainedVoicebasedUserAccessControl

Voice

AntiSpoofing

Anti-SpoofingDemo

Arm®Core

墓RealtyAIToolse

RenesasCore

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page19

TARGETUSECASES

PRESENCEDETECTIONANDDRIVER/OCCUPANTBEHAVIORANDPROFILING

GestureRecognition

OccupantMoving

OccupantBreathing

UseAI/MLtolearnGestures

OccupantDetection

AI/MLtrainedtorecognizehumansounds

PresenceDetection

Method1:Sound

Video

RenesasCore

PresenceDetectionandOccupantBehaviorandProfilingMethod2:DopplerRadar

Video

Method3:UWB

UsesdopplerradarorUWBtodistinguishbetweenhumans,petsandinanimateobjects(shiftinginseat,jostling,breathing,gestures,etc.)

©2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page20

RealityCheckMotor

AI/MLUnbalancedLoadDetection+Sensor-lessFOC

UnbalancedDetection

NOADDITIONALSENSORS!

LeveragesAvailableElectrical&MotorSignals

BalancedUnbalanced

12mmM4screw,11mmoffcentertocreateunbalancedcondition

Motor

Speed/Torque

AI/MLMotor

Speed/Torque+Sensor-less

FOC

LowSpeedHighToque

HighSpeedLowTorque

TARGETUSECASES:CONDITIONMONITORINGELECTRICMOTOR–(RealityCheckMotor)

AI/MLMotor

Mis-alignment+Sensor-less

FOC

Aligned

Mis-Aligned

Motor

Mis-alignment

DevelopmentLevelforSmallmotors=Pre-Production

DemoVideo

Enablessensor-lessMLmodelstobedeployeddirectlytothemotorcontrolboard:

•Useselectricalinformationalreadyavailableontheboard

•Takeshighsample-ratecurrentandvoltageasaproxyextracomponents(like