2022年增强现实(AR)技术开发指南(版)

MobiDev：2022年增强现实(AR)技术开发指

南(pdf版)

MobiDev是一家软件开发公司，致力于增强现实(AR)、

人工智能与机器学习以及物联网等创新技术领域。近日

MobiDev发布了《2022年增强现实(AR)技术开发指南》。报

告探讨了AR技术与移动AR的发展现状，分析了该技术在

2022年进一步发展将面临的障碍并给出了对应策略。

AR技术概述

AR的受众正在增长。研究机构ARtillery

Intelligence报告称，美国成年人对AR功能的移动使用量增加

了29%o这主要是受在iPhone和Android设备的原生ARKit

和ARCore架构上运行的专用AR应用程序所推动。

ARtilleryIntelligence还报告称，与完全基于增强现实的应用程

序相比，应用程序中包含的AR功能的参与度要高得多。

游戏、社交和视觉搜索是AR最受欢迎的消费者使用案

例。PokemonGo在2020年带来了19.2亿美元的收入，且正

在试验Microsoft

Hololens,其最终目标是为更多的观众带来可穿戴游戏体验。

AR技术并不局限于普通消费者，制造商和智能城市是其

瞄准的另一个重要领域。通过使用增强现实技术，可以进行实

时远程协助。智慧城市解决方案还包括导航和城市规划等

AR技术。

移动AR技术

移动设备协同处理能力、硬件摄像头和互联网连接，在

设备显示屏上为用户提供增强现实体验。这种方法可以让公司

接触到大多数用户，因为很多人已经拥有具有增强现实功能的

智能手机。事实上，据AR

Insider称，29%的美国成年人已经体验过移动AR。许多品牌

都利用AR体验来提高客户参与度。2021年，Facebook的AR

功能有2.33亿活跃用户。

开发具有增强现实功能的移动应用程序需要了解适用于

Android和iOS的ARCore和ARKit开发工具。这两个开发平

台支持几乎所有基于移动应用的AR体验。2020年，ARKit拥

有1.47亿用户，而ARCore拥有1.22亿用户;到2021年，这两

个数字分别增至1.92亿和1.82亿用户。

2022年增强现实技术发展重点

克服硬件和软件的限制是增强现实开发的主要挑战之一。

尽管AR技术将在2022年飞速发展，但仍有一些重要的障碍

需要考虑，如移动设备的硬件限制和操作系统版本。其中一个

主要挑战便是Android设备的多样性以及开发者必须面对的独

特规格。理解这些问题是找到解决方案的关键。

writtenby

AndrewMakarov

HeadofMobileDevelopmentatMobiDev

TopMachineLearningBestAgency

maglevDevelopmentCompanyinUkraine

SOFTWAREDEVELOPMENT

FORVISIONARIESWHO

CREATEPRODUCTS

AlANDMACHINEIOT&HARDWARE

LEARNINGINTEGRATION

ForStartupsForEmergingcompaniesForEnterprises

。

J一

口

一

GuaranteeddeliveryYoucanadapttoevolvingTopUS-levelquality

ontimeandonbudgetbusinessneedsandincreaseROIfor1/3thepricetobring

Nosurpriseswithourflexible,provenprocesses3xfeaturestoyourproduct

g2八八4Englishspeaking

450+™100%:Zo；rDUU■professionals

Findmoreat

白info@S/p>

UNITEDSTATESOFFICEUNITEDKINGDOMOFFICEENGINEERINGOFFICESINUKRAINE

Atlanta,GeorgiaSheffieldKharkiv,Chernivtsi,Mykolaiv

TableofContents

AugmentedRealityTechnologiesOverview

MobileAugmentedRealityTechnologies

ARDevelopmentforiOSDevices:ARKil5

LocationAnchors

DepthAPI

RaycastiagAPI

AdvancedFaceTracking

ARDevelopmentforAndroidDevices:ARCore

CloudAnchors

ARFaces

ARImages

RecordingandPlaybackAPIs

ARKi【vsARCorefeaturescomparison

AugmenledRealityDevelopmentForWearables

MicrosoftHoloLens

AugmentedRealityDeveloDmentforSmartGlasses

Web-BasedAugmentedRealityTechnologies

ExampleofWebAR:DellImmersivePortals

LimitationsofWebAR

Where」oStaHDevelopinganARApplication

AugmentedRealityTechnologiesin2022:MainTakeaways

Withtheaugmentedrealitymarketvaluingat$9.5billionUSDandgrowing,this

isanincrediblyimportantsegmenttotakeadvantageoftoremaincompetitive

inARtechnologies-drivenapplicationsforanaudienceof810millionactiveAR

usersonmobile.ThisAugmentedRealityDevelopmentGuidewilltakeyou

throughthemaintechnologies,devicesandplatformsforARappdevelopment

in2022.

1

AugmentedRealityStatisticsin2021

Source:

MOBILEARMARKETSIZE2021

9.5bnUSD

NUMBEROFMOBILEARACTIVEUSERS2021

810m

ARCLASSESUNITSALES2021

410k

mobidev

AugmentedRealityTechnologiesOverview

TheaudienceforARisgrowing.ARtilleryIntelligencereportsthatthemobile

usageofARfeaturesisup29%foradultsintheUnitedStates.Thisismostlydue

todedicatedARappsrunningonthenativeARKitandARCorearchitecturesfor

iPhoneandAndroiddevices.ARtilleryIntelligencealsoreportsthatthereismuch

moreengagementwithARfeaturesthatareincludedwithinappsratherthan

appsbasedentirelyaroundaugmentedreality.

Gaming,social,andvisualsearcharethemostpopularconsumerusecasesfor

AR.It'snosurprisethenthatPokemonGobroughtin$1.92billionUSDin2020

andisexperimentingwithMicrosoftHololenswiththeeventualaimofbringing

wearablegamingexperiencestoalargeraudience.

2

Thetechnologydoesn'tstopwithgeneralconsumers.Manufacturersandsmart

citiesareanotherimportantsegment.ByusingARtechnologies,remote

assistancecanbeperformedinrealtime.Thisallowsforbetterinspectionand

maintenance,aswellastechnicalsupport.SmartcitysolutionsalsoincludeAR

technologieslikenavigationandurbanplanning.

Letstalkmoreaboutthesegroundbreakingandadvancingtechnologiesinour

2022augmentedrealitytechnologyguide.

Augmentedrealityencompassesmanyfacetsofdifferentindustries.Oneofthe

mostusefulwaysofdifferentiatingaugmentedrealitytechnologiesisviathe

hardwarethatisusedastheplatform.Generallyspeaking,thesearemobile

devices,smartglasses,head-mounteddisplays,andwebbasedARfordesktop

andmobiledevices.

AugmentedRealityHardware

MobileSmartImmersiveHeadWeb

DevicesClassesMountedDisplaysBasedAR

SmartphonesGoogleGlassMicrosoftDesktop

TabletsEnterpriseEditionHoloLensMobile

motiidev

Augmentedrealitydevelopmentisveryaccessibleandsimpleapplicationsare

veryeasytogetinto.Togetstarted,abasicunderstandingofgeometryand3D

graphicsisallthat'sneeded.However,morecomplexprojectscanrequiremuch

moreexpertise.MachinelearningandDataSciencecanalsoplayamajorrolein

thedevelopmentofmoreadvancedAR.

3

MobileAugmentedRealityTechnologies

Mobiledevicessynergizetheprocessingpower,hardwarecamera,andInternet

connectiontodeliveraugmentedrealityexperiencesforusersonthedevice's

display.Thismethodallowscompaniestoreachthemostusersbecauseso

manypeoplealreadyownARcapablesmartphones.Infact,accordingtoAR

Insider,29%ofU.S.adultshavealreadytriedmobileAR.Manybrandstake

advantageofARexperiencestodrivecustomerengagement.In2021,Facebook

saw233millionactiveusersofitsARfeatures.

MobileAR2020-2021:ARKit♦ARCore

Developingaugmentedreality-capableappsformobilerequiresan

understandingofARCoreandARKitdevelopmenttoolsforAndroidandiOS.

Thesetwodevelopmentplatformspowernearlyallapp-basedARexperiences

onmobile.In2020,ARKithad147millionuserscomparedtoARCore^122

million,andin2021thesenumberswere192millionusersand182million

users,respectively.

ARCore'slargerdiversityofAndroiddevicesnecessitatesthatitbemoreflexible

fordifferentsmartphones.LastyearwesawGooglebegintocrowdsource

GoogleMapsStreetViewphotossolongastheymetcertainguidelines.ARCore

softwaremadeitpossibleforuserstotakephotoswiththequalityGooglewas

lookingfortoenhancetheirStreetViewexperience.

4

ARKitseemstooutpaceARCorebyasignificantmargin,butbothhavegrownat

asignificantpacesincetheirinitialintroduction.From2017to2018,13million

downloadsontheAppleAppStorewereAR-kitonlyapps.

ArepositorysearchonGithubshownbelowsuggeststhatdeveloperspreferto

buildappswithARKitoverARCore.

DateARKitrepositoryresultsARCorerepositoryresults

May20203,9671r483

May20214,5041,874

January20224,7702,062

ARDevelopmentforiOSDevices:ARKit5

Letstalkaboutafewnewfeaturesthatcanbeinvaluableforthesuccessofthe

developmentofyourARKitapp.

LOCATIONANCHORS

ARKit4introducedlocationanchors,acrucialfeaturethatallowsdevelopersto

affixvirtualobjectsintherealworldbyusinggeographiccoordinates.Thisallows

forfurtherimmersionthroughvisuallocalization.Withlatitude,longitude,and

elevation,virtualobjectscanbeplacedintotheworldwithevenmoreaccuracy

thanbeforewithgeographicinformation.InARKit5locationanchorshaveseen

someimprovements.

Forexample,locationanchorscoulddisplayathreedimensionaliconortextin

spacenexttoaniconicbuilding.Supportforthisfeaturehasbeenexpandedto

LondonandothercitiesintheUnitedStatesinARKit5.

Oneofthemostgroundbreakingelementsoflocationanchorsishowlocationis

approximated.GPSissimplynotpreciseenoughtoprovidelocationanchorson

auser'sscreen.Toassist,developerscanlookaroundwiththeircameratoallow

thedevicetoscanforfeaturesonbuildingsaroundthem.Byusingthese

architecturalfeaturesinconjunctionwithAppleMapsLookArounddata,ausers

locationcanbebetterapproximatedforlocationanchors.

5

SincelocationanchorsaredependentonAppleMapsdata,thismeansthatifthe

cityisnotsupportedbyAppleMaps,functionalityforlocationanchorsmaybe

limited.

DEPTHAPI

DepthAPIisanothervaluablefeatureofARKit4whichcontinuestoplayan

importantroleinARKit5.Itutilizesoneofthemostpowerfulhardwarefeatures

forARonamobiledevice,theLiDARscannerontheiPadProandPhone12Pro,

12ProMax,13Pro,and13ProMaxsmartphones.Thisenablesmuchbetter

analysisofscenesandallowsforrealworldobjectstooccludevirtualobjects

withmuchbetteraccuracy.

RAYCASTINGAPI

Apple'sRaycastingAPIenablesenhancedobjectplacementinconjunctionwith

depthdata.Thisallowsmuchmoreaccurateplacementofvirtualobjectsona

varietyofsurfaceswhiletakingintoaccounttheircurvatureandangle.For

example,it/spossibletousethistoplaceanobjectonthesideofawalloralong

thecurvesofasofaratherthansimplyflatonthefloor.TheLiDARsensorallows

forthescanningofsurfacestotakeplacemuchmorequicklycomparedto

traditionalmethods.

ADVANCEDFACETRACKING

Duetomorepowerfulsoftware,facetrackingtechnologythatwasoncelimited

toTrueDepthcamerascannowbeusedwithiPhoneswithanA12Bionic

processororgreater.Thisallowsfortoolssuchasfaceanchorstobe

incorporatedintoappfeatures.AlsofacetrackingisnowsupportedbytheUltra

WidecamerainthelatestiPadPro(5thgeneration).Thisfeatureallowsfor

trackingofuptothreefacesatonce.

ThevideobelowshowshowiPhoneemploysAR-basedgaze-trackingtechnology

thatcreatesusecasesfordevicecontrolperformedwitheyes,usedforserving

userswithlimitedabilitiestoallowforahands-freeexperienceandmonitoring.

Combinedwithcomputervision,face-basedARopensupfurtheropportunities

forfacialrecognitionapps.

6

ARDevelopmentforAndroidDevices:ARCore

Inanefforttoremaincompetitive,GooglehaspushedARCorefartoremainone

ofthemostversatileARdevelopmentplatformsintheworld.Letsexploresome

ofthefeaturesthatareusedbydevelopers.

CLOUDANCHORS

Thistoolallowsuserstoplacevirtualobjectsinphysicalspacethatcanbe

viewedbyotherusersontheirowndevices.GooglehasmadesurethatCloud

AnchorscanbeseenbyusersoniOSdevicesaswell.

ThefollowingMobiDevdemodemonstrateshowARCoreobjectdetection

featuresforavirtualusermanual.

AugmentedRealityUserManualSfliObidEvo4

WatchlaterShare

VirtualManualviaAugmentedReality

info@

WatchonQYouTube

ARFACES

ARCoreenablesdeveloperstoworkwithhigh-qualityfacialrenderingsby

generatinga468-point3Dmodel.Masksandfilterscanbeappliedafterthe

user'sfaceisidentified.ThisisoneofthemostpopularusecasesforARapp

features.

ARIMAGES

Virtualbusinesscardsandadvertisingpostersarejustafewoftheapplications

thatarepossiblewithaugmentedrealityimages.2Dmarkerscanbeusedto

implementthesefeaturesaswellasmoreadvancedsolutionslikeARindoor

navigation.

LearnhowanARCore-basedindoornavigationapplicationworksbywatching

thisMobiDevdemo.

AugmentedRealityIndoorNavigationDemo-ARCorebased.口£JIjiL®"

Ll一224、xIWal金■旭®Share

AugmentedReality

arindoornavigation

WatchonOYouTubeinfo@

RECORDINGANDPLAYBACKAPIS

Lastyear,GoogleintroducedanewfeatureforARCorecalledrecordingand

playbackAPIs.ThisallowsdeveloperstorecordvideoswithARmetadata.Using

theIMU'sdepthandmotionsensordata,developerscanrecreatethesame

environmentasinthevideoanduseitforfurthertesting.

UserscanalsobenefitfromrecordingandplaybackAPIs.Forexample,usingthis

feature,userscanapplyAReffectsonpreviouslyrecordedvideos,sincethe

videocancontainmetadataabouttheenvironment.

8

ARKitvsARCorefeaturescomparison

AppleandGooglehavebothpositionedtheirrespectiveARplatforms

competitivelyagainsteachother.Thismeansthatbothwilllikelybecomparable

incapabilityfortheforeseeablefuture.ApplehaspositionedARKitwithaslight

advantagegiventhebettercontroloverthesoftwareandhardwareoftheir

devices.TheLiDARsensorisoneofthemostadvanceddepthsensing

technologiesonamobiledevicecurrentlyavailable.However,itwon'tbelong

beforeAndroiddevelopersstarttopickupthepace.

GoogleandApplearebattlingoverplatforms,installedusers,andmoney.

Apple'slifestylebrandimagehasattractedagreatdealofdeveloperattention

sinceitisperceivedasabettersourceofrevenueforstartupsneedingfunding.

AlthoughdevelopersfavorARKit,Google'sARCore'smarketsizeismuchlarger

andgrowing.

ComparisonofARKitandARCorefeaturesin2022

FMtur*ARKHARCor*

WorWTracking••

imageXtacking••

faceTracking••

CsoTracking••

BodyTracking••

PtenceDetection••

Sc«n«Depth••

ObjectDetection••

SharedAR••

MMhing(UOAR).•

■th*IwtureHp,eMnte<lonthaplMlormandK«ff»ctive'th«toMurei*I«Mornot«v«4abletotdcvttofMre

•tb«fMtur*i»notpm«nt«donth*piMform

Sincetheyaimtoprovidecompetitivefunctionality,bothplatformsoffersimilar

featuresandperformancesuchasmotionsensors,understanding

environments,andinspectingchangesinlightingconditions.

Inthefollowingdemovideo,wedemonstratehowARKitandARCoreperformin

recognizingandaugmentingimages.

9

WatchonOYouTube

ByputtingtheMonaLisapaintingintothecameraframe,theappisableto

animatethesubjectbymakingitblinkhereyes.Thevideodemonstratesthe

differencesinperformancebetweenARKitandARCoreinexecutingthistask.

Despitetheirdifferences,thewaythatARscenesarecalculatedinARCoreand

ARKitarenearlyidentical.However,it'smostlikelythatthereasonwhyARKit

outpacesARCoreinperformanceisbecauseApplehasmuchmorecontrolover

thehardwareandsoftwareoftheirdevices.SincetheydevelopbothiPhones,

iPads,aswellasiOS,ApplecanbetteroptimizethemforARexperiences.

It'scrucialtoensurethatadevelopmentteamisusingthelatesttools,technologies

andapproachesontheproject.Usingoldertoolsposesagreatriskasthere'salways

thechancethattheywillbecomeoutdatedandunsupportedinthenot-too-distant

future.There'salsotheriskthatoutdatedsoftwarewillaffecttheimplementationof

newfeatureslaterdowntheroad.Whileitmayworkperfectlynow,youmustthinkof

thefuture.

RomanMarkov

//

Android/iOSExpert

10

AugmentedRealityDevelopmentFor

Wearables

Whenwetalkaboutwearabletechnologyforaugmentedreality,wetypicallyare

referringtodisplaytechnologystrappedtoone'sheadorapairofglasses.

Althoughtheshapeandformfactorofthistechnologycanvary,themost

popularformsaregearlikeMicrosoftHoloLensandmoreportableand

comfortableglasseslikeGoogleGlass.

Thereisabigdifferencebetweenthesetwotypesofdevices,notonlyinpurpose

butinpotential.Largerhead-mounteddisplayslikeHoloLensarecapableof

muchhigherprocessingpower.Thisenablesthemtoprovideexperiences

otherwisecurrentlyimpossibleformorecompactdeviceslikeGoogleGlass.

Becauseofthesmallerformfactorandreducedprocessingpower,ARglasses

aremorefocusedonprovidingtwodimensionalexperiencessimilartothatof

yoursmartphone.Forexample,displayingnotifications,text,directions,photos,

inyourfieldofvisionaresolutionsthatARglassescanreasonablyprovide.

Voice-basedInternetsearchandcontrolarealsofeasible.GoogleGlassandthe

nowdiscontinuedFocalsbyNorthonlyprojectthisinformationintooneeyein

twodimensions.Futureglassesmaybeabletoprojectthisinformationintoboth

eyes.

Hearablesarealsoenteringrelevancy.Technologieslikespatialaudioare

alreadymakingtheirwayontoearbudsliketheAirPodsPro,andmore

office-centrichearableslikeMicrosoft'sSurfaceEarbuds.

Glassesandlargerhead-mounteddisplayshavetwodifferentrolesinthe

marketatthemomentwithspecificusecases.Let'stalkaboutbothofthemin

moredetail.

MICROSOFTHOLOLENS

HoloLenswasfirstannouncedin2015,howeveritsinitialversionwas

discontinued.CurrentlyMicrosoftisofferingHoloLens2tobusinessconsumers

only.Itisbatterypoweredanddoesnotrequireacabledconnectiontoanother

device.AlthoughHoloLens2hasshownenoughpromisetobeacquiredbythe

11

militaryina$22billiondollardeal,thelimitationsandcostoftheproductare

largebarrierspreventingitfrombeingadoptedmorewidely.

Organizationsfacesignificantchallengeswhenadoptingtechnologieslike

HoloLensatscale.Incorporatingnewtechnologiescanbedifficultforbusinesses

becausetheyneedtimeandefforttoadjustandlearntousethem.HoloLens

requiresanentirelynewwayofworkingwithothersandinformation.Thiscalls

intoquestionwhattheimpactwillbenotonlyonthebusinessitselfbutonthe

qualityofitscustomerservice.

MagicLeap,anARheadsetpositionedasapotentialcompetitortoHoloLens,

wasdiscontinuedin2020.

AUGMENTEDREALITYDEVELOPMENTFORSMART

GLASSES

SmartglasseshavegreatpotentialtobecomewhattheiPhonewasfor

consumersintheaugmentedrealityspace.Beingabletohaveacomfortableto

wearandfashionableplatformforaugmentedrealityexperiencesthatcouldbe

massproducedandaffordableforconsumerswouldbegamechangingfor

manyindustries,especiallyretail.

ARexperiencesonsmartphonesareusefulandhelpful.However,viewingvirtual

objectsthroughasmartphoneasaportalcanbeinconvenient.Itsmuchmore

comfortableandrealistictoseevirtualobjectsthroughyournaturalfieldof

visionviasmartglasses.Thiscouldfundamentallychangemanyindustries,

especiallyretail.

However,therearemanychallengeswhenattemptingtomaketechnologythis

sleekandthispowerful.FashionableARglasseswon'tbeabletodonearlythe

samekindsofgraphicallychallengingtasksthatlargerheadsetscan.

FocalsbyNorthwereanotableARGlassesstartuprecentlyacquiredbyGoogle.

Theywereambitiousbutlimited,andhavesincebeendiscontinued.This

suggeststhatGooglemaybeinterestedinbringingbacksmartglassesintothe

mainstream.DreamGlass4Kisanothersmartglassesprojectthatcanbepaired

withasmartphone.

12

Web-BasedAugmentedRealityTechnologies

Althoughthemajorityofusersareexperiencingaugmentedrealitythrough

mobileapps,thisisnottheonlywaytoexperiencethetechnology.Web-based

ARisamoreaccessibleavenueofdevelopmentthatdoesnotrequireusersto

downloadapps.Instead,theARexperiencesareloadedthroughawebbrowser

onadevice.

AR-compatibleDevicesandActiveUsers2020-2021

mobibev

Thisiscrucialduetoappretention.Apptentivereportsthatafterjustonemonth,

anappmayloseonaverageabout35%ofitsusers.Ifusersdon*thaveto

downloadanapptoexperienceARcontent,theywillbemuchmoredriven

towardengagingwithit.

EXAMPLEOFWEBAR:DELLIMMERSIVEPORTALS

DelPsWebARImmersivePortalsareagreatexampleofwebAR.Whenstepping

throughthesevirtual4portals#placedinphysicalspace,userscouldimmerse

themselvesinanewdestinationelsewhereintheworld.AR3Dbusinesscards

areanotherusefulapplicationcreatedbyAircards.Theseexperiencesare

startedbyscanningaQRcodeonthebackofthebusinesscard.Whenviewed

throughadevice,thebusinesscardbecomesfullyanimatedin3Dspace.

ThisMobiDevdemoshowshowAR3Dbusinesscardswork.Youcanalsolearn

moreaboutithere.

13

LIMITATIONSOFWEBAR

SinceWebARapplicationsareextremelylimitedbythewebbrowsersthattheyrunin,WebAR

isnotagreatsolutionforcomplexprojects.WebARmaybesufficientforsimpleprojectssuch

asvirtualbusinesscards,simplefacetrackingandothersolutions.However,formore

complexoperations,mobileARismuchmorecapablesinceitcanunlockthefullpotentialof

themobiledevicethatitsrunningon.Hopefullyinthefuturetechnologywillimproveto

make]AfebARmorecapabletoallowformoreaccessibleARexperiences.

ThekeyforsuccessofWeb-basedARwillbeagreaternumberofdevicestosupportitand

morewebsitestohaveitenabled.Althoughthesestandardshaveyettobeestablished,the

implementationofARinbrowsersisalsoperformedbymeansofeitherportingexisting

libraries(e.g.AR.js)ordevelopingnewones(e.g,A-Frame,React360).

YuriyLuchaninov

JavaScriptGroupLeader

14

WheretoStartDevelopinganARApplication

Onceyou'vedecidedtodevelopanARapp,ther