深度学习会引领到人工智能吗

Will

Deep

LearningLeadto

AI?Haytham

FayekML

/

AI

Melbourne

Meetup

-August

2017What

is

Artificial

Intelligence?What

is

Intelligence?Work

In

ProgressThe

ability

to

perceive

information,

andto

retain

it

as

knowledge

to

be

appliedtowards

adaptive

behaviors

within

anenvironment

or

context.3Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?What

is

Artificial

Intelligence?Think

Rationally Act

Rationally4Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?[*]

Images

from

variousonline

sourcesThink

Like

Human

Behave

Like

HumanAI5Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?LearningMeta-LearningPerceptionAttentionMemoryReasoningPlanningEmotionCommunicationNarrow

AIvsStrong

AIWhat

isDeep

Learning?The

hierarchical

learning

of

feature

representations6Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?[*]

Images

from

variousonline

sourcesNotationNotationLinear

Operation:Non-Linear

Operation:Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?8NotationAFully-Connected

Layer:A

Convolutional

Layer:Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?9NotationxhyHaytham

Fayek,

Will

Deep

Learning

Lead

to

AI?10NotationxhyFind

W

that

minimizes

L:Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?11Notation[*]

/neural-network-zoo/Feedforward

Neural

Network(DNN)Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?12Convolutional

Neural

Network(ConvNets)Recurrent

Neural

Network(RNN)Long

Short-term

Memory(LSTM-RNN)Residual

Network(ResNets)LearningLearningSupervised

LearningPick

a

model,

loss

function

&

training

scheme.Train

a

large

model

on

lots

of

labeled

data.[1]

Zeiler

&

Fergus,

Visualizing

and

Understanding

Convolutional

Neural

Networks,

2013Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?14LearningReinforcement

LearningHaytham

Fayek,

Will

Deep

Learning

Lead

to

AI?15LearningReinforcement

Learning[2]

Heesset

al.,

Emergenceof

LocomotionBehaviours

inRich

Environments,2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?16PerceptionPerceptionSupervised

Learning

has

made

strides

in

Perception:Speech

Recognition,

Computer

Vision,

Natural

Language

Processing.Each

modality

requires

a

fair

amount

of

specialized

engineering.18Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?[*]

Images

from

respectivepapers:

He

et

al.2015,

Hannun

et

al.

2014PerceptionA

single

model

with

good

results

onmultiple

tasks

in

multiple

domains.Tasks:

Object

Recognition,

translation,image

captioning,

speech

recognition,parsing,

..Components:Convolutional

LayersAttention

MechanismMixture

of

Experts[3]

Kaiser

et

al.,

One

Model

toLearnThem

All,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?19Perception[4]

Kaiser

et

al.,

One

Model

toLearnThem

All,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?20AttentionAttentionAllows

selective

processing

of

input

to

avoid

clutter

and

simplify

computation.Allows

processing

of

complex

input

in

nonlinear

ways.Attention

can

make

our

models

more

interpretable.[5]

Cho

et

al.,

Describing

Multimedia

Content

using

Attention

Based

Encoder

Decoder

Networks,

2015Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?22Attention[6]

Xu

et

al.,

Show,

Attend

and

Tell:

Neural

Image

Caption

Generationwith

Visual

Attention,

2015Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?23Image

Caption

GenerationMemoryMemoryNeural

Turing

MachineAnLSTM

ControllerRead

/

Write

Heads

usingAttention

(NN)An

External

MemoryCan

learning

simple

algorithmssuch

as

copying

&

sorting[7]

Graves

et

al.,

Neural

Turing

Machine,

201525Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?ReasoningReasoningProcess

very

unstructured

complex(multi-modal)

inputDeliberate

and

infer

relationsStore

KnowledgeOutput

unstructured

output[8]

Johnson

et

al.,

Learningand

Executing

Programs

for

Visual

Reasoning

,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?27ReasoningVisual

Question

Answering

(VQA)[9]

Yang

et

al.,StackedAttention

for

ImageQuestion

Answering,2015Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?28ReasoningVisual

Question

Answering

(VQA)[9]

Yang

et

al.,StackedAttention

for

ImageQuestion

Answering,2015Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?29ReasoningVisual

Question

Answering

(VQA)[8]

Johnson

et

al.,

Learningand

Executing

Programs

for

Visual

Reasoning

,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?30ReasoningVisual

Question

Answering

(VQA)[8]

Johnson

et

al.,

Learningand

Executing

Programs

for

Visual

Reasoning

,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?31Meta-LearningMeta-LearningLearning

to

Learn[*]

Ravi

et

al.,

Optimization

as

a

Model

for

Few-Shot

Learning,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?33Meta-LearningLearning

to

Learn[*]

Ravi

et

al.,

Optimization

as

a

Model

for

Few-Shot

Learning,

2017Haytham

Fayek,

Will

Deep

Learning

Lead

to

AI?34Meta-Lear