Deep learning: an introduction for applied mathematicians

Catherine F. Higham, Desmond J. Higham

Research output: Contribution to journalArticle

Abstract

Multilayered artificial neural networks are becoming a pervasive tool in a host of application fields. At the heart of this deep learning revolution are familiar concepts from applied and computational mathematics; notably, in calculus, approximation theory, optimization and linear algebra. This article provides a very brief introduction to the basic ideas that underlie deep learning from an applied mathematics perspective. Our target audience includes postgraduate and final year undergraduate students in mathematics who are keen to learn about the area. The article may also be useful for instructors in mathematics who wish to enliven their classes with references to the application of deep learning techniques. We focus on three fundamental questions: what is a deep neural network? how is a network trained? what is the stochastic gradient method? We illustrate the ideas with a short MATLAB code that sets up and trains a network. We also show the use of state-of-the art software on a large scale image classification problem. We finish with references to the current literature.
LanguageEnglish
Number of pages39
JournalSIAM Review
Publication statusAccepted/In press - 7 Feb 2019

Fingerprint

Stochastic Gradient
Approximation Theory
Image Classification
Stochastic Methods
Gradient Method
Applied mathematics
Approximation theory
Linear algebra
Classification Problems
MATLAB
Gradient methods
Artificial Neural Network
Image classification
Calculus
Neural Networks
Target
Software
Optimization
Students
Neural networks

Keywords

  • back progagation
  • chain rule
  • convolution
  • image classification
  • neural network
  • overfitting
  • sigmoid
  • stochastic gradient method

Cite this

Higham, C. F., & Higham, D. J. (Accepted/In press). Deep learning: an introduction for applied mathematicians. SIAM Review.
Higham, Catherine F. ; Higham, Desmond J. / Deep learning : an introduction for applied mathematicians. In: SIAM Review. 2019.
@article{36e52a2cb0cd48179dc10eea89f2f592,
title = "Deep learning: an introduction for applied mathematicians",
abstract = "Multilayered artificial neural networks are becoming a pervasive tool in a host of application fields. At the heart of this deep learning revolution are familiar concepts from applied and computational mathematics; notably, in calculus, approximation theory, optimization and linear algebra. This article provides a very brief introduction to the basic ideas that underlie deep learning from an applied mathematics perspective. Our target audience includes postgraduate and final year undergraduate students in mathematics who are keen to learn about the area. The article may also be useful for instructors in mathematics who wish to enliven their classes with references to the application of deep learning techniques. We focus on three fundamental questions: what is a deep neural network? how is a network trained? what is the stochastic gradient method? We illustrate the ideas with a short MATLAB code that sets up and trains a network. We also show the use of state-of-the art software on a large scale image classification problem. We finish with references to the current literature.",
keywords = "back progagation, chain rule, convolution, image classification, neural network, overfitting, sigmoid, stochastic gradient method",
author = "Higham, {Catherine F.} and Higham, {Desmond J.}",
year = "2019",
month = "2",
day = "7",
language = "English",
journal = "SIAM Review",
issn = "0036-1445",

}

Deep learning : an introduction for applied mathematicians. / Higham, Catherine F.; Higham, Desmond J.

In: SIAM Review, 07.02.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Deep learning

T2 - SIAM Review

AU - Higham, Catherine F.

AU - Higham, Desmond J.

PY - 2019/2/7

Y1 - 2019/2/7

N2 - Multilayered artificial neural networks are becoming a pervasive tool in a host of application fields. At the heart of this deep learning revolution are familiar concepts from applied and computational mathematics; notably, in calculus, approximation theory, optimization and linear algebra. This article provides a very brief introduction to the basic ideas that underlie deep learning from an applied mathematics perspective. Our target audience includes postgraduate and final year undergraduate students in mathematics who are keen to learn about the area. The article may also be useful for instructors in mathematics who wish to enliven their classes with references to the application of deep learning techniques. We focus on three fundamental questions: what is a deep neural network? how is a network trained? what is the stochastic gradient method? We illustrate the ideas with a short MATLAB code that sets up and trains a network. We also show the use of state-of-the art software on a large scale image classification problem. We finish with references to the current literature.

AB - Multilayered artificial neural networks are becoming a pervasive tool in a host of application fields. At the heart of this deep learning revolution are familiar concepts from applied and computational mathematics; notably, in calculus, approximation theory, optimization and linear algebra. This article provides a very brief introduction to the basic ideas that underlie deep learning from an applied mathematics perspective. Our target audience includes postgraduate and final year undergraduate students in mathematics who are keen to learn about the area. The article may also be useful for instructors in mathematics who wish to enliven their classes with references to the application of deep learning techniques. We focus on three fundamental questions: what is a deep neural network? how is a network trained? what is the stochastic gradient method? We illustrate the ideas with a short MATLAB code that sets up and trains a network. We also show the use of state-of-the art software on a large scale image classification problem. We finish with references to the current literature.

KW - back progagation

KW - chain rule

KW - convolution

KW - image classification

KW - neural network

KW - overfitting

KW - sigmoid

KW - stochastic gradient method

UR - https://epubs.siam.org/journal/siread

UR - https://arxiv.org/abs/1801.05894

M3 - Article

JO - SIAM Review

JF - SIAM Review

SN - 0036-1445

ER -