Invariant object recognition is one of the most fundamental cognitive tasks performed by the brain. In the neural state space, different objects with stimulus variabilities are represented as different manifolds. In this geometrical perspective, object recognition becomes the problem of linearly separating different object manifolds. In feedforward visual hierarchy, it has been suggested that the object manifold representations are reformatted across the layers, to become more linearly separable. Thus, a complete theory of perception requires characterizing the ability of linear readout networks to classify object manifolds from variable neural responses. A theoretical understanding of the perceptron of isolated points was pioneered by Elizabeth Gardner who formulated it as a statistical mechanics problem and analyzed it using replica theory. In this thesis, we generalize the statistical mechanical analysis and establish a theory of linear classification of manifolds synthesizing statistical and geometric properties of high dimensional signals. First, we study the theory of linear classification of simple spherical manifolds, such as line segments, L2 balls, or L1 balls. We provide analytical formula for classification capacity of balls, as a function of dimension, radius, and margin. We also find that the notion of support vectors needs to be generalized, and identify different support configurations of the manifolds, which has implications in generalization error. Next, we present a Maximum Margin Manifold Machine (M4), an efficient iterative algorithm that can find a maximum margin linear binary classifier for manifolds with an uncountable set of training samples per each manifold. We provide a convergence proof with a polynomial bound on the convergence time. We further generalize M4 for non-separable manifolds with slack variables. We report that the number of training examples required to achieve the same generalization error is much smaller for M4, compared with traditional support vector machines. Next, we generalize our theory further to linear classification of random general manifolds. We start with classification capacity of random ellipsoids, and generalize to classification capacity of general smooth and non-smooth manifolds. We identify that the capacity of a manifold is determined that effective radius, R_M, and effective dimension, D_M. Finally, we show extensions to directions relevant for applications to real data. We have extended our general manifold classification theory to incorporate correlated manifolds, mixtures of manifold geometries, sparse labels and nonlinear classifications. Then, we analyze how object-based manifolds reformat in a conventional deep network (GoogLeNet). We find that the deep network indeed changes the manifolds in the direction that the capacity is increased. This thesis lays the groundwork for a computational theory of neuronal processing of objects, providing quantitative measures for linear separability of object manifolds. We hope that our theory will provide new insights into the computational principles underlying processing of sensory representations in the brain. As manifold representations of the sensory world are ubiquitous in both biological and artificial neural systems, exciting future work lies ahead.
@phdthesis{chung2017thesis,
year = {2017},
author = {Chung, Sue Yeon},
title = {Statistical Mechanics of Neural Processing of Object Manifolds},
month = may,
school = {Harvard University},
address = {Cambridge, MA},
keywords = {Replica Theory, Perceptron, Support Vector Machines, Object Manifolds, Perceptual Invariance, Object Recognition, Statistical Mechanics}
}