Generalization capability of neural networks

Citation

Ji, Chuanyi (1992) Generalization capability of neural networks. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/HR3F-0410. https://resolver.caltech.edu/CaltechETD:etd-07202007-143215

Abstract

The generalization capability of feedforward multilayer neural networks is investigated from two aspects: the theoretical aspect and the algorithmic aspect.

In the theoretical part, a general relation is derived between the so-called VC-dimension and the statistical lower epsilon-capacity, and then applied to two cases. First, as a general constructive approach, it is used to evaluate a lower bound of the VC-dimension of two layer networks with binary weights and integer thresholds. Second, how the sample complexity may vary with respect to distributions is investigated through analyzing a particular network which separates two binary clusters. Bounds for the capacity of two layer networks with binary weights and integer thresholds are also obtained.

In the algorithmic part, a network reduction algorithm is developed to study generalization in learning analog mappings. It is applied to control a two-link manipulator to draw characters. The network addition-deletion algorithm is described to find an appropriate network structure during learning. It is used to study the effect of sizes of networks on generalization, and applied to various classification problems including hand written digits recognition.

Thesis Files