Imperfect Experience; or, Effects of withholding training data on multi-task question answering in convolutional neural networks

Abstract

This thesis explores the effects of training convolutional neural networks to perform conditional multi-task problems with training data that systematically excludes information. Using the MNIST database of handwritten digits, I prepare two collections of three versions of the database, adding color and an embedded question. Two question-embedding method are used. The base version of each data set has 100 combinations of 10 digits and 10 colors, with a roughly equal distribution of questions. From these base preparations I extract color/shape combinations from the inputs using two strategies, forcing each network to infer progressively more answers during testing. I demonstrate six Convolutional Neural Networks (CNNs) varied by the architecture of their output layers and output activation function, tested with two different question embedding processes. Without otherwise implementing advanced tuning techniques, the networks achieve between 96.78% (±0.41, n=90) and 99.64 % (±0.41, n=90) accuracy on the more difficult task when training on all category combinations. At 50% combination extraction, one network variation demonstrates 98.33% (±0.35, n=90) and 99.87% (±0.09, n=90) accuracy on shape and color classification tasks. The results indicate best performance from Sigmoid output activation and task-shared final fully connected output layers. The thesis contributes to strategy for network design when data is scarce.