Wave-Nets : novel learning techniques, and the induction of physically interpretable models

Abstract

A wavelet network, or Wave-Net is a connectionist network that combines the mathematical rigor and multiresolution character of wavelets with the adaptive learning of artificial neural networks. In this paper, we present some novel techniques for training and adaptation of Wave-Nets, and describe the induction of models that may be physically interpretable, and may provide useful insight into the system being modeled. Learning from empirical data is formulated as a constrained optimization problem. This formulation illustrates the complexity of the learning problem, and highlights the decision variables and the simplifying assumptions necessary for a practical learning methodology. Techniques for Wave-Net training and adaptation are developed for minimizing the L2 or L(infinity) norms. Minimizing the L(infinity) norm is particularly relevant for solving control problems. The connection between Wave-Net parameters, and the error of approximation is derived using the principles of frame theory. The performance of Wave-Nets for different training methodologies, and basis functions is compared via case studies. Wave-Nets with Haar wavelets as activation functions are well-suited for problems where the output consists of a finite set of discrete values, as in classification problems. The mapping learned by Haar Wave-Nets may be represented as simple if-then rules, which provide an explicit and physically meaningful relationship between inputs and outputs. The relationship of learning by Haar Wave-Nets with other rule induction techniques, such as decision trees is explored.