TopoART Neural Networks

Examples showing how to use TopoART neural networks to solve common machine learning problems.

Adaptive Resonance Theory, or ART, has been developed as a model to explain brain mechanisms such as rapid categorisation of objects, remembering information over very long time ranges, and balancing between expected and unexpected information. ART neural networks possess some unique properties such as fast learning, resistance to catastrophic forgetting, and an incremental architecture which allows the insertion of new neurons on demand.

TopoART combines these properties with the ability of topology-learning neural networks to associate related neurons in order to represent the topology of the input. This knowledge can be exploited in various ways, e.g., clusters of arbitrary shapes can be learnt, the spatio-temporal context of input can be used to facilitate network training, and episodes can be formed.

The combined properties of ART and topology-learning neural networks render TopoART neural networks particularly well suited to frequent problems arising in cognitive robotics and advanced machine learning, such as online-learning, lifelong learning from data streams, as well as incremental learning and prediction from non-stationary data, noisy data, imbalanced data, and incomplete data.

The provided code demonstrates how these neural networks can be used as a bidirectional associative memory or for topological clustering of noisy data, function approximation (regression), and classification. It is based on the .NET library LibTopoART.Compatibility.dll (source repository) which needs to be installed before the sample can be run. Scripts for installing and uninstalling this library and its dependencies are included.

Details on TopoART neural networks can be found in the following publications:

Clustering

Marko Tscherepanow (2010). TopoART: A topology learning hierarchical ART network. In Proceedings of the International Conference on Artificial Neural Networks (ICANN), LNCS 6354, pp. 157–167. Berlin, Germany: Springer.

Marko Tscherepanow (2012). Incremental On-line Clustering with a Topology-Learning Hierarchical ART Neural Network Using Hyperspherical Categories. In Poster and Industry Proceedings of the Industrial Conference on Data Mining (ICDM), pp. 22–34. Fockendorf, Germany: ibai-publishing.

Clustering and Associative Memory

Marko Tscherepanow, Marco Kortkamp, and Marc Kammer (2011). A Hierarchical ART Network for the Stable Incremental Learning of Topological Structures and Associations from Noisy Data. Neural Networks 24(8): 906-916. Elsevier.

Episodic Clustering

Marko Tscherepanow, Sina Kühnel, and Sören Riechers (2012). Episodic Clustering of Data Streams Using a Topology-Learning Neural Network. In Proceedings of the European Conference on Artificial Intelligence (ECAI), Workshop on Active and Incremental Learning (AIL), pp. 24-29. Montpellier, France.

Classification

Marko Tscherepanow and Sören Riechers (2012). An Incremental On-line Classifier for Imbalanced, Incomplete, and Noisy Data. In Proceedings of the European Conference on Artificial Intelligence (ECAI), Workshop on Active and Incremental Learning (AIL), pp. 18-23. Montpellier, France.

Regression

Marko Tscherepanow (2011). An Extended TopoART Network for the Stable On-Line Learning of Regression Functions. In Proceedings of the International Conference on Neural Information Processing (ICONIP), LNCS 7063, pp. 562–571. Berlin, Germany: Springer.