Enhancing liquid state machine classification through reservoir separability optimization using swarm intelligence and multitask learning

The Liquid State Machine (LSM) is a framework designed to solve supervised learning tasks in the context of spatiotemporal data streams. Within the LSM architecture, a randomly created Spiking Recurrent Neural Network (SRNN), which remains untrained serves as a kernel (referred to as the »liquid»), mapping inputs to a general representation that is unrelated to the specific task. A simple learning algorithm at the readout layer then utilizes this representation to perform the task. Two key properties support the computational power of LSM: the Separation Property and Approximation Property, where the former is directly related to the kernel and the latter to the readout layer. The modularity of the LSM allows the liquid and the learning algorithm to be treated independently, enabling performance improvements when solving supervised tasks. Given that the liquid is responsible for transforming inputs into a representation that eases the work for the readout, we focus on enhancing the liquid's separation property. This research aims to indirectly enhance the classification performance of Liquid State Machines (LSMs) by improving their separability property and enabling them to solve several related problems through the concept of swarm intelligence-powered multitask learning. Following a progressive research approach, this work is developed in two key phases: first, by implementing Particle Swarm Optimization (PSO) to make liquid more sensitive to distinguish streams of different classes for a single problem; and second, to accomplish our main objective by extending the previous phase to handle multiple related classification problems through the optimization of liquids in a multitask learning approach using Original Multi-Objective PSO (OMOPSO). In both phases, optimized liquids exhibit more regularized behaviors in the firing rates compared to those obtained with untrained liquids across different problems, while the learning algorithm at the readout layers was kept as simple as a single softmax layer. The study tests its proposal using four artificial pattern recognition problems, conducts extensive experimentation and with various SRNN architectures. The results of the proposal are compared with standard LSM versions and with state-of-the-art approaches.