Objective. Machine learning (ML) models have opened up enormous opportunities in the field of brain–computer Interfaces (BCIs). Despite their great success, they usually face severe limitations when they are employed in real-life applications outside a controlled laboratory setting. Approach. Mixing causal reasoning, identifying causal relationships between variables of interest, with brainwave modeling can change one's viewpoint on some of these major challenges which can be found in various stages in the ML pipeline, ranging from data collection and data pre-processing to training methods and techniques. Main results. In this work, we employ causal reasoning and present a framework aiming to breakdown and analyze important challenges of brainwave modeling for BCIs. Significance. Furthermore, we present how general ML practices as well as brainwave-specific techniques can be utilized and solve some of these identified challenges. And finally, we discuss appropriate evaluation schemes in order to measure these techniques' performance and efficiently compare them with other methods that will be developed in the future.
ISSN: 1741-2552
Journal of Neural Engineering was created to help scientists, clinicians and engineers to understand, replace, repair and enhance the nervous system.
Open all abstracts, in this tab
Konstantinos Barmpas et al 2024 J. Neural Eng. 21 036001
Sundari Elango et al 2024 J. Neural Eng. 21 036003
Objective. The functional asymmetry between the two brain hemispheres in language and spatial processing is well documented. However, a description of difference in control between the two hemispheres in motor function is not well established. Our primary objective in this study was to examine the distribution of control in the motor hierarchy and its variation across hemispheres. Approach. We developed a computation model termed the bilateral control network and implemented the same in a neural network framework to be used to replicate certain experimental results. The network consists of a simple arm model capable of making movements in 2D space and a motor hierarchy with separate elements coding target location, estimated position of arm, direction, and distance to be moved by the arm, and the motor command sent to the arm. The main assumption made here is the division of direction and distance coding between the two hemispheres with distance coded in the non-dominant and direction coded in the dominant hemisphere. Main results. With this assumption, the network was able to show main results observed in visuomotor adaptation studies. Importantly it showed decrease in error exhibited by the untrained arm while the other arm underwent training compared to the corresponding naïve arm's performance—transfer of motor learning from trained to the untrained arm. It also showed how this varied depending on the performance variable used—with distance as the measure, the non-dominant arm showed transfer and with direction, dominant arm showed transfer. Significance. Our results indicate the possibility of shared control between the two hemispheres. If indeed found true, this result could have major significance in motor rehabilitation as treatment strategies will need to be designed in order to account for this and can no longer be confined to the arm contralateral to the affected hemisphere.
S Martínez et al 2024 J. Neural Eng. 21 036002
Objective. This paper aims to bridge the gap between neurophysiology and automatic control methodologies by redefining the Wilson–Cowan (WC) model as a control-oriented linear parameter-varying (LPV) system. A novel approach is presented that allows for the application of a control strategy to modulate and track neural activity. Approach. The WC model is redefined as a control-oriented LPV system in this study. The LPV modelling framework is leveraged to design an LPV controller, which is used to regulate and manipulate neural dynamics. Main results. Promising outcomes, in understanding and controlling neural processes through the synergistic combination of control-oriented modelling and estimation, are obtained in this study. An LPV controller demonstrates to be effective in regulating neural activity. Significance. The presented methodology effectively induces neural patterns, taking into account optogenetic actuation. The combination of control strategies with neurophysiology provides valuable insights into neural dynamics. The proposed approach opens up new possibilities for using control techniques to study and influence brain functions, which can have key implications in neuroscience and medicine. By means of a model-based controller which accounts for non-linearities, noise and uncertainty, neural signals can be induced on brain structures.
Nitin Sadras et al 2024 J. Neural Eng. 21 026049
The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.
Milan Rybář et al 2024 J. Neural Eng. 21 029501
Open all abstracts, in this tab
Rongqi Hong et al 2024 J. Neural Eng. 21 021002
Objective: Epilepsy is a complex disease spanning across multiple scales, from ion channels in neurons to neuronal circuits across the entire brain. Over the past decades, computational models have been used to describe the pathophysiological activity of the epileptic brain from different aspects. Traditionally, each computational model can aid in optimizing therapeutic interventions, therefore, providing a particular view to design strategies for treating epilepsy. As a result, most studies are concerned with generating specific models of the epileptic brain that can help us understand the certain machinery of the pathological state. Those specific models vary in complexity and biological accuracy, with system-level models often lacking biological details. Approach: Here, we review various types of computational model of epilepsy and discuss their potential for different therapeutic approaches and scenarios, including drug discovery, surgical strategies, brain stimulation, and seizure prediction. We propose that we need to consider an integrated approach with a unified modelling framework across multiple scales to understand the epileptic brain. Our proposal is based on the recent increase in computational power, which has opened up the possibility of unifying those specific epileptic models into simulations with an unprecedented level of detail. Main results: A multi-scale epilepsy model can bridge the gap between biologically detailed models, used to address molecular and cellular questions, and brain-wide models based on abstract models which can account for complex neurological and behavioural observations. Significance: With these efforts, we move toward the next generation of epileptic brain models capable of connecting cellular features, such as ion channel properties, with standard clinical measures such as seizure severity.
Joana Soldado-Magraner et al 2024 J. Neural Eng. 21 022001
Objective. Brain-computer interfaces (BCIs) are neuroprosthetic devices that allow for direct interaction between brains and machines. These types of neurotechnologies have recently experienced a strong drive in research and development, given, in part, that they promise to restore motor and communication abilities in individuals experiencing severe paralysis. While a rich literature analyzes the ethical, legal, and sociocultural implications (ELSCI) of these novel neurotechnologies, engineers, clinicians and BCI practitioners often do not have enough exposure to these topics. Approach. Here, we present the IEEE Neuroethics Framework, an international, multiyear, iterative initiative aimed at developing a robust, accessible set of considerations for diverse stakeholders. Main results. Using the framework, we provide practical examples of ELSCI considerations for BCI neurotechnologies. We focus on invasive technologies, and in particular, devices that are implanted intra-cortically for medical research applications. Significance. We demonstrate the utility of our framework in exposing a wide range of implications across different intra-cortical BCI technology modalities and conclude with recommendations on how to utilize this knowledge in the development and application of ethical guidelines for BCI neurotechnologies.
C J H Rikhof et al 2024 J. Neural Eng. 21 021001
Objective. The incidence of stroke rising, leading to an increased demand for rehabilitation services. Literature has consistently shown that early and intensive rehabilitation is beneficial for stroke patients. Robot-assisted devices have been extensively studied in this context, as they have the potential to increase the frequency of therapy sessions and thereby the intensity. Robot-assisted systems can be combined with electrical stimulation (ES) to further enhance muscle activation and patient compliance. The objective of this study was to review the effectiveness of ES combined with all types of robot-assisted technology for lower extremity rehabilitation in stroke patients. Approach. A thorough search of peer-reviewed articles was conducted. The quality of the included studies was assessed using a modified version of the Downs and Black checklist. Relevant information regarding the interventions, devices, study populations, and more was extracted from the selected articles. Main results. A total of 26 articles were included in the review, with 23 of them scoring at least fair on the methodological quality. The analyzed devices could be categorized into two main groups: cycling combined with ES and robots combined with ES. Overall, all the studies demonstrated improvements in body function and structure, as well as activity level, as per the International Classification of Functioning, Disability, and Health model. Half of the studies in this review showed superiority of training with the combination of robot and ES over robot training alone or over conventional treatment. Significance. The combination of robot-assisted technology with ES is gaining increasing interest in stroke rehabilitation. However, the studies identified in this review present challenges in terms of comparability due to variations in outcome measures and intervention protocols. Future research should focus on actively involving and engaging patients in executing movements and strive for standardization in outcome values and intervention protocols.
Zachary T Sanger et al 2024 J. Neural Eng. 21 012001
Deep brain stimulation (DBS) using Medtronic's Percept™ PC implantable pulse generator is FDA-approved for treating Parkinson's disease (PD), essential tremor, dystonia, obsessive compulsive disorder, and epilepsy. Percept™ PC enables simultaneous recording of neural signals from the same lead used for stimulation. Many Percept™ PC sensing features were built with PD patients in mind, but these features are potentially useful to refine therapies for many different disease processes. When starting our ongoing epilepsy research study, we found it difficult to find detailed descriptions about these features and have compiled information from multiple sources to understand it as a tool, particularly for use in patients other than those with PD. Here we provide a tutorial for scientists and physicians interested in using Percept™ PC's features and provide examples of how neural time series data is often represented and saved. We address characteristics of the recorded signals and discuss Percept™ PC hardware and software capabilities in data pre-processing, signal filtering, and DBS lead performance. We explain the power spectrum of the data and how it is shaped by the filter response of Percept™ PC as well as the aliasing of the stimulation due to digitally sampling the data. We present Percept™ PC's ability to extract biomarkers that may be used to optimize stimulation therapy. We show how differences in lead type affects noise characteristics of the implanted leads from seven epilepsy patients enrolled in our clinical trial. Percept™ PC has sufficient signal-to-noise ratio, sampling capabilities, and stimulus artifact rejection for neural activity recording. Limitations in sampling rate, potential artifacts during stimulation, and shortening of battery life when monitoring neural activity at home were observed. Despite these limitations, Percept™ PC demonstrates potential as a useful tool for recording neural activity in order to optimize stimulation therapies to personalize treatment.
Khaled M Taghlabi et al 2024 J. Neural Eng. 21 011001
Peripheral nerve interfaces (PNIs) are electrical systems designed to integrate with peripheral nerves in patients, such as following central nervous system (CNS) injuries to augment or replace CNS control and restore function. We review the literature for clinical trials and studies containing clinical outcome measures to explore the utility of human applications of PNIs. We discuss the various types of electrodes currently used for PNI systems and their functionalities and limitations. We discuss important design characteristics of PNI systems, including biocompatibility, resolution and specificity, efficacy, and longevity, to highlight their importance in the current and future development of PNIs. The clinical outcomes of PNI systems are also discussed. Finally, we review relevant PNI clinical trials that were conducted, up to the present date, to restore the sensory and motor function of upper or lower limbs in amputees, spinal cord injury patients, or intact individuals and describe their significant findings. This review highlights the current progress in the field of PNIs and serves as a foundation for future development and application of PNI systems.
Open all abstracts, in this tab
Vogel et al
Deep brain stimulation (DBS) is a therapy for Parkinson's disease (PD) and essential tremor (ET). The mechanism of action of DBS is still incompletely understood. Retrospective group analysis of intra-operative data recorded from ET patients implanted in the ventral intermediate nucleus of the thalamus (Vim) is rare. Intra-operative stimulation tests generate rich data and their use in group analysis has not yet been explored. 

Objective: To implement, evaluate, and apply a group analysis workflow to generate probabilistic stimulation maps (PSM) using intra-operative stimulation data from ET patients implanted in Vim. 

A group-specific anatomical template was constructed based on the Magnetic Resonance Imaging (MRI) scans of 6 ET patients and 13 PD patients. Intra-operative test data (total: n=1821) from the 6 ET patients was analyzed: patient-specific electric field simulations together with tremor assessments obtained by a wrist-based acceleration sensor were transferred to this template. Occurrence and weighted mean maps were generated. Voxels associated with symptomatic response were identified through a linear mixed model approach to form a PSM. Improvements predicted by the PSM were compared to those clinically assessed. Finally, the PSM clusters were compared to those obtained in a multicenter study using data from chronic stimulation effects in ET. 

Regions responsible for improvement identified on the PSM were in the posterior sub-thalamic area (PSA) and at the border between the Vim and ventro-oral nucleus of the thalamus (VO). The comparison with literature revealed a center-to-center distance of less than 5mm and an overlap score (Dice) of 0.4 between the significant clusters. 

Our workflow and intra-operative test data from 6 ET-Vim patients identified effective stimulation areas in PSA and around Vim and VO, affirming existing medical literature. This study supports the potential of probabilistic analysis of intra-operative stimulation test data to reveal DBS's action mechanisms and to assist surgical planning.
Liang et al
Objective Electroencephalogram (EEG) analysis has always been an important tool in neural engineering, and the recognition and classification of human emotions are one of the important tasks in neural engineering. EEG data, obtained from electrodes placed on the scalp, represent a valuable resource of information for brain activity analysis and emotion recognition. Feature extraction methods have shown promising results, but recent trends have shifted toward end-to-end methods based on deep learning. However, these approaches often overlook channel representations, and their complex structures pose certain challenges to model fitting. Approach To address these challenges, this paper proposes a hybrid approach named FetchEEG that combines feature extraction and temporal-channel joint attention. Leveraging the advantages of both traditional feature extraction and deep learning, the FetchEEG adopts a multi-head self-attention mechanism to extract representations between different time moments and channels simultaneously. The joint representations are then concatenated and classified using fully-connected layers for emotion recognition. The performance of the FetchEEG is verified by comparison experiments on a self-developed dataset and two public datasets. Results In both subject-dependent and subject-independent experiments, the FetchEEG demonstrates better performance and stronger generalization ability than the state-of-the-art methods on all datasets. Moreover, the performance of the FetchEEG is analyzed for different sliding window sizes and overlap rates in the feature extraction module. The sensitivity of emotion recognition is investigated for three- and five-frequency-band scenarios. Finally, we perform visualization of the channel-attention to project the model's reliance on different channels onto scalp topographic maps. Significance FetchEEG is a novel hybrid method based on EEG for emotion classification, which significantly improves the performance of recognition and is effective and feasible.
Wu et al
The rapid serial visual presentation (RSVP) paradigm, which is based on the electroencephalogram (EEG) technology, is an effective approach for object detection. It aims to detect the event-related potentials (ERP) components evoked by target images for rapid identification. However, the object detection performance within this paradigm is affected by the visual disparity between adjacent images in a sequence. Currently, there is no objective metric to quantify this visual difference. Consequently, a reliable image sorting method is required to ensure the generation of a smooth sequence for effective presentation. In this paper, we propose a novel semantic image sorting method for sorting RSVP sequences, which aims at generating sequences that are perceptually smoother in terms of the human visual experience. We conducted a comparative analysis between our method and two existing methods for generating RSVP sequences using both qualitative and quantitative assessments. A qualitative evaluation revealed that the sequences generated by our method were smoother in subjective vision and were more effective in evoking stronger ERP components than those generated by the other two methods. Quantitatively, our method generated semantically smoother sequences than the other two methods. Furthermore, we employed four advanced approaches to classify single-trial EEG signals evoked by each of the three methods. The classification results of the EEG signals evoked by our method were superior to those of the other two methods. In summary, the results indicate that the proposed method can significantly enhance the object detection performance in RSVP-based sequences.
Ludwig et al
Objective: The patterns of brain activity associated with different brain processes can be used to identify different brain states and make behavioral predictions. However, the relevant features are not readily apparent and accessible. Our aim is to design a system for learning informative latent representations from multichannel recordings of ongoing EEG activity. Approach: We propose a novel differentiable decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. Specifically, we introduce filters parameterized by generalized Gaussian functions that offer a smooth derivative for stable end-to-end model training and allow for learning interpretable features. For the feature module, we use signal magnitude and functional connectivity estimates. Main Results: We demonstrate the utility of our model on a new EEG dataset of unprecedented size (i.e., 761 subjects), where we identify consistent trends of music perception and related individual differences. Furthermore, we train and apply our model in two additional datasets, specifically for emotion recognition on SEED and workload classification on STEW. The discovered features align well with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening. This agrees with the specialisation of the temporal lobes regarding music perception proposed in the literature. Significance: The proposed method offers strong interpretability of learned features while reaching similar levels of accuracy achieved by black box deep learning models. This improved trustworthiness may promote the use of deep learning models in real world applications. The model code is available at https://github.com/SMLudwig/EEGminer/.
Song et al
Objective. Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning.

Approach. We proposed a linear signal alignment that uses the P300's latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm).

Results. Although the standard approach without signal alignment had an average precision score of 25.5%, the approach demonstrated a 35.8% average precision score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets. 

Significance. We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.
Open all abstracts, in this tab
Konstantinos Barmpas et al 2024 J. Neural Eng. 21 036001
Objective. Machine learning (ML) models have opened up enormous opportunities in the field of brain–computer Interfaces (BCIs). Despite their great success, they usually face severe limitations when they are employed in real-life applications outside a controlled laboratory setting. Approach. Mixing causal reasoning, identifying causal relationships between variables of interest, with brainwave modeling can change one's viewpoint on some of these major challenges which can be found in various stages in the ML pipeline, ranging from data collection and data pre-processing to training methods and techniques. Main results. In this work, we employ causal reasoning and present a framework aiming to breakdown and analyze important challenges of brainwave modeling for BCIs. Significance. Furthermore, we present how general ML practices as well as brainwave-specific techniques can be utilized and solve some of these identified challenges. And finally, we discuss appropriate evaluation schemes in order to measure these techniques' performance and efficiently compare them with other methods that will be developed in the future.
Dorian Vogel et al 2024 J. Neural Eng.
Deep brain stimulation (DBS) is a therapy for Parkinson's disease (PD) and essential tremor (ET). The mechanism of action of DBS is still incompletely understood. Retrospective group analysis of intra-operative data recorded from ET patients implanted in the ventral intermediate nucleus of the thalamus (Vim) is rare. Intra-operative stimulation tests generate rich data and their use in group analysis has not yet been explored. 

Objective: To implement, evaluate, and apply a group analysis workflow to generate probabilistic stimulation maps (PSM) using intra-operative stimulation data from ET patients implanted in Vim. 

A group-specific anatomical template was constructed based on the Magnetic Resonance Imaging (MRI) scans of 6 ET patients and 13 PD patients. Intra-operative test data (total: n=1821) from the 6 ET patients was analyzed: patient-specific electric field simulations together with tremor assessments obtained by a wrist-based acceleration sensor were transferred to this template. Occurrence and weighted mean maps were generated. Voxels associated with symptomatic response were identified through a linear mixed model approach to form a PSM. Improvements predicted by the PSM were compared to those clinically assessed. Finally, the PSM clusters were compared to those obtained in a multicenter study using data from chronic stimulation effects in ET. 

Regions responsible for improvement identified on the PSM were in the posterior sub-thalamic area (PSA) and at the border between the Vim and ventro-oral nucleus of the thalamus (VO). The comparison with literature revealed a center-to-center distance of less than 5mm and an overlap score (Dice) of 0.4 between the significant clusters. 

Our workflow and intra-operative test data from 6 ET-Vim patients identified effective stimulation areas in PSA and around Vim and VO, affirming existing medical literature. This study supports the potential of probabilistic analysis of intra-operative stimulation test data to reveal DBS's action mechanisms and to assist surgical planning.
Nitin Sadras et al 2024 J. Neural Eng. 21 026049
The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.
Siegfried Ludwig et al 2024 J. Neural Eng.
Objective: The patterns of brain activity associated with different brain processes can be used to identify different brain states and make behavioral predictions. However, the relevant features are not readily apparent and accessible. Our aim is to design a system for learning informative latent representations from multichannel recordings of ongoing EEG activity. Approach: We propose a novel differentiable decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. Specifically, we introduce filters parameterized by generalized Gaussian functions that offer a smooth derivative for stable end-to-end model training and allow for learning interpretable features. For the feature module, we use signal magnitude and functional connectivity estimates. Main Results: We demonstrate the utility of our model on a new EEG dataset of unprecedented size (i.e., 761 subjects), where we identify consistent trends of music perception and related individual differences. Furthermore, we train and apply our model in two additional datasets, specifically for emotion recognition on SEED and workload classification on STEW. The discovered features align well with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening. This agrees with the specialisation of the temporal lobes regarding music perception proposed in the literature. Significance: The proposed method offers strong interpretability of learned features while reaching similar levels of accuracy achieved by black box deep learning models. This improved trustworthiness may promote the use of deep learning models in real world applications. The model code is available at https://github.com/SMLudwig/EEGminer/.
Diego Lopez-Bernal et al 2024 J. Neural Eng. 21 026048
Objective. In recent years, electroencephalogram (EEG)-based brain–computer interfaces (BCIs) applied to inner speech classification have gathered attention for their potential to provide a communication channel for individuals with speech disabilities. However, existing methodologies for this task fall short in achieving acceptable accuracy for real-life implementation. This paper concentrated on exploring the possibility of using inter-trial coherence (ITC) as a feature extraction technique to enhance inner speech classification accuracy in EEG-based BCIs. Approach. To address the objective, this work presents a novel methodology that employs ITC for feature extraction within a complex Morlet time-frequency representation. The study involves a dataset comprising EEG recordings of four different words for ten subjects, with three recording sessions per subject. The extracted features are then classified using k-nearest-neighbors (kNNs) and support vector machine (SVM). Main results. The average classification accuracy achieved using the proposed methodology is 56.08% for kNN and 59.55% for SVM. These results demonstrate comparable or superior performance in comparison to previous works. The exploration of inter-trial phase coherence as a feature extraction technique proves promising for enhancing accuracy in inner speech classification within EEG-based BCIs. Significance. This study contributes to the advancement of EEG-based BCIs for inner speech classification by introducing a feature extraction methodology using ITC. The obtained results, on par or superior to previous works, highlight the potential significance of this approach in improving the accuracy of BCI systems. The exploration of this technique lays the groundwork for further research toward inner speech decoding.
Minseok Song et al 2024 J. Neural Eng.
Objective. Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning.

Approach. We proposed a linear signal alignment that uses the P300's latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm).

Results. Although the standard approach without signal alignment had an average precision score of 25.5%, the approach demonstrated a 35.8% average precision score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets. 

Significance. We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.
Francisco David Guerreiro Fernandes et al 2024 J. Neural Eng.
Purpose:
Brain-Computer Interfaces (BCIs) have the potential to reinstate lost communication faculties. Results from speech decoding studies indicate that a usable speech BCI based on activity in the sensorimotor cortex (SMC) can be achieved using subdurally implanted electrodes. However, the optimal characteristics for a successful speech implant are largely unknown. We address this topic in a high field blood oxygenation level dependent (BOLD) functional Magnetic Resonance Imaging (fMRI) study, by assessing the decodability of spoken words as a function of hemisphere, gyrus, sulcal depth, and position along the ventral/dorsal-axis.
Methods:
Twelve subjects conducted a 7T fMRI experiment in which they pronounced 6 different pseudo-words over 6 runs. We divided the SMC by hemisphere, gyrus, sulcal depth, and position along the ventral/dorsal axis. Classification was performed on in these SMC areas using multiclass Support Vector Machine (SVM).
Results:
Significant classification was possible from the SMC, but no preference for the left or right hemisphere, nor for the precentral or postcentral gyrus for optimal word classification was detected. Classification while using information from the cortical surface was slightly better than when using information from deep in the central sulcus, and was highest within the ventral 50% of SMC. Confusion matrices where highly similar across the entire SMC. An SVM-searchlight analysis revealed significant classification in the superior temporal gyrus in addition to the SMC.
Conclusion:
The current results support a unilateral implant using surface electrodes, covering the ventral 50% of the SMC. The added value of depth electrodes is unclear. We did not observe evidence for variations in the qualitative nature of information across SMC. The current results need to be confirmed in paralyzed patients performing attempted speech.
Ariel Tankus et al 2024 J. Neural Eng.
Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding.
Approach. We intraoperatively recorded single neuron activity in the left Vim of 8 neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space.
Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other.
Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.
Yuya Ikegawa et al 2024 J. Neural Eng.
Objective:
Invasive brain-computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis (ALS). An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions.
Approach:
In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortexes. A CLIP image vector was inferred from the high-γ power of the iEEG signals recorded while viewing the images.
Main results:
Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy.
Significance:
The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication.
Niccolò Calcini et al 2024 J. Neural Eng.
Objective: Traditional quantification of fluorescence signals, such as
∆F/F, relies on ratiometric measures that necessitate a baseline for compar-
ison, limiting their applicability in dynamic analyses. Our goal here is to
develop a baseline-independent method for analyzing fluorescence data that
fully exploits temporal dynamics to introduce a novel approach for dynami-
cal super-resolution analysis, including in subcellular resolution.
Approach: We introduce ARES (Autoregressive RESiduals), a novel method
that leverages the temporal aspect of fluorescence signals. By focusing on
the quantification of residuals following linear autoregression, ARES obviates
the need for a predefined baseline, enabling a more nuanced analysis of signal
dynamics.
Main Result: We delineate the foundational attributes of ARES, illustrat-
ing its capability to enhance both spatial and temporal resolution of calcium
fluorescence activity beyond the conventional ratiometric measure (∆F/F).
Additionally, we demonstrate ARES's utility in elucidating intracellular cal-
cium dynamics through the detailed observation of calcium wave propagation
within a dendrite.
Significance: ARES stands out as a robust and precise tool for the quan-
tification of fluorescence signals, adept at analyzing both spontaneous and
evoked calcium dynamics. Its ability to facilitate the subcellular localiza-
tion of calcium signals and the spatiotemporal tracking of calcium dynam-
ics—where traditional ratiometric measures falter—underscores its potential
to revolutionize baseline-independent analyses in the field