Skip to main content

On the role of generative artificial intelligence in the development of brain-computer interfaces

Abstract

Since their inception more than 50 years ago, Brain-Computer Interfaces (BCIs) have held promise to compensate for functions lost by people with disabilities through allowing direct communication between the brain and external devices. While research throughout the past decades has demonstrated the feasibility of BCI to act as a successful assistive technology, the widespread use of BCI outside the lab is still beyond reach. This can be attributed to a number of challenges that need to be addressed for BCI to be of practical use including limited data availability, limited temporal and spatial resolutions of brain signals recorded non-invasively and inter-subject variability. In addition, for a very long time, BCI development has been mainly confined to specific simple brain patterns, while developing other BCI applications relying on complex brain patterns has been proven infeasible. Generative Artificial Intelligence (GAI) has recently emerged as an artificial intelligence domain in which trained models can be used to generate new data with properties resembling that of available data. Given the enhancements observed in other domains that possess similar challenges to BCI development, GAI has been recently employed in a multitude of BCI development applications to generate synthetic brain activity; thereby, augmenting the recorded brain activity. Here, a brief review of the recent adoption of GAI techniques to overcome the aforementioned BCI challenges is provided demonstrating the enhancements achieved using GAI techniques in augmenting limited EEG data, enhancing the spatiotemporal resolution of recorded EEG data, enhancing cross-subject performance of BCI systems and implementing end-to-end BCI applications. GAI could represent the means by which BCI would be transformed into a prevalent assistive technology, thereby improving the quality of life of people with disabilities, and helping in adopting BCI as an emerging human-computer interaction technology for general use.

Peer Review reports

Background

A Brain-Computer Interface (BCI) is a system that allows direct communication between the brain and the external environment [1]. BCI is a promising emerging technology that could help persons with disabilities regain, at least partially, their lost abilities. It could be of help when the disability is not caused by a brain damage. BCIs can be classified into invasive and non-invasive. Invasive BCIs have demonstrated success in a multitude of tasks given the high spatiotemporal resolution of acquired brain activity using these systems, which could discern the response of single neurons in the brain [2]. However, this is associated with the high-risk of performing a surgery that aims at implanting electrodes in the brain, which could be required for some medical conditions. On the other hand, non-invasive BCIs that mainly rely on recording Electroencephalogram (EEG) signals by wearing a head cap of electrodes on the scalp represent a safer and a more practical solution [3]. Recent advances in BCIs have demonstrated the efficacy of translating recorded EEG signals into actions that represent users’ intentions. Successful examples of BCIs include EEG-speller systems [4,5,6,7], wheelchair control [8, 9], upper- and lower-limb prosthetics control [10,11,12], robot control [13, 14], and brain-controlled games [15]. In addition, BCI has also been demonstrated to represent a novel human-computer interaction technology that is not limited only to people with disabilities [16,17,18].

Despite the success BCI has demonstrated in tackling many challenges faced by people with disabilities, the adoption of such a solution as a long-term assistive technology has been lacking [19]. This can be attributed to a number of challenges that hinder the widespread use of BCI. First, calibrating BCIs requires recording an extensive amount of EEG data from users. This is needed to train the machine learning algorithms that are typically used in such a context to recognize the user’s intentions. However, recording EEG calibration data for long periods of time is inconvenient for users, especially for patients with medical or mental conditions. Second, EEG systems that provide high channel count with high resolution are typically used in hospitals for clinical purposes and of high cost. These systems are not suitable for everyday use that requires the portability of the EEG recording system. Recently, a number of EEG headsets have been made commercially available at reasonable prices [20]. These headsets overcome the aforementioned limitation by providing portable wireless units with sensors that do not require the application of a conductive gel. However, they mainly offer a limited number of recording channels at low spatiotemporal resolution. Finally, developing universal BCI systems calibrated using data of subjects other than the user has proven to be challenging due to the inter-subject variability that is typically observed in BCI tasks [21]. This requires recording EEG signals from each subject to calibrate the BCI system and operate it in a subject-independent manner.

Since the inception of BCI, multiple efforts in the community have been directed towards tackling the aforementioned challenges [22]. Artificial intelligence (AI) techniques, and machine learning in particular, have represented a core component in any BCI system to help in recognizing the user’s target command. Recently, advances in the Generative AI (GAI) field have demonstrated significant success in a multitude of long-standing tasks that is revolutionizing many aspects of today’s life [23]. GAI has been used in generating different types of realistic data including text, images, audio and video [24]. GAI techniques rely on deep learning models with an extremely large number of tunable parameters that are trained using large amounts of data.

In this article, the recent adoption of GAI techniques in application to non-invasive BCIs is highlighted. First, an overview is provided of how GAI techniques are employed to generate synthetic data to augment the limited amount of EEG data typically recorded in BCI applications. Such augmentation enhances the performance of machine learning algorithms typically used in different BCI applications. Second, a description is provided of how GAI techniques are used to generate EEG signals with high spatiotemporal resolution from EEG signals recorded with lower spatiotemporal resolution. Third, examples are given for using GAI techniques to enhance the cross-subject performance by generating subject-invariant features or signals. Finally, using GAI techniques in developing end-to-end applications is demonstrated, were GAI techniques are used to transform the recorded EEG signals to other forms including audio and images.

There are multiple GAI models that have been utilized in the aforementioned four applications of GAI in BCI development; however, of particular interest to this article are Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), transformers and diffusion models [24, 25]. A VAE consists of two back-to-back networks: An encoder network which encodes the input data into a latent space of lower dimensions by mapping the input data into a latent space distribution, followed by a decoder network that maps the latent space back to the input data space [26]. Once trained in a regularized manner, the decoder can be used independently to generate new data. A GAN, in its most basic form, consists of two networks: a generator network followed by a discriminator network [27]. The generator is trained to synthesize outputs that are as similar as possible to the input data. On the other hand, the discriminator is trained to discriminate between the input real data and the synthetic (fake) data generated by the generator. The inability of the discriminator to differentiate between real and synthesized data indicates the success of the training of the generator. Once the training converges, the generator can be independently used to generate additional data similar in nature to the input data. A transformer, similar to VAEs, uses the concept of an encoder followed by a decoder. However, in addition to using such a general architecture, an attention mechanism is employed in which the decoder would be able to focus on relevant components of the input data sequence [28]. Transformers are by design developed to handle sequential data. Finally, diffusion models attempt in one forward direction (resembling encoders) to add Gaussian noise to the input data along a Markov chain, while in the backward direction (resembling decoders) attempt using a neural network to reverse the process of noise addition to generate a data point from the same space of the input data [29].

This article provides an overview of the use of GAI techniques to enhance BCI systems. Using GAI in developing BCI systems represents one step forward towards the widespread adoption of BCI as a reliable assistive/ human-computer interaction technology.

GAI for data augmentation

GAI techniques have been demonstrated to augment limited EEG data recorded for BCI systems calibration by generating synthetic EEG data as illustrated in Fig. 1. BCI systems rely on different EEG patterns in their operation. One EEG pattern that has been utilized in multiple applications is the P300 pattern [30]. This is a positive peak that appears in EEG signals approximately 300ms after the presentation of a rare anticipated stimulus. Such a stimulus could be auditory or visual. P300-based BCI systems typically rely on a machine learning binary classifier to discriminate between the P300 pattern and background activity (non-P300) [4]. To achieve acceptable subject-dependent classification accuracy, a large number of trials is required from the subject where one epoch typically consists of more than 10 trials for a single target. As a result, GAI techniques were introduced to augment the limited amount of data. Different GAN architectures have been utilized in this task including Deep Convolutional GAN (DCGAN), Conditional GAN (cGAN), Auxiliary Classifier GAN (ACGAN) and Gradient Penalty-based Wasserstein GANs (WGAN-GP) [31,32,33,34]. These GANs rely on Convolutional Neural Network.

Fig. 1
figure 1

Using GAI for EEG data augmentation

(CNN) architecture for both the Generator and the Discriminator, and/or Recurrent Neural Networks (RNNs). Synthetic data generated using GANs when added to the real recorded data led to an increase in the P300 recognition accuracy by 10–18% when either balancing the two classes of P300 and non-P300 or generating synthetic data that is at least 4 times the size of the real recorded data. In addition, reducing the training data size, in terms of the number of targets, by 30% and compensating for the dropped data using GANs does not significantly impact the accuracy. These results indicate that using GAI can reduce the amount of training data needed, leading to a shorter calibration time and less inconvenience for the users.

Another EEG pattern that has been utilized in a multitude of BCI applications is the Steady-state Visual Evoked Potential (SSVEP). This pattern is characterized by recording EEG signals with a frequency spectrum that resembles the frequency spectrum of a presented flickering visual stimulus [35]. In many SSVEP-based BCIs, machine learning classifiers are used to discriminate between the spectral signature of the SSVEPs of different flickering frequencies. One use of GAI in this context is, similar to that used with P300 patterns, to generate synthetic data to increase the size of the training dataset of the classifiers. Multiple GAI models were used in this task including VAE, DCGAN, WGAN and StarGAN to generate synthetic data in the time-domain as well as in the frequency-domain improving the SSVEP recognition accuracy by 2–20% [36,37,38]. Another use of GAI with SSVEP patterns is to generate synthetic data to increase the duration of the recorded real data which can help in obtaining a better spectrum estimation as well as reduce the amount of data to be recorded from the users for system calibration [39]. A Long Short-term Memory (LSTM)-based GAN is used along with ACGAN to generate signals of longer duration, which increases the SSVEP recognition accuracy by \(\sim\) 30%.

Motor Imagery (MI) represents one of the main BCI paradigms that relies on recognizing spectral patterns in the EEG that correspond to imagined movements [40]. Event Related Synchronization (ERS) and Event Related Desynchronization (ERD) patterns are recognized mainly in signals recorded from electrodes placed around the motor cortex brain area in both hemispheres. The majority of the literature on using GAI in BCI data augmentation is on augmenting MI data. Multiple studies considered using different GAI techniques in augmenting the time-domain EEG signals using VAE, DCGAN, and conditional WGAN (cWGAN) leading to an enhancement in the MI pattern classification accuracy of 1–18% depending on the dataset, the GAN architecture and the classification model [41,42,43]. Another approach is to first transform the recorded EEG signals into the frequency domain using Short-time Fourier Transform (STFT) and subsequently represent each trial as an image of the spectrum versus time. GAI is then used to generate synthetic images of STFTs to augment the recorded data. Using this approach, vanilla GAN and DCGAN were examined in this task reaching an enhancement in the overall MI pattern classification accuracy of 2.5–12% [44, 45].

Finally, it is worth mentioning that GAI has been used for BCI data augmentation in other tasks. For instance, to augment EEG data in the task of emotion recognition, the OpenAI Improved-diffusion model has been examined [46]. Additionally, multiple GAN approaches were examined in the same task including cWGAN and ACGAN increasing emotion recognition accuracy by 1.5–20% [47, 48].

GAI for EEG resolution enhancement

One of the applications in which GAI has demonstrated significant improvement in domains such as computer vision is generating super-resolution images from lower resolution versions [49]. In EEG signals analysis for BCIs, GAI has been applied to enhance both the spatial and temporal resolutions of the recorded EEG signals as illustrated in Fig. 2. First, considering the spatial resolution enhancement, recording EEG signals with high spatial resolution in terms of the number of electrodes used in the headset and the proximity of the electrodes is typically challenging and requires expensive equipment that might not be suitable for practical daily-usage BCI applications [50]. As a result, a number of studies has explored using GAI to enhance the spatial resolution of EEG signals. This is typically done by estimating the signals from electrodes that are not present in the EEG recording headset. For instance, multiple studies have employed Deep CNNs in this task in which the input is the data recorded from a limited number of electrodes and the target.

Fig. 2
figure 2

Using GAI for enhancing EEG spatial and temporal resolutions

output of the CNN is signals from a larger number of electrodes [51,52,53]. In these studies, the number of input electrodes was as low as 4 electrodes only while the number of output synthesized electrodes reached 60 electrodes. In addition, WGAN was also explored in the same task where signals from only 25% of the channels were used to generate signals from the remaining 75%. Surprisingly, this results in a minimal reduction of only 9% in the accuracy of mental imagery classification when comparing the performance achieved using the generated data to that achieved using the ground-truth data [54]. The generated high spatial-resolution signals demonstrated strong similarity with the reference signals indicating the efficacy of using GAI in this challenging task.

Another perspective of enhancing EEG spatial resolution is to map EEG signals to the underlying neural activity that could be recorded using other invasive neural data recording techniques. EEG is considered a low-pass filtered and distorted version of single-neuron activity [55]. Using GAI to map EEG to other underlying neural signals not only could help enhancing the spatial resolution of EEG for better BCI performance, it could also help in understanding how single neuron activity and local field potentials brain dynamics map to EEG. Deep neural networks comprised of residual networks (ResNet) and long short-term memory (LSTM) were utilized in this task to map EEG to mesoscale neural activity, achieving a mean correlation of 0.83 comparing the generated signals to the ground-truth [56]. Other studies explored using autoencoders, vanilla GANs, and conditional GANs to map EEG to invasively recorded intracranial EEG [57, 58]. Results reported by Abdi-Sargezeh et al. demonstrated the utility of using GANs to detect interictal epileptiform discharges with an accuracy reaching 76%, outperforming previous approaches [59].

In addition to enhancing EEG spatial resolution, GAI was also employed in enhancing the temporal resolution of the recorded EEG. This could be used to generate high temporal resolution EEG signals from lower temporal resolution ones. This could alleviate the challenges associated with developing high temporal resolution EEG recording devices such as the complexity and high cost of developing and operating such devices [60]. Vanilla GAN and WGANs were used to double the sampling rate of the recorded EEG leading to an enhancement in the classification accuracy of a MI dataset by\(\sim\)4% [43, 61]. Although not being thoroughly examined as the enhancement of spatial resolution, enhancing EEG temporal resolution using GAI techniques is promising. Further advances in this area could help in reducing the cost of EEG recording devices by augmenting the capabilities of low temporal-resolution devices.

GAI for cross-subject performance enhancement

Inter-subject variability represents one of the main challenges hindering the widespread use of BCI. Such significant variability requires recording a significant amount of training (calibration) data from each BCI user to train a subject-dependent model to be used to determine the user’s command or intention [21]. For a given target subject, achieving similar accuracy for that subject using a BCI trained using other subjects’ data, as shown in Fig. 3, compared to a BCI trained using the target subject’s data would imply that the trained BCI can be used without the need for any training data from the target subject. This could significantly simplify the setup of BCI systems and enhance their usability and scalability.

GAI techniques have been explored in the task of generating subject-independent data that could be used to train machine learning classifiers used in BCIs. GAI-based approaches were used to enhance P300-based BCI cross-subject performance, where hybrid CNN-LSTM GAN, DCGAN, cDCAGN and WGAN-GP GAN were examined in this task by balancing the dataset overcoming the imbalance between P300 and non-P300 samples [31, 33, 62]. The hybrid CNN-LSTM GAN showed the best performance in this case increasing the cross-subject accuracy by 10%. A similar idea was explored to enhance SSVEP-based BCIs in terms of their cross-subject performance. DCGAN, WGAN, ACGAN and VAEs were examined in this task resulting in performance improvements in the range of 3–35% [36, 63]. Finally, in MI-based BCIs, transformer models were examined with attention mechanisms enhancing the cross-subject performance in the range of 0.88–2.11% [64]. A CNN-based GAN that uses common spatial pattern filtering was also examined in this task along with DCGAN and VAE; enhancing the overall cross-subject accuracy by 15.85% [65]. Further utilization of other GAI techniques could help in identifying subject-independent features that would eliminate the need for subject-dependent data recording.

Fig. 3
figure 3

Using GAI to enhance BCI cross-subject performance

GAI for end-to-end BCI development

The previous sections demonstrated how GAI techniques have enhanced already existing BCI paradigms that use brain patterns such as P300, SSVEP and MI. However, the advances in GAI could be leveraged to design end-to-end BCI applications as shown in Fig. 4, not only enhancing the training of the BCI systems or enhancing the resolution of the recorded EEG signals.

One end-to-end BCI application is generating speech from recorded EEG signals. This application could be of particular interest to subjects with reduced communication abilities. Generating speech from recorded EEG signals have been demonstrated to be an extremely challenging task given the complexity of human speech and the lack of clear understanding of the mechanisms that could map EEG signals to speech [66]. However, GAI techniques have been shown to provide a framework that could succeed in this task. To achieve the objective of generating speech from recorded EEG, a number of studies examined first using GAI to generate heard speech from recorded EEG. A dual GAN architecture was used in this task achieving a similarity of 78.5% between the actual and generated speech for both single words and long sentences [67]. A hybrid CNN-RNN along with HiFi-GAN was examined in generating spoken as well as imagined speech from EEG signals demonstrating a strong similarity between the spectrograms of both with a root mean square error of 0.19 [68]. Denoising diffusion probabilistic models (DDPMs) were also used exceeding the accuracy of other models in generating speech by 14.5% [69].

Fig. 4
figure 4

Using GAI for developing end-to-end BCI applications generating audio, images and video from EEG signals

The extreme success GAI techniques demonstrated in generating realistic images and videos has inspired multiple research groups to explore the use of GAI techniques to generate images from EEG signals. Pre-trained stable diffusion models, VAEs and GANs were demonstrated to generate images from EEG signals [70, 71]. Stable diffusion models were shown to outperform other GAI techniques achieving a similarity of 45.8% between the actual and generated images. Another diffusion-based model that extracts multi-level semantics from recorded EEG signals achieved an accuracy of 85.6% when the generated images were evaluated using semantic classifiers [72]. A dual autoencoder was also employed in this task that is based on a CNN architecture [73]. Using semantic classifiers, images generated using this approach were classified with an accuracy of 85% using both inter-subject and intra-subject data. Another study demonstrated the use of ACGAN with attention modules in the same task reaching an accuracy enhancement of 22% compared to using an ACGAN architecture without attention [74].

Conclusions and future perspectives

While BCI has traditionally been considered a technology that allows people with motor disabilities to interact with computers, it is now considered as one emerging human-computer interaction (HCI) technology that could be used in the future by all users. Adopting BCI as a novel HCI technology for general users is expected to attract more research efforts in this domain, which will have a positive impact on the development of BCI application developed as assistive technologies for people with disabilities. A glimpse of the increased attention BCI is now given is manifested in the recent utilization of GAI techniques in this domain attempting to overcome challenges that hinder the smooth operation of current BCIs. In this article, four of these challenges were discussed; namely, limited data availability, low signal resolution, inter-subject variability and developing end-to-end BCI applications. Examples of GAI techniques that were recently utilized to tackle these challenges are reviewed in this article. In all four challenges, GAI techniques resulted in a significant enhancement in the performance of different BCI applications, which opens the door to further development of GAI techniques in this domain.

While GAI has been used, as demonstrated in this article, to augment recorded EEG data used in training machine learning classifiers in various applications, GAI techniques could also be explored in augmenting the test data. For instance, in P300 speller applications, the flashing of each row and column of characters is typically repeated for 10 to 15 times, and the corresponding recorded training signals are averaged across flashings (trials). This is necessary to overcome the low signal-to-noise ratio encountered in single trials. This process is also performed during the online use of the speller system, which requires significant amount of time in the order of minutes to type a single word [75, 76]. One venue in which GAI techniques could be utilized is to generate additional trials from a few recorded trials during the online use of the system. For instance, signals corresponding to only 5 trials could be recorded and then use GAI techniques to generate additional trials to be averaged with the signals of the recorded trials. This could reduce the time needed to type using P300 speller systems and hence enhance the information bit rate and the overall user experience. Another direction that could be pursued, in addition to speech and image synthesis discussed in this article, is video synthesis from EEG signals. The success different GAI techniques demonstrated in image synthesis from EEG signals indicates that video synthesis could be feasible. This can be further utilized in futuristic end-to-end BCI applications that aim at generating subject-specific creative content such as art or music.

The success of GAI techniques in enhancing BCI applications as shown here encourages further adoption of GAI to overcome other challenges associated with BCIs. One venue where GAI can be fused in the development of BCIs is to use Large Language Models (LLMs) to enhance the performance of BCIs used as spellers. LLMs have recently been demonstrated to solve many long-standing problems in natural language understanding and processing [77]. LLMs could be utilized for speller BCIs by using LLMs to provide word or even sentence suggestions based on a few EEG-based character selections made by a BCI user. While the use of language models in character and word predictions has been introduced before to enhance BCIs [78], the use of LLMs with BCIs is yet to be fully explored.

Despite the promise GAI holds in developing BCIs, a number of challenges need to be alleviated in the near future to leverage the full capabilities of GAI. First, GAI requires extensive training data to obtain reliable and realistic synthetic outputs [79]. While one of the main applications of GAI techniques is to augment limited data, training GAI models with limited data could result in overfitting which limits the ability of GAI techniques to generalize. For subject-dependent BCI applications, recording extensive data from each subject is challenging since it requires long-duration recording sessions during which subjects would typically have limited ability to move to avoid EEG motion artifacts [80]. However, recording such data might be feasible given the advances in consumer-grade EEG devices which target developing portable EEG recording headsets for daily use. This could enable GAI techniques to achieve further improvement in BCI applications performance. Additionally, applying GAI techniques to enhance the cross-subject performance of BCI applications requires the availability of a dataset with a large number of subjects. This dataset needs to be of sufficient size to provide GAI techniques with enough data to be used to extract subject-independent generalized representative features. Currently, most datasets used are in the order of tens of subjects. A large-scale study that collects data from hundreds (or ideally thousands) of subjects could be the key to training GAI techniques that overcome inter-subject variability, hence allowing BCIs to get closer to widespread adoption as a reliable assistive technology. Second, the convergence and stability of GAI techniques training cannot be easily achieved, which typically requires extensive hyperparameter tuning [81]. This might not be feasible for BCI applications since it would require performing hyperparameter tuning for each subject which requires significant computational resources. This would further complicate the calibration phase of BCIs. Additionally, using GAI techniques to generate data for BCI applications in real-time is another related challenge since GAI techniques are characterized by their inherent computational complexity. Third, GAI techniques have been demonstrated to hallucinate if their function is not controlled [82]. While this might not result in severe consequences in other domains, GAI hallucinations could represent a major problem in the context of BCI development since BCIs could be used in critical applications such as controlling the movement of a wheelchair or a robotic arm. A hallucinating GAI could lead to a life-threatening situation in this case. Finally, GAI techniques used in BCI development, as demonstrated in this article, are predominantly GAN-based. The utilization of other GAI techniques is yet to be explored. Tackling the aforementioned challenges could pave the way for further utilization of GAI techniques in BCI development, which could transform the field leading to the wide-spread use of BCIs.

Finally, while GAI can contribute to revolutionizing BCI development as demonstrated in this article, BCI could also play a role in developing GAI techniques. For decades, the brain has represented an inspiration to the development of AI models leading to significant strides in the evolution of AI models [83]. However, there is a gap between how AI models are trained and operate compared to how the brain learns and processes information [84, 85]. This gap is argued to be one of the reasons AI is still not capable of reaching the general cognitive abilities of humans. BCI could bridge this gap by recording human brain activity in different tasks which can be then used to train GAI techniques to generate embeddings that resemble the brain representation of information. Such a brain-in-the-loop approach could provide GAI techniques with additional dimensions of information representation, which could augment the capabilities of GAI techniques. Thus, the benefits of integrating GAI and BCI are bi-directional that can lead to significant enhancements in both domains in the near future.

Data availability

Not applicable.

References

  1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophys. 2002;113:767–91.

    Article  Google Scholar 

  2. Saha S, Mamun KA, Ahmed K, Mostafa R, Naik GR, Darvishi S, Khandoker AH, Baumert M. Progress in brain computer interface: challenges and opportunities. Front Syst Neurosci. 2021;15:578875.

    Article  Google Scholar 

  3. MILLÁN DELR, Ferrez J, Galán PW, Lew F, Chavarriaga E. Non-invasive brain-machine interaction. Int J Pattern Recognit Artif Intell. 2008;22:959–72.

    Article  Google Scholar 

  4. Krusienski DJ, Sellers EW, Cabestaing F, Bayoudh S, McFarland DJ, Vaughan TM, Wolpaw JR. A comparison of classification techniques for the P300 speller. J Neural Eng. 2006;3:299.

    Article  Google Scholar 

  5. Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J Neurosci Methods. 2008;167:15–21.

    Article  Google Scholar 

  6. Elsawy AS, Eldawlatly S, Taher M, Aly GM. MindEdit: a P300-based text editor for mobile devices. Comput Biol Med 2017, 80:97–106.

  7. Perdikis S, Leeb R, Williamson J, Ramsay A, Tavella M, Desideri L, Hoogerwerf E-J, Al-Khodairy A, Murray-Smith R., d R Millán J: clinical evaluation of BrainTree, a motor imagery hybrid BCI speller. J Neural Eng. 2014;11:036003.

    Article  Google Scholar 

  8. Tanaka K, Matsunaga K, Wang HO. Electroencephalogram-Based Control of an Electric Wheelchair. IEEE Trans Robot. 2005;21:762–6.

    Article  Google Scholar 

  9. Tonin L, Perdikis S, Kuzu TD, Pardo J, Orset B, Lee K, Aach M, Schildhauer TA, Martínez-Olivera R. Millán JdR: learning to control a BMI-driven wheelchair for people with severe tetraplegia. Iscience 2022, 25.

  10. Tariq M, Trivailo PM, Simic M. EEG-based BCI control schemes for lower-limb assistive-robots. Front Hum Neurosci. 2018;12:312.

    Article  Google Scholar 

  11. Vidaurre C, Klauer C, Schauer T, Ramos-Murguialday A, Müller K-R. EEG-based BCI for the linear control of an upper-limb neuroprosthesis. Med Eng Phys. 2016;38:1195–204.

    Article  Google Scholar 

  12. Baniqued PDE, Stanyer EC, Awais M, Alazmani A, Jackson AE, Mon-Williams MA, Mushtaq F, Holt RJ. Brain–computer interface robotics for hand rehabilitation after stroke: a systematic review. J Neuroeng Rehabil. 2021;18:1–25.

    Article  Google Scholar 

  13. Chamola V, Vineet A, Nayyar A, Hossain E. Brain-computer interface-based humanoid control: a review. Sensors. 2020;20:3620.

    Article  Google Scholar 

  14. Tonin L, Millán JR. Noninvasive brain–machine interfaces for robotic devices. Annual Rev Control Rob Auton Syst. 2021;4:191–214.

    Article  Google Scholar 

  15. Nijholt A, Bos DO, Reuderink B. Turning shortcomings into challenges: brain-computer interfaces for games. Entertain Comput. 2009;1:85–94.

    Article  Google Scholar 

  16. Kohli V, Tripathi U, Chamola V, Rout BK, Kanhere SS. A review on virtual reality and augmented reality use-cases of Brain Computer Interface based applications for smart cities. Microprocess Microsyst. 2022;88:104392.

    Article  Google Scholar 

  17. Chavarriaga R, Ušćumlić M, Zhang H, Khaliliardali Z, Aydarkhanov R, Saeedi S, Gheorghe L, Millán JR. Decoding neural correlates of cognitive states to enhance driving experience. IEEE Trans Emerg Top Comput Intell. 2018;2:288–97.

    Article  Google Scholar 

  18. Mustafa Y, Elmahallawy M, Luo T, Eldawlatly S: A brain-computer interface augmented reality framework with auto-adaptive ssvep recognition. In. 2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE). IEEE; 2023: 799–804.

  19. Simon C, Bolton DA, Kennedy NC, Soekadar SR, Ruddy KL. Challenges and opportunities for the future of brain-computer interface in neurorehabilitation. Front NeuroSci. 2021;15:699428.

    Article  Google Scholar 

  20. Sabio J, Williams NS, McArthur GM, Badcock NA. A scoping review on the use of consumer-grade EEG devices for research. PLoS ONE. 2024;19:e0291186.

    Article  Google Scholar 

  21. Saha S, Baumert M. Intra-and inter-subject variability in EEG-based sensorimotor brain computer interface: a review. Front Comput Neurosci. 2020;13:87.

    Article  Google Scholar 

  22. Yadav D, Yadav S, Veer K. A comprehensive assessment of Brain Computer interfaces: recent trends and challenges. J Neurosci Methods. 2020;346:108918.

    Article  Google Scholar 

  23. Gozalo-Brizuela R, Garrido-Merchan EC. ChatGPT is not all you need. A state of the art review of large generative AI models. arXiv Preprint arXiv:230104655 2023.

  24. Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, Sun L. A comprehensive survey of Ai-generated content (aigc): a history of generative ai from gan to chatgpt. arXiv Preprint arXiv:230304226 2023.

  25. Habashi AG, Azab AM, Eldawlatly S, Aly GM. Generative adversarial networks in EEG analysis: an overview. J Neuroeng Rehabil. 2023;20:40.

    Article  Google Scholar 

  26. Kingma DP, Welling M. Auto-encoding variational bayes. arXiv Preprint arXiv:13126114 2013.

  27. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Commun ACM. 2020;63:139–44.

    Article  Google Scholar 

  28. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst 2017, 30.

  29. Yang L, Zhang Z, Song Y, Hong S, Xu R, Zhao Y, Zhang W, Cui B, Yang M-H. Diffusion models: a comprehensive survey of methods and applications. ACM Computing Surveys; 2022.

  30. Picton TW. The P300 wave of the human event-related potential. J Clin Neurophysiol. 1992;9:456–456.

    Article  Google Scholar 

  31. Kunanbayev K, Abibullaev B, Zollanvari A. Data augmentation for p300-based brain-computer interfaces using generative adversarial networks. In 2021 9th International Winter Conference on Brain-Computer Interface (BCI). IEEE; 2021: 1–7.

  32. Abdelghaffar Y, Hashem A, Eldawlatly S. Generative Adversarial Networks for Augmenting EEG Data in P300-based Applications: A Comparative Study. In 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2022: 1–6.

  33. Bhandari V, Londhe ND, Kshirsagar GB. TL-P3GAN: an efficient temporal learning-based Generative Adversarial Network for Precise P300 Signal Generation for P300 spellers. IEEE Trans Cogn Dev Syst 2023.

  34. Zhang R, Zeng Y, Tong L, Shu J, Lu R, Yang K, Li Z, Yan B. Erp-wgan: a data augmentation method for EEG single-trial detection. J Neurosci Methods. 2022;376:109621.

    Article  Google Scholar 

  35. Vialatte F-B, Maurice M, Dauwels J, Cichocki A. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Prog Neurobiol. 2010;90:418–38.

    Article  Google Scholar 

  36. Aznan NKN, Atapour-Abarghouei A, Bonner S, Connolly JD, Al Moubayed N, Breckon TP. Simulating brain signals: Creating synthetic eeg data via neural-based generative models for improved ssvep classification. In 2019 International joint conference on neural networks (IJCNN). IEEE; 2019: 1–8.

  37. Kwon J, Im C-H. Novel Signal-to-Signal translation method based on StarGAN to generate artificial EEG for SSVEP-based brain-computer interfaces. Expert Syst Appl. 2022;203:117574.

    Article  Google Scholar 

  38. Rosa G, Laitz T, Vargas G, Boccato L. Application of Generative Models for Synthesizing EEG Signals in BCI-SSVEP Systems. Available at SSRN 4385951.

  39. Pan Y, Li N, Zhang Y. Short-time SSVEP data extension by a novel generative adversarial networks based framework. arXiv Preprint arXiv:230105599 2023.

  40. Singh A, Hussain AA, Lal S, Guesgen HW. A comprehensive review on critical issues and possible solutions of motor imagery based electroencephalography brain-computer interface. Sensors. 2021;21:2173.

    Article  Google Scholar 

  41. Fahimi F, Dosen S, Ang KK, Mrachacz-Kersting N, Guan C. Generative adversarial networks-based data augmentation for brain–computer interface. IEEE Trans Neural Networks Learn Syst. 2020;32:4039–51.

    Article  Google Scholar 

  42. Li Z, Yu Y. Improving EEG-based motor imagery classification with conditional Wasserstein GAN. In 2020 International Conference on Image, Video Processing and Artificial Intelligence. SPIE; 2020: 437–443.

  43. Luo T-j, Fan Y, Chen L, Guo G, Zhou C. EEG signal reconstruction using a generative adversarial network with wasserstein distance and temporal-spatial-frequency loss. Front Neuroinformatics. 2020;14:15.

    Article  Google Scholar 

  44. Zhang K, Xu G, Han Z, Ma K, Zheng X, Chen L, Duan N, Zhang S. Data augmentation for motor imagery signal classification based on a hybrid neural network. Sensors. 2020;20:4485.

    Article  Google Scholar 

  45. Habashi AG, Azab AM, Eldawlatly S, Aly GM. Motor Imagery Classification Enhancement using Generative Adversarial Networks for EEG Spectrum Image Generation. In 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2023: 354–359.

  46. Tosato G, Dalbagno CM, Fumagalli F. EEG Synthetic Data Generation Using Probabilistic Diffusion Models. arXiv preprint arXiv:230306068 2023.

  47. Luo Y, Zhu L-Z, Wan Z-Y, Lu B-L. Data augmentation for enhancing EEG-based emotion recognition with deep generative models. J Neural Eng. 2020;17:056021.

    Article  Google Scholar 

  48. Fu B, Li F, Niu Y, Wu H, Li Y, Shi G. Conditional generative adversarial network for EEG-based emotion fine-grained estimation and visualization. J Vis Commun Image Represent. 2021;74:102982.

    Article  Google Scholar 

  49. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4681–4690.

  50. Mihajlović V, Grundlehner B, Vullers R, Penders J. Wearable, wireless EEG solutions in daily life applications: what are we missing? IEEE J Biomedical Health Inf. 2014;19:6–21.

    Article  Google Scholar 

  51. Svantesson M, Olausson H, Eklund A, Thordstein M. Virtual EEG-electrodes: convolutional neural networks as a method for upsampling or restoring channels. J Neurosci Methods. 2021;355:109126.

    Article  Google Scholar 

  52. Kwon M, Han S, Kim K, Jun SC. Super-resolution for improving EEG spatial resolution using deep convolutional neural network—feasibility study. Sensors. 2019;19:5317.

    Article  Google Scholar 

  53. Tang Y, Chen D, Liu H, Cai C, Li X. Deep EEG superresolution via correlating brain structural and functional connectivities. IEEE Trans Cybernetics 2022.

  54. Corley IA, Huang Y. Deep EEG super-resolution: Upsampling EEG spatial resolution with generative adversarial networks. In 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI). IEEE; 2018: 100–103.

  55. Olejniczak P. Neurophysiologic basis of EEG. J Clin Neurophysiol. 2006;23:186–9.

    Article  Google Scholar 

  56. Sun R, Sohrabpour A, Worrell GA, He B. Deep neural networks constrained by neural mass models improve electrophysiological source imaging of spatiotemporal brain dynamics. Proceedings of the National Academy of Sciences 2022, 119:e2201128119.

  57. Antoniades A, Spyrou L, Martin-Lopez D, Valentin A, Alarcon G, Sanei S, Took CC. Deep neural architectures for mapping scalp to intracranial EEG. Int J Neural Syst. 2018;28:1850009.

    Article  Google Scholar 

  58. Hu M, Chen J, Jiang S, Ji W, Mei S, Chen L, Wang X. E2SGAN: EEG-to-SEEG translation with generative adversarial networks. Front NeuroSci. 2022;16:971829.

    Article  Google Scholar 

  59. Abdi-Sargezeh B, Oswal A, Sanei S. Mapping Scalp to Intracranial EEG using Generative Adversarial Networks for Automatically Detecting Interictal Epileptiform Discharges. In 2023 IEEE Statistical Signal Processing Workshop (SSP). IEEE; 2023: 710–714.

  60. Qin Z, Li Q. High rate BCI with portable devices based on EEG. Smart Health. 2018;9:115–28.

    Article  Google Scholar 

  61. Hartmann KG, Schirrmeister RT, Ball T. EEG-GAN: generative adversarial networks for electroencephalograhic (EEG) brain signals. arXiv Preprint arXiv:180601875 2018.

  62. Panwar S, Rad P, Quarles J, Huang Y. Generating EEG signals of an RSVP experiment by a class conditioned wasserstein generative adversarial network. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE; 2019: 1304–1310.

  63. Aznan NKN, Atapour-Abarghouei A, Bonner S, Connolly JD, Breckon TP: Leveraging synthetic subject invariant EEG signals for zero calibration BCI. In. 2020 25th International Conference on Pattern Recognition (ICPR). IEEE; 2021: 10418–10425.

  64. Xie J, Zhang J, Sun J, Ma Z, Qin L, Li G, Zhou H, Zhan Y. A transformer-based approach combining deep learning network and spatial-temporal information for raw EEG classification. IEEE Trans Neural Syst Rehabil Eng. 2022;30:2126–36.

    Article  Google Scholar 

  65. Song Y, Yang L, Jia X, Xie L. Common spatial generative adversarial networks based EEG data augmentation for cross-subject brain-computer interface. arXiv Preprint arXiv:210204456 2021.

  66. Panachakel JT, Ramakrishnan AG. Decoding covert speech from EEG-a comprehensive review. Front NeuroSci. 2021;15:392.

    Article  Google Scholar 

  67. Guo Y, Liu T, Zhang X, Wang A, Wang W. End-to-end translation of human neural activity to speech with a dual–dual generative adversarial network. Knowl Based Syst. 2023;277:110837.

    Article  Google Scholar 

  68. Lee Y-E, Lee S-H, Kim S-H, Lee S-W. Towards voice reconstruction from EEG during imagined speech. In Proceedings of the AAAI Conference on Artificial Intelligence. 2023: 6030–6038.

  69. Kim S, Lee Y-E, Lee S-H, Lee S-W. Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG. arXiv preprint arXiv:230714389 2023.

  70. Bai Y, Wang X, Cao Y, Ge Y, Yuan C, Shan Y. DreamDiffusion: Generating High-Quality images from Brain EEG signals. arXiv Preprint arXiv:230616934 2023.

  71. Kavasidis I, Palazzo S, Spampinato C, Giordano D, Shah M. Brain2image: Converting brain signals into images. In Proceedings of the 25th ACM international conference on Multimedia. 2017: 1809–1817.

  72. Lan Y-T, Ren K, Wang Y, Zheng W-L, Li D, Lu B-L, Qiu L. Seeing through the brain: Image Reconstruction of Visual Perception from human brain signals. arXiv Preprint arXiv:230802510 2023.

  73. Zeng H, Xia N, Tao M, Pan D, Zheng H, Wang C, Xu F, Zakaria W, Dai G. DCAE: a dual conditional autoencoder framework for the reconstruction from EEG into image. Biomed Signal Process Control. 2023;81:104440.

    Article  Google Scholar 

  74. Mishra R, Sharma K, Jha R, Bhavsar A. NeuroGAN: image reconstruction from EEG signals via an attention-based GAN. Neural Comput Appl. 2023;35:9181–92.

    Google Scholar 

  75. Lu J, Speier W, Hu X, Pouratian N. The effects of stimulus timing features on P300 speller performance. Clin Neurophysiol. 2013;124:306–14.

    Article  Google Scholar 

  76. Ryan DB, Frye G, Townsend G, Berry D, Mesa-G S, Gates NA, Sellers EW. Predictive spelling with a P300-based brain–computer interface: increasing the rate of communication. Intl J Human–Computer Interact. 2010;27:69–84.

    Article  Google Scholar 

  77. Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z. A survey of large language models. arXiv Preprint arXiv:230318223 2023.

  78. Speier W, Arnold C, Pouratian N. Integrating language models into classifiers for BCI communication: a review. J Neural Eng. 2016;13:031002.

    Article  Google Scholar 

  79. Bansal MA, Sharma DR, Kathuria DM. A systematic review on data scarcity problem in deep learning: solution and applications. ACM Comput Surv (CSUR). 2022;54:1–29.

    Article  Google Scholar 

  80. Soufineyestani M, Dowling D, Khan A. Electroencephalography (EEG) technology applications and available devices. Appl Sci. 2020;10:7453.

    Article  Google Scholar 

  81. Saxena D, Cao J. Generative adversarial networks (GANs) challenges, solutions, and future directions. ACM Comput Surv (CSUR). 2021;54:1–42.

    Article  Google Scholar 

  82. Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM-CSUR. 2023;55:1–38.

    Google Scholar 

  83. Fan J, Fang L, Wu J, Guo Y, Dai Q. From brain science to artificial intelligence. Engineering. 2020;6:248–52.

    Article  Google Scholar 

  84. Song Y, Millidge B, Salvatori T, Lukasiewicz T, Xu Z, Bogacz R. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nat Neurosci 2024:1–11.

  85. Hole KJ, Ahmad S. A thousand brains: toward biologically constrained AI. SN Appl Sci. 2021;3:743.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

Authors

Contributions

The author of this article (S.E.) performed the literature review and wrote the manuscript.

Corresponding author

Correspondence to Seif Eldawlatly.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eldawlatly, S. On the role of generative artificial intelligence in the development of brain-computer interfaces. BMC biomed eng 6, 4 (2024). https://doi.org/10.1186/s42490-024-00080-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s42490-024-00080-2

Keywords