Autism Detection from 2D Transformed EEG Signal using Convolutional Neural Network

: Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder relating to speech complications, nonverbal and social communication, and repetitive behaviors. There is no remedy for ASD but early diagnosis, mediation, and supportive care can aid the development of language, conduct, and communication skills. As the cause of ASD is a neurodevelopmental disorder, its diagnosis based on brain function analyzing different brain signals, especially Electroencephalography (EEG), has drawn attention recently. Brain activity is recorded over time as an EEG signal from the scalp of a human and is used to investigate complicated neuropsychiatric disorders in the brain. In this study, the data from the EEG channels are translated into two-Dimensional (2D) form through correlation, and classification is performed using Convolutional Neural Networks (CNN), the well-known deep learning method for image analysis and classification. Two different CNN models are considered for classification purposes: Generic CNN and Residual Network (ResNet), a well-known deep CNN model. The proposed method with Resnet achieved 88% classification accuracy on a five-fold cross-validation mode, whereas it was 100 on 20% of test samples. Experimental evaluations using clinical EEG data revealed the efficacy of the proposed method outperforming other existing methods.


Introduction
Autism Spectrum Disorder (ASD) or autism is caused by a variety of factors in the brain that result in behavior and communication abnormalities as well as limited stereotyped behaviors (Peya et al., 2020;Johnson and Myers, 2007). It is known as a spectrum disorder due to a wide variety of types and severity of symptoms such as repetitive behavior, language, and sensory problems, cognitive deficits, anxiety, social retreat, etc. ASD is exposed in childhood and progresses through adolescence and adulthood (WHO, 2013). The majority of ASD side effects occur during the first five years of life. Some indicators become visible from facial appearance. The case of an autistic child is depicted in Fig. 1. Nowadays ASD discovery is a vital issue because it influences the mental as well as physical development of kids. Numerous studies in the last few decades have indicated that one out of every 68 children have ASD, however, according to more recent reports, one out of every 59 children is diagnosed with ASD (de Diego-Otero and Salgado-Cacho, 2019). The prevalence of ASD is increasing globally according to other studies conducted around the world (Hossain, 2018).
ASD is a lifelong condition and no cure exists for ASD, but there are useful treatments/interventions available. There is an increasing number of proof that ASD may not be a long-lasting condition (CDC, 2018). Several studies revealed that a significant number of children who were initially identified with autism no longer follow the diagnosis later on. This has led to increased interest in the early detection of ASD which can help in later life. While ASD can be detected as early as two years old, most children are diagnosed after the age of four. At the age of three, less than half of young patients had been diagnosed with ASD. Early detection and intervention are most beneficial, as they can improve mental, communication, and language skills development (Landa, 2008). ASD can be diagnosed in a variety of ways. Symptoms of ASD include persistent deficits in social communications with others (Walsh et al., 2011). Some methods diagnose ASD utilizing a questionnaire-based screening approach that only makes recommendations for further lookup. In screening (Greenspan et al., 2008), the child gets a brief test and the parents have to complete a questionnaire about the kid's development including language, thinking, behavior and emotions following a standard questionnaire or checklist. Screening is also performed by doctors, nurses, or other experts in medical care, local area, or school settings. Some approaches include communicating with patients and collaborating with a multidisciplinary team (Huerta and Lord, 2012) to diagnose ASD and the team may be comprised of medical specialists, therapists, psychologists, and special educators. The method considers different areas of functioning during a diagnostic assessment. Smartphone and tablet apps are commercially available for autistic patients and their families (Collins, 2019) which employ activities to diagnose ASD that encourage kids to act as artists or the sensation of being surprised. The children's play is videotaped to assess and detect signs of autism.
As the cause of ASD is a neurodevelopmental disorder, its diagnosis based on brain function analyzing different brain signals has drawn attention recently. Magnetic Resonance Imaging (MRI), functional MRI (fMRI), Electroencephalography (EEG), and Magnetoencephalography (MEG) are a few well-known noninvasive neuroimaging techniques for measuring brain functions. EEG, among the various brain signal methods, is increasingly being investigated as a possible diagnostic tool for tracking brain activities due to its ease of use and low cost (Bosl et al., 2018;Korik et al., 2018Korik et al., , 2016a. EEG observes the brain's electrical activity using electrodes attached to various parts of the skull. Individual channels' data are collected as an EEG signal for a certain period. The number of data points depends on the number of channels, frequency of data collection, and time. EEG signals have been investigated for studying human brain activities. Hence, its use in studying ASD is increasing (Bosl et al., 2018;Korik et al., 2018) (Grossi et al., 2017;Hadoush et al., 2019;Ibrahim et al., 2018;Kang et al., 2019).
Machine Learning (ML) methods have been popular with researchers in recent studies for early ASD detection from EEG signals (Ari et al., 2022). ML employs computer algorithms to model or learns patterns in data and signals. There are two major steps in ML-based ASD detection from EEG: Processing data or signals and then classifying them using the appropriate ML method. In the first step, the collected data are pre-processed, transformed, and represented in the required format for the intended ML method. Finally, an ML method is prepared or trained with the data to recognize ASD. Along with different processing techniques, different ML methods are investigated in the last several years for ASD detection (Brihadiswaran et al., 2019) including Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), and Artificial Neural Network (ANN) (Bosl et al., 2018;Grossi et al., 2017;Hadoush et al., 2019;Ibrahim et al., 2018;Kang et al., 2019). Deep learning methods have recently been used to detect ASD from EEG signals (Radhakrishnan et al., 2021;Khodatars et al., 2021;Ali et al., 2020). The performance of existing ML-based methods is promising, which motivates the current investigation, especially the deep learning method, intending to improve efficiency.
Convolutional Neural Network (CNN) has been employed in ASD detection from EEG signals in this study. CNN is a deep learning technique with the added advantage of incorporating the feature extraction process within its structure, which makes CNN suitable for 2D (i.e., two-dimensional) image analysis and classification (Akhand et al., 2018).
This feature of CNN does not restrict its application to problems involving 1D signals such as EEG signals. Firstly, independent EEG channel data are required for conversion into a 2D image employing Pearson's Correlation Coefficient (PCC) in the proposed approach (Pearson, 1895). Finally, generic CNN and residual neural network (ResNet), the widely recognized deep CNN model, are applied to the image like 2D data for classification. Although CNN-based ASD detection has been outlined in our previous study (Peya et al., 2020), the present study is an extended and complete presentation in terms of both theoretical analysis and experimental outcomes.
The rest of the paper reviews several existing approaches to ASD detection clarifies the suggested approach and the experimental findings and comparison with existing methods. Finally, the paper concludes with a few remarks.

Literature Review
Many ML-based methods are investigated in the last several years for early ASD detection using different brain signals (Bosl et al., 2018;Grossi et al., 2017;Hadoush et al., 2019;Ibrahim et al., 2018;Kang et al., 2019). Most of the 697 methods are based on EEG and a few are with other brain signals, e.g., fMRI. Akter et al. (2017) used fMRI images for detecting autism from the activation region of the fusiform gyrus of the human brain to diagnose ASD. MRI images of the normal and the autistic brain were preprocessed and features were extracted to calculate area, perimeter, and eccentricity for further analysis. Different mean areas, perimeter, and eccentricity showed a significant difference between normal and autistic brains. Zou et al. (2017) build up a deep learning-based Attention Deficit Hyperactivity Disorder (ADHD) classification, a type of autism, strategy through 3D CNNs applied to MRI images. They designed the 3D CNN model to investigate the local spatial patterns of MRI features. Structural MRI (sMRI) and fMRI features were combined with the proposed CNN model. The method achieved 69.15% accuracy on hold-out testing data of the ADHD-200 global competition. Jamal et al. (2014) investigated the nearness of mental imbalance utilizing the utilitarian mind availability estimates got from EEG signals of kids during face observation tasks. The leave one out cross-validation of the characterization calculation gives 94.7% precision as the best execution with relating affectability and explicitness esteems as 85.7 and 100% individually. Mullick et al. (2013) conducted a cross-sectional analysis to determine the proportion of sMRI and EEG abnormalities in autistic children, as well as any connection between MRI and EEG changes and comorbid mental illness. A total of 42 autistic children between the ages of two and twelve participated in the study at a child and adolescent consultation center. EEG changes were linked to a rise in the number of co-morbid illnesses (epilepsy, mental retardation, and others). They observed various anomalies indicative of relations among structural and physiological dysfunctions. Grossi et al. (2017) proposed an algorithm named MS-ROM/I-FAST for detecting ASD from EEG signals using ANN. They collected real-time EEG data from Vila Santa Maria, Italy using DSM-IV criteria. Their proposed system predicted a feature named TWIST and formed an invariant feature vector input of EEG signal which was blindly classified with ANN. The accuracy of their system for distinguishing ASD subjects from typically developing was 100% using the train-test protocol and 92.8% using the leave-one-out protocol. Raja and Priya (2006) used EEG signals to characterize autistic and normal children through ANN. The authors collected a real-time dataset from ASD children in various special schools. They used two ANN models named Layered Recurrent Neural network (LRN) and pattern recognition neural network for classification. They obtained a classification accuracy of 94.62% combining Autoregressive (AR) Burg and LRN. Heunis et al. (2018) investigated Recurrence Quantification Analysis (RQA) as a possible biomarker for autism through methodological investigation of technical confounders. On continuous 5 sec segments of resting-state EEG data, RQA feature extraction was accomplished. They performed classification through a leave-one-out protocol with a nonlinear SVM classifier and obtained 92.9% accuracy. Bosl et al. (2018) observed EEG as a potential clinical method for tracking atypical brain growth. Starting at the age of three months and lasting until the age of 36 months, they gathered real-time EEG data from 89 low-risk control subjects and 99 children with an older child who was diagnosed with ASD. The classification was performed through SVM. The prognosis of the clinical diagnostic outcome of autism and the control group was surprisingly accurate when utilizing EEG predictions as early as 3 months old enough. Harun et al. (2018) proposed an effective diagnostic method for ASD using EEG signals based on emotions. They analyzed EEG signals of autistic and normal subjects for different types of emotions and then measured the difference. Emotions of normal subjects and autistic subjects were classified separately employing an ANN and SVM which achieved ASD detection with accuracies of 90.5 and 88.1%, respectively.
Recently, Ali et al. (2020) performed a CNN-based ASD diagnosis using an EEG signal. The data was collected using a 16-channel acquisition system at a sampling rate of 256 Hz. Every second's data is converted into 16 × 256 sized 2D matrix form, which was then fed into CNN for classification of ASD. This method achieved an accuracy of 80%.

Materials and Methods
Recently, brain signals, especially EEG, are widely used for detecting ASD and other neurodevelopmental disorders (Büchel et al., 2021) (Zhang et al., 2020). To apply CNN to 1D EEG signals, the novel aim of this study is to represent 1D EEG signals in 2D (two-dimensional) format and then classify ASD data using CNN. PCC is used to convert EEG data into a two-dimensional format. Fig. 2 illustrates the proposed ASD detection method showing three key steps: EEG signal acquisition, the 2D transformation of signal, and classification using CNN. The outcome of the proposed system is the classification of input EEG signals into ASD and control (i.e., non-ASD) categories. The following subsections briefly explain individual steps.

EEG Signal Acquisition
Electroencephalography (EEG) is a non-invasive technique for observing brain electrical activity by putting electrodes on the skull (Misra and Kalita, 2018). EEG is a useful method for analyzing human brainwaves and reflecting knowledge arrangement on acquired signals (Gao and Lee, 2015). An EEG can assess changes in brain activity that may be useful in the diagnosis of brain disorders. A system of electrode placement is needed to establish a consistent relationship between electrode position and underlying cerebral structures. The international 10-20 method of electrode placement for EEG signal recording is shown in schematic form in Fig. 3. In the figure, the anteroposterior estimations are focused on the gap between nasion and inion over the vertex in the midline with five category positions: Central (C), Frontal pole (Fp), Frontal (F), Parietal (P) and Occipital (O). It was agreed to utilize even numbers as the index for the right hemispheres and odd numbers for the left hemispheres to distinguish between homologous situations on the left and right sides of the hemispheres. The first point of Fp (i.e., Fp1), for example, is 10% of the nasion-inion distance from the nasion. Data from several channels is collected and preserved in a computer system.

Transformation of EEG Data into 2D Image using PCC
EEG signals collected via 19 channels are inherently 1D data. Classification of these data using a CNN classifier poses challenges while CNN requires data to be input in 2D form. There have been several methods appeared in the literature. Wu et al. (2018) showed that 2D CNN works better than 1D CNN for ECG signal classification. They transformed 1D ECG signals into 2D images to apply to 2D CNN. In general, the standard 1D EEG signal analysis requires feature extraction, which mostly disregards the spatial, spectral, and temporal structure present in the data. To extract spatial, spectral, and temporal features, (Sharma et al., 2019) employed the transformation of non-image data into image data to apply to CNN. Sun et al. (2021) suggests a transformation of 1D EEG data into 2D images for maintaining the spatial structure of the data. Naz et al. (2021) used a segmentation technique for transforming 1D ECG signals into 32 × 32 binary images for applying to CNN.
To analyze and classify EEG signals using 2D CNN, Pearson Correlation Coefficient (PCC) has been investigated in this research. PCC is a method for evaluating the demographic relationship, or correlation, between two consistent variables (Kirch, 2008). Pearson product-moment correlation coefficient, Pearson's r, or the bivariate correlation are popular concepts widely used in statistics refer to as PCC.
PCC is calculated by multiplying the covariance of two variables by their standard deviations. The term "product moment" refers to the mean (first moment about the origin) of the product of mean-adjusted random variables, which is included in the definition. When applied to a population, PCC is usually denoted by ρ (rho) and is also known as the population correlation coefficient or the population PCC. For a given pair of random variables (X, Y), ρ is defined by: +1 indicates complete positive linear correlation, 0 means no straight relationship and -1 signifies all-out negative linear correlation. The advantage of using PCC is that the PCC matrix can be presented as a heat map (Friendly and Kwan, 2003). The strength of heat maps to display the features of a PCC matrix was investigated by Friendly and Kwan (Friendly and Kwan, 2003). Correlation plots, regarded as a heat map, can be used to visualize association matrices and represents multiple correlation statistics (Haarman et al., 2015). There has been little reported on the use of 2D images produced from the PCC matrix applied for the analysis and classification of signals.
In this study, PCC is measured between individual channels' data and transformed into a 2D image. Initially, the PCC is resolved to individual PCC values by combining the two channels. After that, the values are ordered in a matrix form where an information cluster can be found. The size of the PCC matrix is 19 × 19 as data of 19 channels are considered in this study. Utilizing the seaborn heatmap library of Python, the matrix was visualized as a color image. Instances of produced color images are shown in Fig. 4 for twosample data. Figure 4(a) shows the image of the ASD subject and Fig. 4(b) shows the image of the control subject. The presentation visualizes the difference between the ASD subject in Fig. 4(a) and the control (i.e., non-ASD) subject in Fig. 4(b).

Classification using CNN
CNN is the widely used method with significant features (e.g., convolutional, pooling) to handle 2D objects such as images (Akhand et al., 2018;Namatēvs, 2017;Saha et al., 2005). In this study, two different CNN models are investigated for classifying transformed EEG data: Generic CNN and residual network (ResNet) (Alom et al., 2019), a well-known deep CNN model. The input is a 2D image of size 64 × 64, which is used for both models. Hence, all the images were resized to 64 × 64.
The architecture of the generic CNN is shown in Fig. 5; there are four convolution layers and four max-pooling layers. Four convolutional layers are used to suit the particular dataset. The CNN architecture with 4 convolution layers verified on the dataset provided the best result. In each convolution layer, the Rectified Linear unit (ReLu) activation function was used. The inputs to the CNN are the EEG data transformed into 2D images (i.e., 64 × 64 sized RGB images) which are 699 placed in the first convolution layer (Convolutional #1) operation. Using 3 × 3 sized 32 kernels, 62 × 62 sized 32 convolved Feature Maps (FMs) were the outcome of the Convolutional #1 layer. Furthermore, the convolved FMs were fed into the next layer, which was the max-pooling layer (Max Pooling #1) with a pool size of 2 × 2 for feature extraction. This layer down-sampled the representation of the input by taking the maximum value for each dimension by pool size along with the axis of features and the resulting 31 × 31 sized 32 FMs. In the rest of the convolution layers, the kernel size was the same but the number of kernels (hence feature maps) was increased to 64 in the second (Convolutional #2), 128 in the third (Convolutional #3), and fourth (Convolutional #4) convolution layers. After each convolution layer, the max-pooling method with a pool size of 2 × 2 was implemented.
Total 2 × 2 sized 128 matrices are the outcomes after successive operations in four convolution-pooling layers. The resulting matrices were flattened into 512 nodes (= 128 × 2 × 2) in a single column. The next two dense layers (i.e., fully connected) having nodes 512 and 64 were served by this flattening layer; and finally, the output of CNN comes from a second dense layer with a single node indicating if the subject is either ASD or Control. The node has a threshold value of 0.5. When output falls below the threshold, it is in the ASD group otherwise it is the control group.
ResNet (Alom et al., 2019), a popular CNN model with a significant connection mechanism, is also considered in this study. A ResNet architecture is made up of several blocks, called residual blocks. Figure 6 shows a block diagram of a typical residual block showing skipping connections between convolutional layers. Skipping connections can transform greater gradients into initial layers and these layers can also learn as quickly as final layers. There are several ResNet models available having a different number of convolutional layers. In this study, the ResNet20v1 (CIFAR-10 ResNet) model is employed, and the model's numeric value represents the depth (i.e., number of convolutional layers) of the model. In the model, each convolutional layer employed a 3 × 3 sized kernel. The average pooling layer is used to replace completely connected layers at the end of the model. In the output layer, a SoftMax activation function is used to calculate the probability for classification. A detailed description and implementation of ResNet can be found in (Peixeiro, 2019).

Experimental Studies
This section depicts the experimental results of the proposed ASD detection system and then presents a performance comparison with similar existing methods.

Experimental Data
Clinical EEG data were obtained from the Villa Santa Maria Institute in Italy for this study (Grossi et al., 2017). The enlisted subjects' EEGs were collected while they were at rest. If the data is acquired during an activity, there is a risk of contaminating data with muscle artifacts. The ASD group consisted of thirteen boys and two girls aged between seven and twelve years, while the control group consisted of four boys and six girls aged between seven and twelve years. EEG recordings with a bandpass filter of 0.3-70 Hz were made using a Micromed device provided by System Plus Evolution software, which used pre-wired headsets with cotton elastic inside from nineteen electrodes. The international 10-20 method was used for placing the electrodes. Fifteen children and adolescents were given an objective diagnosis of ASD based on DSM-V criteria, then that was confirmed at Villa Santa Maria by a qualified child and juvenile specialist.

Experimental Setup
Anaconda software was used to implement the algorithm. Keras, Sklearn, NumPy, Pandas, Os, Pyplot, Matplotlib, and Tensor flow are among the libraries used by ResNet. To make a fair comparison with existing methods, both train-test split and cross-validation approaches are considered for performance measurement.
In the train-test protocol, 17 cases were used for training and 8 cases were used for testing. In the case of five-fold cross-validation, 20 subjects were used for training, and 5 subjects were used for testing in each fold. The tests were carried out on a PC (Intel® Core™ i7 -4720HQ CPU @ 2.60 GHz, 8 GB RAM) with Windows 10 OS.

Experimental Results and Performance Comparison
At first, EEG data transformed into 2D images is classified using generic CNN and ResNet. Figure 7 shows the classification accuracies (Y-axis) on both the training and test sets varying training epochs up to 200 (X-axis). For the generic CNN case, the training set accuracy (i.e., Train_CNN) reached 100% after 50 epochs and the test set accuracy (i.e., Test_CNN) reached up to 87.5% after 180 epochs. ResNet's training set accuracy was 100% after 25 epochs and the test set accuracy was also 100% after 160 epochs, which is promising. Accuracy on the test set is important which indicates the system's generalization ability, i.e., performance on the unseen data. The 100% accuracy on the test set by ResNet indicates the proficiency of deeper as well as skip connection architecture. It is worth mentioning that a previous study (Grossi et al., 2017) on the same data had similar findings. Table 1 compares the proficiency of the proposed ASD detection method on the test set to that of other existing methods on clinical EEG data. The table also includes the techniques used by various studies as well as the dataset size. ANN, LDA, SVM, Multi-Layer Perceptron (MLP), Layered Recurrent NN (LRN), Principal Component Analysis (PCA), and CNN were the methods considered. It is already mentioned that the data of (Grossi et al., 2017) is utilized in the present study. Other methods considered different data which varied in number. As a result, comparisons with other approaches except (Grossi et al., 2017) are not entirely justified. The proposed method, however, outperformed all others, especially the recent CNN-based method (Ali et al., 2020) which applied CNN to the raw EEG data. The proposed method achieved 100% accuracy using ResNet on PCC-based transformed EEG data, similar to the data provider's experimental results. In the case of cross-validation, the proposed method achieved an average accuracy of 88% for five-fold cross-validation. The data provider (Grossi et al., 2017) achieved 92.8% accuracy as they performed leave-one-out cross-validation. At a glance, the achieved performance of the proposed method is better or more competitive compared to the existing state-of-arts. Finally, the significance of this research lies in the advancement of a 2D representation of EEG data without mutilating the inherent properties of signals, which has proven to be a promising tool for ASD detection from EEG signals.  702

Conclusion
Early detection of ASD is unquestionably significant and it is accepted as the clinical best practice, which helps children improve their learning ability at an early stage of life. To diagnose ASD, only standardized tests are used in clinical settings that require long diagnostic time and cost. On the other hand, any ASD symptom can be confused with natural developmental variation. For these reasons, the most powerful approach for early detection of ASD is brain signal analysis as it is a neurodevelopmental disorder. EEG signal is a low-cost and relatively easy-touse technique for detecting ASD among existing several brain signal methods, so it is promising for ASD detection. This study investigated EEG-based ASD detection on clinical EEG data where EEG signals are transformed into 2D images, which are then classified using generic CNN and Resnet models individually. Achieved classification accuracy of 88% on five-fold cross-validation mode (and 100% accuracy on 20% test samples) is promising. Hence, the proposed method outperformed other existing methods and is suggested to be an effective method for ASD detection.
A possible extension of the current research would be to use larger datasets. There are other correlation analysis methods apart from PCC, which can also be investigated to compare the efficiency of PCC. There are other classification methods, which can also be explored in the future.