Improved Segmentation for Automated Seizure Detection - PowerPoint Presentation

Slide1

Improved Segmentation for Automated Seizure Detectionusing Channel Dependent Posteriors

Presented By:

Vinit Shah

Neural Engineering Data Consortium,Temple University

Slide2

1Abstract

An important factor of seizure detection problem, known as segmentation: defined as the ability to detect start and stop times within a fraction of a second, is a challenging and under-researched problem.

Accurate onset detection will allow neurologists to take actions as soon as seizure appears, and offset detection will help deliver therapies designed to mitigate post-seizure symptoms.

A multiphase model designed to mimick expert’s interpretation process by focusing on detecting epileptiform activities during the first phase and seizures associated with these activities in the second phase.We use a channel specific long short term memory (LSTM) networks in the first phase. Posteriors of which are used to create new augmented features for the second phase of the model which uses convolutional neural networks (CNNs) followed by the LSTM network.Compared to our previous baseline model, multiphase model improves seizure detection performance by 9.46% increase in sensitivity with reducing 0.97 false alarms per day. The multiphase model detects 106 seizure segment boundaries within 2 seconds of reference annotations compared to baseline system’s 9 segments.

Slide3

EEGs and Epileptic Seizures

Electroencephalography (EEG) is a commonly used means to diagnose brain-related diseases including epilepsy. EEG signals contain information in temporal and spatial domain.

EEG signals are categorized in specific frequency bands called “rhythms”.

Seizures are identified by observing evolution of abnormal epileptiform events such as Spike and wave discharges/complexes, generalized polyspikes, etc. Typical seizures show high frequency and low amplitude signals which evolve by showing progressively slowed down (lower) frequency with high amplitude signals.Such epileptiform events observed at at least 3 Hz frequency lasting for 10 seconds is considered as a seizure event (A hard decision paradigm).

Rhythm

Frequency (Hz)

Amp. Range (μv)

Activity

Delta

0-4

20-200

Deep sleep

Theta

4-7

20-100

Creativity, intuition

Alpha

8-13

20-60

Relaxation

Beta

13-30

2-20

Memory

Gamma

30-100

20-70

Cognition, learning

Slide4

The seizure events can be categorized into two forms:

Metamorphic

Isomorphic

Metamorphic events show clear phases of evolution.Isomorphic events evolves very slowly and does not show any clear onset and/or offset point.

Periodic epileptiform discharges evolving at lower frequencies are hard to confirm as seizures which requires additional clinical information about patient’s history, medications and mental status.

“Generalized spike and wave patterns slower than 3/s; and evolving discharges that remain slower than or equal to 4/s does not imply that these patterns are not ictal, but simply that they may or may not be” (American Clinical Neurophysiology Society, 2012)

Artifacts, burst attenuation/suppression and post-ictal slowing makes interpretation process further difficult.

Why Annotating Seizures is hard?

Slide5

4Seizure Segmentation & Hypothesis

Seizure segmentation remains an under-researched problem due to difficulties associated with:

(1) Noise and artifacts,

(2) Lack of standardized rules for poorly defined signal morphologies, (3) Low inter-rater agreement (IRA) among experts, (4) Lenient evaluation metricsOften, algorithms designed to solve the seizure detection problem analyze the signals (or its derived features) in a single iteration. Such models are expected to learn the traits of a seizure event without any knowledge of important components of these events such as spike and wave discharges.We propose a multiphase seizure detection model to learn the underlying components of a seizure in the first phase and seizures in the second phase.Posteriors of the first phase model are utilized to create new types of features which embed history of the detected epileptiform events. These augmented history features help the following models learn deficiencies of the first phase and further localize the detected events.

Slide6

Multilayer Perceptron, also known as fully connected network or a dense network, are the building blocks of any neural network architecture.

All the neurons of each layer are connected with every other neuron of their adjacent layers.

Where W and b are the parameters; and x & y are the input and output vectors/tensors respectively.

Each layer is encapsulated by a nonlinear function known as activation function.

 

Neural Networks (Multilayer Perceptron)

Slide7

Vanilla recurrent neural networks are the building blocks which can efficiently learn sequential information presented in input data.

There are various ways to use these networks to handle variable length input/output sequences (e.g. many to many, many to one)

For EEG seizure detection, we use RNNs as a many to one model where multiple time-steps of an EEG signal is passed to the model and binary decision is made at the output layer.

Unlike other variants of neural networks, RNNs have ability to influence its adjacent neurons.Recurrent Neural Networks

Slide8

RNNs suffer from vanishing or exploding gradient problems during backpropagation because gradients gets multiplied by identical weight matrices of the neurons.

Long short-term memory networks are designed to better handle gradient flow properties in a way that the weight matrices are able modified by the additional intermediate gates called input, ouput, forget gates.

Long Short-term Memory Networks

Slide9

8Phase-1 model (Channel based LSTM network)

Phase 1 of the model uses an LSTM network specific to learn the epileptiform activities associated with the seizure events.

We use a 3-layer LSTM network with size (256, 64, 16).

Input layer includes a 7 second window (70 frames with LFCC feature vectors) with context size of 20 frames.Training is performed in 3 stages: Exploration of error space (high learning rates)Stagnate with back-off (pleatued cross validation and training loss difference)

Annealing stage (gradual reduction in learning rate)

Slide10

9

Feature augmentation

Posteriors from the P1 model are used as additional features for the following model/s

The onset and offset marks detected by the first model is used to weight the distance from the input samples.Where n is the time-stamp from the mark and is a constant optimized for events. 

Slide11

10Convolutional Neural Networks (CNNs)

CNNs are designed to classify structured data (such as images) which are optimized by learning local correlations between the features.

CNNs are trained by optimizing a parameter matrix known as “convolutional kernels”. Outputs generated by a kernel are called feature-maps.

Convolutional channels, different from EEG channels, are able to learn a specific dimension of the data such as RGB colors in a colored image. Output of a 2D CNN network is:

 

 

 

 

CNN channels

CNN kernels and feature maps

Slide12

11Convolutional Neural Networks (CNNs)

Earlier layers of the network are designed to learn low-level properties of the image such as edges, curves and color gradients.

The kernels at the later stages learn global patterns and shapes in the data such as whether the edges or curves are associated with specific objects or properties of the image.

Slide13

12Phase 2 model (CNN + LSTM hybrid model)

During the phase 2 of the model, we learn temporal and spatial information using convolutional neural networks (CNNs) followed by LSTM networks.

This phase is designed to

mimick human interpretation process where 11 seconds of context is observed, and first CNN layer observes 3 seconds of data (lower threshold for seizures).CNN channels are assigned to learn each feature dimension independently. 192 out of 1024 total kernels are assigned to learn only posteriors and onset/offset features derived from the marks.Finally, two small LSTM networks followed by a dense layer is used to classify the seizure event.

Slide14

13CNN Feature-maps of an Easy Seizure

During the first layer, each feature dimension (channel) of the CNN network is optimized independently.

We can visualize what they learned by plotting the feature maps associated with each kernel.

Easy Brief Seizure

Slide15

14

CNN Feature-maps of an Easy Seizure

Kernels associated with conventional features (K0-K25) and posterior features (K26-K31) both show periodic patterns learned by the model.

The detected events of all kernels align well compared to the onset and offset of the seizure.

Slide16

15

CNN Feature-maps of a Subtle Seizure

Onset of a seizure can be observed at the middle of the EEG segment which evolves very slowly.

Slide17

16

CNN Feature-maps of a Subtle Seizure

Kernels associated with the conventional features (K0, K1) show very small changes in the projected feature-maps.

Posterior Kernels show a few learned patterns but are not well-aligned with the seizure onset.

Slide18

17Results on TUSZ Devtest Set (Seizure Detection)

Each phase of the model is postprocessed by heuristic postprocessing approaches such as setting thresholds for duration, probability of the event, etc.

Metric

MeasureCNN/LSTMCh. LSTM P1CNN/LSTM P2MultiPhase P3

OVLP

Sensitivity

30.83%

39.46%

41.16%

40.29%

Specificity

97.10%

95.20%

95.29%

97.56%

FAs/24 hrs

6.74

11.62

11.69

5.77

TAES

Sensitivity

11.33%

32.57%

32.87%

32.59%

Specificity

95.58%

85.48%

89.28%

90.72%

FAs/24 hrs

7.62

27.44

20.85

17.03

The multiphase model performs at 40.29% sensitivity with 5.77 FAs/24 hours on TUSZ

devtest

set.

Slide19

18

Results on TUSZ Devtest Set (Seizure Detection)

The multiphase model outperforms baseline CNN-LSTM model in the lower region of the ROC plot generated by the OVLP scoring metric.

The multiphase model outperforms baseline CNN-LSTM model throughout all the regions of the ROC plot generated by the TAES scoring metric.

Slide20

19

MetricMeasureCNN/LSTMCh. LSTM P1CNN/LSTM P2

MultiPhase P3

OVLPSensitivity33.11%36.39%42.02%42.96%Specificity

92.54%

96.55%

96.09%

95.53%

FAs/24 hrs

19.89

8.74

10.02

11.45

TAES

Sensitivity

5.01%

30.97%

30.27%

35.55%

Specificity

90.43%

93.05%

93.87%

91.79%

FAs/24 hrs

19.89

13.98

12.75

17.22

Results on TUSZ

Evalset

(Seizure Detection)

The multiphase model outperforms the baseline CNN-LSTM model where both models are able to maintain the similar level of sensitivities but with increased FA rates.

Slide21

20

20

Metric

MeasureCNN/LSTMCh. LSTM P1CNN/LSTM P2MultiPhase P3OVLPSensitivity33.71%42.32%

41.75%

43.75%

Specificity

70.72%

86.93%

85.71%

91.01%

FAs/24 hrs

40.4

14.26

16.4

9.33

TAES

Sensitivity

19.77%

36.47%

35.46%

37.13%

Specificity

45.46%

46.43%

64.57%

54.03%

FAs/24 hrs

43.75

33.03

22.45

26.92

Results on DUSZ

Evalset

(Seizure Detection)

Baseline model’s FA rate increases to 9 times the FA rate observed for the

devtest

set whereas multiphase model maintains its performance to the similar levels.

Slide22

Performance for seizure boundaries were evaluated by checking the number of event boundaries detected within the margin of 5 seconds.

P2 and P3 model consistently outperforms the baseline model.

For example, multiphase model evaluated on a blind evaluation set detects 106 segment boundaries within a 2-second margin of error.

Results (Segmentation)

Slide23

22Results (Segmentation)

Score/Duration (sec.)DevTestSens.BlindEvalSens.Duke

Sens.

0 – 1027.82%0%0%10 - 3027.31%14.28%16.44%30 - 120

53.42%

58.62%

44.46%

120 - 300

48.57%

69.23%

69.76%

300 >

51.72%

83.33%

69.23%

Score/Duration (sec.)

DevTest

Sens.

BlindEval

Sens.

Duke

Sens.

0 – 10

10.71%

9.75%

0%

10 - 30

28.49%

16.83%

15.13%

30 - 120

32.09%

45.87%

34.76%

120 - 300

56.25%

43.39%

59.30

300 >

37.93%

54.54%

53.84%

Both models show poor performance on the small duration seizures (0-30 seconds)

Baseline system show moderate performance on the longer seizure events (>300 seconds) whereas Multiphase model shows relatively higher performance.

Baseline CNN-LSTM

Multiphase Model

Poor TAES scoring performance can be linked to Multiphase model’s ability to detect longer and very brief seizures since TAES tend to penalize these events heavily.

Slide24

23Summary and Research Aims

A multiphase model is introduced that estimates epileptiform features for individual channels in the first phase and temporal-spatial context in the second phase.

Embedding the history of detected events into the second-pass model improves model’s ability to detect seizures and align its segment boundaries.

Compared to the previous baseline model, seizure detection performance improves by 9.46% increase in sensitivity with reducing 0.97 false alarms per day. Additionally, this new model design is able to detect ~35%*** of seizures within only 5 second margin of reference annotation’s onset/offset boundaries.

Slide25

24Research Aims

Task 1: Analysis of the Convolutional Neural Network kernels assigned for the augmented feature vectors.

Task 2: Fine tune the segmentation boundaries using traditional energy-based detectors and novel neural network algorithms. Compare and contrast algorithms and their pros and cons.

Task 3: Implement an offline seizure detection algorithm to perform classification on detected seizure segments to reduce the false alarm rate.Task 4: Investigate the underlying reasons about why the algorithm works well on specific seizures and fails on others. This will include correlating performance with the duration of the event and the type of seizure. Error analysis on specific sets of patients should reveal the underlying pathologies that result in degraded performance.

Slide26

25Future Directions

Published: 1 journal (1 author), 5 peer-reviewed conference papers (1 author / 3 co-author), 2 conference presentations (1 author / 1 co-author).Relevant Journal papers:Shah, V., von Weltin, E., Lopez, S., McHugh, J. R., Veloso, L., Golmohammadi, M., … Picone, J. (2018). The Temple University Hospital Seizure Detection Corpus. Frontiers in Neuroinformatics, 12, 1–6. Relevant Conference Papers:Shah, V., Golmohammadi, M., Ziyabari, S., von Weltin, E., Obeid, I., & Picone, J. (2017). Optimizing Channel Selection for Seizure Detection. In I. Obeid & J.

Picone (Eds.), Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (pp. 1–5). Philadelphia, Pennsylvania, USA: IEEE.

Shah, V., Anstotz, R., Obeid, I., & Picone, J. (2018). Adapting an Automatic Speech Recognition System to Event Classification of Electroencephalograms. In I. Obeid & J. Picone (Eds.), Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (p. 1). Philadelphia, Pennsylvania, USA. Golmohammadi, M., Ziyabari, S., Shah, V., Obeid, I., & Picone, J. (2018). Deep Architectures for Spatio-Temporal Modeling: Automated Seizure Detection in Scalp EEGs. In Proceedings of the IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 745–750). Orlando, Florida, USA. Lin, R., Marquez, D., Jacobson, M., Castaldi, H., Buckman, S., Shah, V., & Picone, J. (2020). Accuracy of Automated Machine Learning Software in Identifying EEGs with Prolonged Seizures. In Annual Meeting of the American Academy of Neurology (AAN) (p. P6.002). Toronto, Canada. Ziyabari, S., Shah, V., Obeid, I., & Picone, J. (2018). Curriculum Learning Based on Sample Selection Using Posterior Probabilities. In 26th Conference on Intelligent Systems for Molecular Biology (p. 1). Chicago, Illinois, USA.

Slide27

26Selected Publications

Golmohamedemadi, M., Ziyabari, S., Shah, V., Obeid, I., & Picone, J. (2017). Gated Recurrent Networks for Seizure Detection. In I. Obeid & J. Picone (Eds.), Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (pp. 1–5). Philadelphia, Pennsylvania, USA: IEEE. https://doi.org/10.1109/SPMB.2017.8257020von Weltin, E., Ahsan, T., Shah, V., Jamshed, D., Golmohammadi, M., Obeid, I., & Picone, J. (2017). Electroencephalographic Slowing: A Primary Source of Error in Automatic Seizure Detection. In I. Obeid & J. Picone (Eds.), Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (pp. 1–5). Philadelphia, Pennsylvania, USA: IEEE.

Future publications and publications in review:Shah, V., Campbell, C., Obeid, I., & Picone

, J. (2021). Improved Spatio-Temporal Modeling in Automated Seizure Detection using Channel-Dependent Posteriors. In International Conference on Machine Learning (ICML) . Shah, V., von Weltin, E., Ahsan, T., Obeid, I., & Picone, J. (2021). On the Use of Non-Experts for Generation of High-Quality Annotations of Seizure Events. Journal of Clinical Neurophysiology.