Salma Afifi
M.S. FinalMay 05, 2023, 2:00 pm - 4:00 pm
ECE Conference Room
Silicon Photonic Hardware Accelerators for Transformers and Graph Neural Networks
Abstract: The rapid growth of artificial intelligence (AI) applications has revolutionized the way we process data, make decisions, and interact with machines. Specifically, artificial neural networks (ANNs) have significantly evolved and now encompass various advanced neural networks such as transformers and graph neural networks (GNNs). This has enabled the development of innovative AI applications that can transform several industries, including healthcare, recommendation systems, and robotics. Transformer and transformer-based neural networks have outperformed multiple ANNs, such as convolution neural networks (CNNs) and recurrent neural networks (RNNs), across many natural language processing (NLP) tasks. Moreover, transformers are currently being integrated into vision tasks through using the vision transformer model (ViT). Similarly, GNNs have witnessed a surge of advancements over the past few years and have established their proficiency in dealing with graph-structured data.
Nevertheless, each of these neural networks imposes unique challenges, hindering their inference and usage in resource-constrained systems. For instance, the transformer model’s size, number of parameters, and complexity of operations lead to long inference times, large memory footprint, and low computation-to-memory ratio. On the other hand, GNNs inference challenges are due to their dense and very sparse computations. Additionally, the wide variety of possible input graphs structure and algorithms dictate the need for a system capable of efficiently adapting their execution and operations to the specific graph structure and effectively scaling to extremely large graphs. Accordingly, conventional computing processors and ANN accelerators are not tailored to cater for such challenges and using them to accelerate transformers and GNN execution can be highly inefficient.
Furthermore, the utilization of traditional electronic accelerators entails a number of limitations, including escalating fabrication costs due to low yields and diminishing performance improvements, associated with semiconductor-technology scaling. This has led researchers to start investigating other technologies for ANN acceleration such as silicon photonics which enables performing complex operations in the optical domain with low energy consumption and at very high throughput. While several hardware accelerators leveraging silicon photonics have been presented for networks such as CNNs, none have been customized for emerging complex neural networks such as transformers and GNNs. Due to the various challenges associated with each of these networks, designing reliable and efficient inference hardware accelerators for transformers and GNNs is a non-trivial problem.
This thesis introduces two novel silicon-photonic based hardware architectures for energy efficient and high throughput inference acceleration. As our first contribution, we propose a non-coherent silicon photonic hardware accelerator for transformer neural networks, called TRON. We demonstrate how TRON is able to accommodate a wide range of transformer and transformer-based neural networks while surpassing GPU, CPU, TPU and several state-of-the-art transformer hardware accelerators. For GNN inference acceleration, we propose GHOST, a hardware accelerator that integrates various device-, circuit- and architecture-level optimizations which enable it to efficiently process a broad family of GNNs and real-world graph structures and sizes. When compared to multiple state-of-the-art GNNs hardware accelerators, GPUs, CPUs, and TPUs, our experiments showcase how GHOST exhibits significantly better performance and energy efficiency.
Nevertheless, each of these neural networks imposes unique challenges, hindering their inference and usage in resource-constrained systems. For instance, the transformer model’s size, number of parameters, and complexity of operations lead to long inference times, large memory footprint, and low computation-to-memory ratio. On the other hand, GNNs inference challenges are due to their dense and very sparse computations. Additionally, the wide variety of possible input graphs structure and algorithms dictate the need for a system capable of efficiently adapting their execution and operations to the specific graph structure and effectively scaling to extremely large graphs. Accordingly, conventional computing processors and ANN accelerators are not tailored to cater for such challenges and using them to accelerate transformers and GNN execution can be highly inefficient.
Furthermore, the utilization of traditional electronic accelerators entails a number of limitations, including escalating fabrication costs due to low yields and diminishing performance improvements, associated with semiconductor-technology scaling. This has led researchers to start investigating other technologies for ANN acceleration such as silicon photonics which enables performing complex operations in the optical domain with low energy consumption and at very high throughput. While several hardware accelerators leveraging silicon photonics have been presented for networks such as CNNs, none have been customized for emerging complex neural networks such as transformers and GNNs. Due to the various challenges associated with each of these networks, designing reliable and efficient inference hardware accelerators for transformers and GNNs is a non-trivial problem.
This thesis introduces two novel silicon-photonic based hardware architectures for energy efficient and high throughput inference acceleration. As our first contribution, we propose a non-coherent silicon photonic hardware accelerator for transformer neural networks, called TRON. We demonstrate how TRON is able to accommodate a wide range of transformer and transformer-based neural networks while surpassing GPU, CPU, TPU and several state-of-the-art transformer hardware accelerators. For GNN inference acceleration, we propose GHOST, a hardware accelerator that integrates various device-, circuit- and architecture-level optimizations which enable it to efficiently process a broad family of GNNs and real-world graph structures and sizes. When compared to multiple state-of-the-art GNNs hardware accelerators, GPUs, CPUs, and TPUs, our experiments showcase how GHOST exhibits significantly better performance and energy efficiency.
Adviser: Dr. Sudeep Pasricha
Co-Adviser: N/A
Non-ECE Member: Dr. Yashwant Malaiya
Member 3: Dr. Mahdi Nikdast
Addional Members: N/A
Co-Adviser: N/A
Non-ECE Member: Dr. Yashwant Malaiya
Member 3: Dr. Mahdi Nikdast
Addional Members: N/A
Publications:
1) Salma Afifi, Febin Sunny, Mahdi Nikdast and Sudeep Pasricha, “ TRON: Transformer Neural Network Acceleration with Non-Coherent Silicon Photonics”, to appear, ACM GLSVLSI, 2023.
2) Salma Afifi, Febin Sunny, Amin Shafie, Mahdi Nikdast and Sudeep Pasricha, "GHOST: A Graph Neural Network Accelerator using Silicon Photonics", IEEE/ACM CASES (ESWEEK), 2023. [SUBMITTED].
1) Salma Afifi, Febin Sunny, Mahdi Nikdast and Sudeep Pasricha, “ TRON: Transformer Neural Network Acceleration with Non-Coherent Silicon Photonics”, to appear, ACM GLSVLSI, 2023.
2) Salma Afifi, Febin Sunny, Amin Shafie, Mahdi Nikdast and Sudeep Pasricha, "GHOST: A Graph Neural Network Accelerator using Silicon Photonics", IEEE/ACM CASES (ESWEEK), 2023. [SUBMITTED].
Program of Study:
ECE 528
ECE 544
ECE 545
ECE 554
ECE 558
ECE 571/ ECE 575
ECE 699
GRAD 530
ECE 528
ECE 544
ECE 545
ECE 554
ECE 558
ECE 571/ ECE 575
ECE 699
GRAD 530