Document Type

Thesis

Degree Name

Master of Applied Computing

Department

Physics and Computer Science

Faculty/School

Faculty of Science

First Advisor

Azam Asilian Bidgoli

Advisor Role

Thesis Supervisor

Abstract

Deep neural networks have achieved remarkable success across a wide range of applications; however, their increasing size and computational complexity present significant challenges for deployment on resource-constrained platforms. Network pruning has emerged as an effective model compression technique for reducing computational cost while maintaining predictive performance. Nevertheless, pruning deep neural networks remains a challenging large-scale optimization problem due to the enormous number of parameters, complex structural dependencies, and inherently conflicting objectives of maximizing predictive accuracy while minimizing model complexity.

This thesis investigates deep neural network pruning through the perspective of multi-objective optimization and presents two complementary evolutionary optimization frameworks targeting both unstructured and structured pruning: Hierarchical Multi-Objective Unstructured Pruning (HMUP) framework and the Dependency-Aware Structured Pruning (DASP) framework. HMUP introduces a hierarchical two-phase multi-objective optimization framework for unstructured pruning. Rather than directly optimizing millions of binary pruning decisions, the proposed approach decomposes the optimization process into two consecutive stages. A continuous evolutionary search first performs coarse exploration of pruning thresholds to identify promising regions of the sparsity-accuracy trade-off, followed by a localized binary evolutionary refinement that operates only within the reduced search space. This hierarchical formulation substantially reduces the effective optimization dimensionality while improving the coverage of Pareto-optimal solutions.

DASP extends evolutionary pruning to structurally efficient model compression by integrating dependency based structural analysis with multi-objective optimization. Dependency Graph (DepGraph) is first employed to automatically identify coupled channel groups and estimate an initial pruning configuration that preserves architectural consistency. These learned group-level pruning ratios are subsequently used to initialize a population of candidate solutions for multi-objective evolutionary optimization, where multi-objective evolutionary algorithms refine the pruning ratios to jointly optimize classification accuracy and computational complexity measured by floating-point operations. Unlike unstructured pruning, the resulting structurally pruned networks physically remove redundant channels, enabling real reductions in computational cost and inference latency. A post-pruning fine-tuning stage is then performed to recover predictive performance.

Extensive experiments are conducted on multiple convolutional neural network architectures using the CIFAR-10 and CIFAR-100 benchmark datasets, while DASP is additionally evaluated on a vision transformer using the Tiny ImageNet dataset. HMUP demonstrates that hierarchical search-space reduction can effectively improve the exploration of high-dimensional pruning problems while generating well-distributed sparsity-accuracy Pareto fronts. DASP further demonstrates that combining dependency-aware channel grouping with evolutionary multi-objective optimization produces compact networks that achieve favorable trade-offs between predictive accuracy and computational efficiency while providing practical hardware acceleration..

Overall, this thesis demonstrates that evolutionary multi-objective optimization provides a flexible and effective framework for deep neural network compression. By addressing both fine-grained unstructured pruning and deployment-oriented structured pruning, the proposed methodologies contribute practical optimization strategies for generating efficient neural networks across a range of compression objectives.

Convocation Year

2026

Download

Available for download on Sunday, November 29, 2026

Share

COinS