This project involves exploring cutting-edge techniques in distributed stream processing, including partitioning strategies, dynamic scheduling, and performance modeling. Students will develop or extend existing frameworks to handle massive and high-velocity data streams.
Focus
Designing and evaluating novel algorithms and system architectures for real-time data processing in large-scale, heterogeneous environments (e.g., cloud-edge networks). Emphasis on resource allocation, parallelism, and load balancing to achieve high throughput and low latency.
Tasks
- Dynamic Resource Allocation: Create adaptive load balancing and scheduling strategies.
- Scalability Mechanisms: Explore partitioning, parallelism, and distributed execution techniques.
- Performance Evaluation: Develop benchmarks and simulation frameworks for empirical validation.
Required Skills
Theoretical Skills:
- Distributed Systems Concepts
- Scalability and Performance Modeling
- Queueing Theory / Scheduling Algorithms
Practical Skills:
- Programming in Java, Scala, or Python
- Familiarity with Apache Spark, Flink, or Storm
- Cloud Deployment (AWS/Azure)
