Amirhossein Razavi

Amirhossein Razavi

Second-year MASc Student at Toronto Metropolitan University

Real-Time Time Series Data Prediction (Under Development)

Supervisor: N/A
Type: Personal Project (Data Engineering, Deep Learning, Real-time Data Processing)

Overview

This ongoing project is focused on utilizing Apache Flink, Kafka, and deep learning techniques to analyze and predict trends in time series data. The primary goal is to dynamically integrate live data for immediate, accurate forecasting, advancing the field of real-time data analysis.

Methodology

The methodology includes:

  1. Data Ingestion and Streaming: Implementing Apache Flink and Kafka for efficient data streaming and management.
  2. Deep Learning Model Integration: Developing and integrating deep learning models for predictive analysis of time series data.
  3. Real-time Data Processing: Utilizing Apache Flink for real-time data processing and trend analysis.
  4. Dynamic Data Integration: Ensuring the system can incorporate live data streams for immediate forecasting.

Evaluation

The evaluation criteria for the project are:

  • Model Accuracy: Assessing the predictive accuracy of deep learning models on time series data.
  • System Latency: Measuring the latency in data processing and forecasting.
  • Scalability and Efficiency: Evaluating the system’s scalability and efficiency in handling real-time data streams.

Key Findings

  • Implementation of a robust system using Apache Flink and Kafka for real-time data streaming and management.
  • Integration of deep learning models enhanced the accuracy of trend predictions in time series data.
  • Demonstrated low-latency and high-efficiency in real-time data processing.

Personal Contributions

  • System Architecture Design: Designing the overall architecture involving Apache Flink, Kafka, and deep learning models.
  • Model Development and Integration: Developing and integrating predictive models for time series analysis.
  • Performance Optimization: Optimizing the system for low latency and high-efficiency data processing.

Conclusion

This project demonstrates the effective use of Apache Flink, Kafka, and deep learning for real-time analysis and prediction of time series data, showcasing advancements in real-time data processing and predictive analytics.

Github page of this project can be viewed using this link