Dynamic Distributed Scheduling for Data Stream Computing: Balancing Task Delay and Load Efficiency

Authors
  • Xiaoxuan Sun

Abstract

In data stream computing, distributed scheduling is a key technology to improve system performance and resource utilization. Traditional static and rule-based scheduling methods are difficult to adapt to task fluctuations and resource changes in dynamic environments, often resulting in high task delays and resource waste. To solve this problem, this paper proposes a distributed scheduling algorithm based on global optimization, which aims to achieve efficient task allocation and resource utilization by balancing task delay and load balancing. By introducing real data sets and multiple experimental scenarios, this paper conducts a comprehensive evaluation of the algorithm. The experimental results show that the proposed algorithm shows significant advantages under low, medium, and high loads and node failures, especially under high loads and high failure rates. The improvement in task delay and load balancing is particularly obvious. In addition, this paper designs robustness experiments for task bursts and node failures to verify the algorithm's adaptability and resource reallocation efficiency in dynamic scenarios. Although the algorithm performs well under various conditions, there is still room for further improvement. Future work will explore its adaptive ability in ultra-high loads and heterogeneous computing environments. This study provides new optimization ideas for distributed scheduling in data stream computing, which will help promote the efficient and intelligent development of distributed systems.

References
Cover Image
Downloads
Published
2025-01-30
Section
Articles
License
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Academic Scholar Hub Press also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.

How to Cite

Sun, X. (2025). Dynamic Distributed Scheduling for Data Stream Computing: Balancing Task Delay and Load Efficiency. Journal of Computer Technology and Software, 4(1). https://doi.org/10.5281/zenodo.14785261