Survey On Improving Straggler Tasks In Hadoop

. Hadoop is a well-known parallel computing system for processing massive amounts of data, but there is a task in Hadoop called "Straggling task" that has a significant impact on Hadoop. Speculative Execution (SE) is a good technique to deal with "Straggling tasks" since it monitors the rate of running processes in real time. backing up the "Straggler" on another node to increase the opportunity of completing the backup task ahead of the original. This proposal tackles dealing with the "Straggling task" by creating a strategy able to deal with misjudgment, improper selection of backup nodes, and making speculative tasks start from the checkpoint, by leveraging the checkpoint of the original tasks. BY this work we can achieve comparing with common methods in this sector, such as LATE, ESAMR, and the real remaining time for Wordcount and Sort benchmarks, it was demonstrated to be capable of detecting straggler tasks and properly estimating execution time. It also allows for job execution to be sped up.