Whats the best way to implement a distributed data queue for processing across a Windows Server cluster?
I am looking to implement a Windows Server-based cluster for crunching through large volumes of homogenous processing (i.e. multiple instances of the same function, only with each running over different a different data set) which had previously been performed on a single box. The model I would like to operate is one where the “head” node int he cluster maintains a queue of data waiting to be processed. Each individual processing node iteratively obtains a data item from this central queue, processes it, obtains another data item from the queue, processes it, etc. In this way work is distibuted dynamically across the cluster so that overall the cluster is balanced. The question is: how best to implement the distributed data queue machanism? My current leaning is towards holding the data queue in a database table on the head node with a listener sitting next to it managing the retrieval of data items and their distribution. Each processing node would connect to the listener to retrieve
Related Questions
- I am backing up a cluster of machines (clients) to a backup server (server). Whats the best way to pull data from each machine in the cluster?
- Whats the best way to implement a distributed data queue for processing across a Windows Server cluster?
- Why my data on hard disk cannot be protected under Windows NT/2000/XP/Server 2003?
