What is Data Decomposition?
• The roundRobin program was meant only to illustrate basic interprocessor communication, so we did not really focus on data. Processor 0 generated the data. Then each processor received and sent the data back and forth. • But in real-world programming, it’s never that simple! In most cases, data will have to be shared among the processors so that each cares about only its own portion of the data. “Data decomposition” refers to the sharing data between parallel processes in the best possible (i.e. most efficient) fashion. No process should step on another process’ data either. Parallelism (vs) InterProcessor Communication (IPC) Overhead It is very important to figure out how suitable an algorithm/piece of code is to parallelization; you should guard against a tendency to parallelize code without enough consideration to the InterProcessor Communication, which is very expensive on distributed memory parallel systems. In general, the following guidelines are recommended: • More parallelis
