Walk-in office hours
with Dr. Ryan Johnson,
Applications Scientist

Wednesdays 10 am to noon

4. Embarrassingly Parallel vs. Tightly-Coupled Parallel Jobs

Running jobs in parallel means that you are using more than one processor or core at a time to get your work done. There are two main types of parallelism. 

“Embarrassingly parallel” (EP) means multiple copies of the same code are run with differing inputs. The inputs may only differ slightly, or they may differ greatly. When an EP job is run, copies of the code are paired with the differing inputs, and each pair is then run on a different processor. This is known as “embarrassingly parallel” due to the relative efficiency with which the entire job may be spread across multiple processors; there is little or no need for communication between processors during job execution.

Tightly-coupled parallelism involves a code designed to process a single data domain using multiple processors. The data domain is split up into subdomains. Each subdomain is sent to a different processor along with a copy of the code. The processor runs the code on the subdomain. At every time-step of the code, the boundary conditions of each subdomain are interchanged with the boundary conditions of its neighboring subdomains. Due to this data interchange, a fast internal network, such as InfiniBand,is often essential for the timely execution of tightly-coupled codes, whereas EP jobs can often run nicely on machines with modest internal networks (e.g., Gbiv E/N). Tightly-coupled parallelism also differs from EP in that it can be tricky to efficiently split up the data domain among many processors, a process known as domain decomposition.

Center for Advanced Research Computing

MSC01 1190
1601 Central Ave. NE
Albuquerque, NM 87106

p: 505.277.8249
f:  505.277.8235