High Performance and Distributed Computing

High performance computing (HPC) most generally refers to the practice of aggregating computing power in a way that delivers much higher performance than we could get out of a typical desktop computer or workstation in order to solve large problems in science, engineering or business. This chapter provides an overview of the most common processor options used today for HPC. HPC has seen in recent years an increasingly large role played by general purpose graphics processor unit (GP-GPU or GPU). The chapter analyzes the architecture of most common CPUs, and GPUs, and of programming frameworks required to develop codes for these systems. The typical HPC system for scientific computing today is a large cluster of nodes based on commodity multi-core CPUs and interconnected by a high-speed switched network, such as InfiniBand. NVIDIA GPUs are multi-core processors with each core capable of processing several operations in parallel.