标签：
0.1 Topic
Notes of Lin C., Snyder L.. Principles of Parallel Programming. Beijing: China Machine Press. 2008.
(1) Parallel Computer Architecture - done 2015/5/24
(2) Parallel Abstraction
(3) Scable Algorithm Techniques
(4) PP Languages: Java(Thread), MPI(local view), ZPL(global view)
0.2 Audience
Navie PP programmers who want to gain foundamental PP concepts
0.3 Related Topics
Computer Architecture, Sequential Algorithms,
PP Programming Languages
--------------------------------------------------------------------
###1 introduction
real world cases:
house construction, manufacturing pipeline, call center
ILP(Instruction Level Parallelism)
(a+b) * (c+d)
Parallel Computing V.S. Distributed Computing
the goal of PC is to provide performance, either in terms of
processor power or memory that a single processor cannot provide;
the goal of DC is to provide convenience, including availability,
realiablity and physical distribution.
Concurrency V.S. Parallelism
CONCURRENCY is widely used in OS and DB communities to describe
exceutions that are LOGICALLY simultaneous;
PARALLELISM is typically used by the architecture and supercomputing
communities to describe executions that PHYSICALLY execute simultaneoulsy.
In either case, the codes that execute simultaneously exhibit unknown
timing characteristics.
iterative sum/pair-wise summation
parallel prefix sum
Parallelism using multiple instruction streams: thread
multithreaded solutions to count 3‘s number in an array
good parallel programs‘ characteristics:
(1) correct;
(2) good performance
(3) scalable to large number of processors
(4) portable across a wide variety to parallel platforms
###2 parallel computers
6 parallel computers
(1) Chip multiprocessors *
Intel Core Duo
AMD Dual Core Opteron
(2) Symmetric Multiprocessor Architecture
Sun Fire E25K
(3) Heterogeneous Chip Design
Cell
(4) Clusters
(5) Supercomputers
BlueGene/L
sequential computer abstraction
Random Access Machine(RAM) model, i.e. the von Neumann Model
abstract a sequential computer as a device with an instruction
execution unit and an unbounded memory.
2 abstract models of parallel computers:
(1) PRAM: parallel random access machine model
the PRAM consists of an unspecified number of instruction execution units,
connected to a single unbounded shared memory that contains both
programs and data.
(2) CTA: candidate type architecture
the CTA consists of P standard sequential computers(processors,processor element),
connected by an interconnection network(communication network);
seperate 2 types of memory references: inexpensive local reference
and expensive non-local reference;
Locality Rule:
Fast programs tend to maximize the number of local memory references, and
minimize the number of non-local memory references.
3 major communication(memory reference) mechanisms:
(1) shared memory
a natural extension of the flat memory of sequential computers.
(2) one-sided communication
a relaxation of the shared memory concepts: support a single shared address space,
all threads can reference all memory location, but it doesn‘t attempt to keep the
memory coherent.
(3) message passing
memory references are used to access local memory,
message passing is userd to access non-local memory.
### 3 reasoning about parallel performance
thread: thread-based/shared memory parallel programming
process: message passing/non-shread memory parallel proframming
latency: the amount of TIME it takes to complete a given unit of work
throughput: the amount of WORK that can be completed per unit time
## source of performance loss
(1) overhead
communication
synchronization
computation
memory
(2) non-parallelizable computation
Amdahl‘s Law: portions of a computation that are sequential will,
as parallelism is applied, dominate the execution time.
(3) idle processors
idle time is often a consequence of synchronization and communication
load imbalance: uneven distribution of work to processors
memory-bound computaion: bandwidth, lantency
(4) contention for resources
spin lock, false sharing
## parallel structure
(1) dependences
an ordering relationship between two computations
(2) granularity
the frequency of interactions among threads or processes
(3) locality
temporal locality: memory references are clustered in TIME
spatial locality: memory references are clustered by ADDRESS
## performance trade-off
sequential computation: 90/10 rule
communication V.S. Computation
Memory V.S. Parallelism
Overhead V.S. Parallelism
## measuring performance
(1) execution time/latency
(2) speedup/efficiency
(3) superliear speedup
## scable performance *
is difficult to achieve
Notes of Principles of Parallel Programming - partial
标签：
原文地址：http://www.cnblogs.com/zhoujiagen/p/4525880.html
