for Motivation Examining Performance
Hardware performance is often key to the effectiveness of an entire system of hardware and software Why certain piece of software performs the way it does? Why one instruction set can be implemented to perform better than another?
CIT 595 Spring 2008
Computer Performance
CIT 595
2
What defines Performance?
If you are running a program on 2 different workstations
Then faster one is the one that gets the job done first As a user you are interested reducing the response time
i.e. time from start to completion of task (a.k.a. execution time)
Measuring Response Time
Response Time is one of the measure of computer performance So how do we measure the time? We could just time from started our program till the time it was done executing (i.e. elapsed or response time) However modern computers are often timeshared and may simultaneously work on several programs
So we want distinguish between the time the processor is working on our behalf vs. the time spent waiting
3 CIT 595 4
If you are running a computer center with 2 timeshared computers running jobs submitted by many users
Then faster computer is the one that completed most jobs during the day As system administrator you are interested increasing throughput
i.e. the total amount of work done in a given time
CIT 595
1
Execution Time or Latency Response Time = CPU Execution Time + Wait Time CPU (processor) execution time i.e. time spent computing for a particular program
Not running other programs or waiting on I/O Also called as Latency
CPU Execution Time or Latency
CPU execution time of program depends on:
Dependent on the total number of instructions executed Each instruction performance is dependent on the number of clock cycles it takes to complete the instruction cycle (Fetch thru Store) called as cycles per instruction (CPI) One clock cycle is time interval with clock ticks i.e. clock cycle time time program time = cycle cycles instruction CPI instructions program Instruction Count
CPU Time =
x
x
Clock Cycle time
CIT 595
5
CIT 595
6
CPU Execution Time (contd..) Clock cycle time = 1/Clock Rate or 1/Frequency Clock rate is given in Hertz (Hz) = cycles/sec
Cycles Per Instruction (CPI) CPI is an average of all the instructions executed in the program Depends on
Instruction Count x CPI CPU Time = Clock Rate
Instruction Set Architecture (ISA) Design details (micro-architecture)
CPI varies among different implementations of the same ISA
CPI is obtained by performing a detailed simulation of an implementation
CIT 595 7 CIT 595 8
2
Example: Calculating CPI
Given:
# Instructions & Clock Cycle Time Number of Instructions depends on:
Instruction Load Store ALU Branch/Jump Others
# cycles 5 5 5 3 6
% Instr. Count 25 15 40 15 5
Algorithm Compiler ISA
Clock Cycle Time depends on:
Hardware Technology (transistor technology) Design (Micro-architecture) Simulate the longest propagation path of your circuit
The average CPI is = 0.25 x 5 + 0.15 x 5 + 0.40 x 5 + 0.15 x 3 + 0.05 x 5 = 4.75 cycles
CIT 595 9 CIT 595
10
Maximizing Performance in relation to Execution Time
One, way to maximize performance, we want to minimize execution time Thus we can relate performance and execution time for a machine X as: PerformanceX = 1/ Execution TimeX For low latency we want to minimize all 3 factors CPI, # instructions and Clock cycle time
This is difficult as trying to decrease one often leads to increasing the other parameter E.g. Two ISA Philosophies
RISC low CPI/clock period, high inst count CISC low insn count, high CPI/clock period
CIT 595 11
Performance Ratio (Comparing Performance) To claim that Machine X performs better than Machine Y:
PerformanceX > PerformanceY i.e. 1 > Execution TimeX Performancex Performance Ratio = PerformanceY = Execution TimeX Execution TimeY Execution TimeY 1
CIT 595
12
3
Relative Performance Example
A Clock Cyle Time (ns) CPI 1 2.0 B 2 1.2
Misleading Metrics Clock
Which is faster?
ClockA = 5 GHz, ClockB = 3 GHz
2 implementations of the same ISA Which machine (A or B) is faster and by how much? Let I be the number instructions per program
Ex TimeA = 1 x 2.0 x I = 2 x I ns Ex TimeB = 2 x 1.2 x I = 2.4 x I ns
Probably A, but make this assumption only if the same ISA & compiler
If stated that CPIA = 2 and CPIB = 1, then which is faster?
MIPS Millions of instructions per second
instr/sec = 1/[(cycles/instr) * (seconds/cycle)]
2.4 x I ns
Performance Ratio =
TimeB TimeA
=
2.0 x I ns
=
1.2
Higher the MIPS, faster the machine
Different ISAs/Compilers have incomparable MIPS
Machine A is 1.2 times than faster B
CIT 595 13 CIT 595 14
Maximizing Performance in relation to Throughput
Throughput is amount of work that can be done in a given time Measured in tasks per time unit
E.g. x tasks per hour
Evaluating Performance
Step I Choose a workload
Workload: set of programs whose performance you care about
Use standardized workload (known as benchmarks)
Step II Decide the metric Execution Time or Throughput Step III - Evaluate data gathered using statistical analysis Statistical method use depends on
Metric Distribution of the data
So higher the throughput, better the performance Also, Throughput and Response Time are inversely related
If system carries out a task in k seconds then its throughput is 1/k tasks per second
Throughput can be increased:
Pipelining at instruction level (chp 5) Timesharing a computer system (chp 8 - OS) Achieving Parallelism (chp 9)
Superscalar Processor Use multiprocessors (Shared Memory, Distributed, Chip Multiprocessors)
CIT 595 15
CIT 595
16
4
Standard Performance Evaluation Corporation (SPEC)
Is consortium that collects, distributes and standardizes benchmarks Produces benchmark suites for various classes of CPU, Java, I/O, Web, Multithreaded etc. E.g. CPU 2006 consists of 29 CPU intensive C/C++, Fortran programs
integer: perl, gcc,bzip2(compression),sjeng(AI: chess) floating point: povray(ray tracing), wrf (weather prediction),sphynx3 (speech recognition)
Statistical Analysis: Arithmetic Mean
Arithmetic Mean =
Used for units that proportional to time The arithmetic mean can be misleading if the data are skewed or scattered
Like SPEC, there is Transaction Processing Council (TPC) Used for web/database server workloads Programs are I/O or network intensive rather than CPU intensive
CIT 595 17 CIT 595 18
11.3 Mathematical Preliminaries Represented as:
Geometric Mean
Geometric Mean Example
Unlike an arithmetic mean, tends to dampen the effect of skew Used for unit less quantities e.g. performance ratio/speedup
Performance results are stated in relation to the performance of a common machine used as reference
CIT 595 19
Ratio = Ex (Machine Reference)/ Ex(Your Machine) Geometric Mean for System A normalized with respect to System B: = (100/50 x 400/200 x 500/250 x 800/400 x 4100/5000)1/5 = 1.6733
CIT 595 20
5
Geometric Mean Consistency
The results that we got when using System B and System C as reference machines are given below We find that Geometric Mean of A to Geometric Mean of B to be
Harmonic Mean
Used to units that inversely proportional to time i.e. throughput Unlike latencies, throughput cannot be added
E.g. 1st - 10 mile @ 30 mph, 2nd - 10 miles @ 40 mph and last 10 miles @ 60 mph
Average is not 43.3 mph
consistent regardless which system is taken as reference
1.6733/1...