SelectionOfTechniquesAndMetrics

# SelectionOfTechniquesAndMetrics - CPE 619 Selection of...

This preview shows pages 1–9. Sign up to view the full content.

CPE 619 Selection of Techniques and Metrics Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama in Huntsville http://www.ece.uah.edu/~milenka http://www.ece.uah.edu/~lacasa

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Overview One or more systems, real or hypothetical You want to evaluate their performance What technique do you choose? Analytic Modeling? Simulation? Measurement? What metrics do you use?
3 Outline Selecting an Evaluation Technique Selecting Performance Metrics Case Study Commonly Used Performance Metrics Setting Performance Requirements Case Study

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 Selecting an Evaluation Technique (1 of 4) Which life-cycle stage the system is in? Measurement only when something exists If new, analytical modeling or simulation are only options When are results needed? (often, yesterday!) Analytic modeling only choice Simulations and measurement can be same But Murphy’s Law strikes measurement more often (“If anything can go wrong, it will.”) What tools and skills are available? Maybe languages to support simulation Tools to support measurement (e.g.: packet sniffers, source code to add monitoring hooks) Skills in analytic modeling (e.g.: queuing theory)
5 Selecting an Evaluation Technique (2 of 4) Level of accuracy desired? Analytic modeling coarse (if it turns out to be accurate, even the analysts are surprised!) Simulation has more details, but may abstract key system details Measurement may sound real, but workload, configuration, etc., may still be missing Accuracy can be high to none without proper design Even with accurate data, still need to draw proper conclusions E.g.: so response time is 10.2351 with 90% confidence. So what? What does it mean ?

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 Selecting an Evaluation Technique (3 of 4) What are the alternatives? Can explore trade-offs easiest with analytic models , simulations moderate, measurement most difficult Cost? Measurement generally most expensive Analytic modeling cheapest (pencil and paper) Simulation often cheap but some tools expensive Traffic generators, network simulators
7 Selecting an Evaluation Technique (4 of 4) Saleability? Much easier to convince people with measurements Most people are skeptical of analytic modeling results since they are hard to understand Often validate with simulation before using Can use two or more techniques Validate one with another Most high-quality performance analysis papers have analytic model + simulation or measurement

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
8 Summary Table for Evaluation Technique Selection Criterion Modeling Simulation Measurement 1. Stage Any Any Prototype+ 2. Time Small Medium Varies required 3. Tools Analysts Some Instrumentation languages 4. Accuracy Low Moderate Varies 5. Trade-off Easy Moderate Difficult evaluation 6. Cost Small Medium High 7. Saleabilty Low Medium High More important Less important
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 38

SelectionOfTechniquesAndMetrics - CPE 619 Selection of...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online