In Reality, Reactivity, Relevance and Repeatability in Java Application Profiling we looked at the multiple realities proposed by different performance profiling solutions and performance measurement techniques. Hopefully in that article you would have seen how difficult it can be for a novice to discern which of the measurement models most closely resembles the truth though the degree of measurement overhead is a good starting point. Naturally we think that some approaches to measurement, such as ones based intelligent adaptive technologies, are far better but even then we still must be careful in how we execute the benchmark and how we configure the adaptive measurement as these can give us multiple realities even with the very same measurement solution. In this article we look at the impact of different adaptive measurement thresholds as well as the impact that slow or fast ramp up of concurrent workload has on a benchmark and its performance model.

http://www.jinspired.com/site/performance-measurement-truths-rarely-pure-and-never-simple