Benchmarking (computing)

A performance test or comparative (in English benchmark) is a technique used to measure the performance of of a system or one of its components. More formally, it can be understood that a performance test is the result of executing a computer program or a set of programs on a machine, with the aim of estimating the performance of a specific element, and being able to compare the results with similar machines. In the field of computers, a performance test could be carried out on any of its components, be it the CPU, RAM, graphics card, etc. It can also be targeted specifically at a function within a component, such as the floating point unit of the CPU, or even at other programs. The benchmarks return detailed information on all the features it has and based on this information, it can be evaluated if it is fully optimized to run or run the applications we need depending on the area.

The task of running a test was originally reduced to estimating the processing time it takes to run a program (usually measured in thousands or millions of operations per second). Over time, improvements in compilers and the wide variety of architectures and situations that exist have made this technique a specialty. Choosing the conditions under which two different systems can be compared with each other is especially difficult, and the publication of the results is often the subject of heated debates when they are released to the community.

Qualities

Performance tests have the following functionalities:

• Check if the specifications of the components are within the margin itself of the same
• Maximize performance with a given budget
• Minimize costs by maintaining a maximum level of performance
• Get the best cost-benefit ratio (with a budget or certain requirements)

Types of tests

Synthetics vs. applications

• Synthetics: they are specially designed to measure the performance of an individual component of a computer, usually taking the selected component to its maximum capacity.

Examples:

• Dhrystone
• Whetstone

• Applications: tools based on real applications, simulate a workload to measure overall team behavior.

Low vs. High Level

• Low-level test: directly measure the performance of the components. Example: the CPU clock, the DRAM times and the SRAM cache, medium access time to the hard drive, latency, time to change track, etc.

• High-level test: they are more focused on measuring the performance of the component/controller/SO combination of a specific aspect of the system, such as E/S performance with files, or the performance of a particular combination of components/controller/SO/appplication. Example: zip compression speed.

Other types of evidence

• Energy consumption
• Compliance with environmental standards, material content and end-of-life management of the product
• Heat dissipation
• Networks
• Noise reduction
• Shared work: measure performance in modern process distribution technologies
• Servers
• Technical support

Some programs to measure the performance of a piece of equipment or component are:

• Crystalmark
• Ciusbet
• Dhrystone
• iCOMP
• Linpack
• Livermore
• SPEC (SPECint and SPECfp, geared to the unit of integers and floating point, respectively)
• Whetstone