ISO/IEC 25023: ISO/IEC 25023 – Tutorial (about 60 minutes)

ISO/IEC 25023 is the international standard that turns the software product quality model into numbers. Where ISO/IEC 25010 tells you which quality characteristics matter (reliability, security, maintainability and so on), ISO/IEC 25023 gives you a catalogue of quality measures, each with a defined formula and defined inputs, so that you can state quality requirements precisely and check whether a system meets them. Testers, quality engineers, architects, acquirers, suppliers and independent evaluators use it. After this tutorial you will be able to explain how the standard is organised, choose and tailor measures for a real product, set targets, collect the data behind each measure and interpret the results honestly.

Free tutorial · about 60 minutes · 7 lessons · 5 diagrams · then a 25-question knowledge check and a certificate.

Course overview

Edition and status: The current published edition is ISO/IEC 25023:2016, the first edition, published in June 2016 by ISO and IEC through Joint Technical Committee ISO/IEC JTC 1, Subcommittee SC 7 (Software and systems engineering). Its full title is "Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of system and software product quality". It cancelled and replaced the technical reports ISO/IEC TR 9126-2:2003 (external metrics) and ISO/IEC TR 9126-3:2003 (internal metrics). The 2016 edition was confirmed at a systematic review in 2022, and in February 2024 SC 7 decided that it should be revised. A replacement is now well advanced: it is numbered ISO/IEC 25000-23, "Part 23: Measurement of product quality", and in September 2026 its second Draft International Standard (DIS) ballot was opened. ISO's catalogue states that the 2016 edition is expected to be replaced by this draft within the coming months. Until the new part is published, ISO/IEC 25023:2016 remains the valid edition, and that is the edition this course teaches. Where the revision is likely to matter, the tutorial says so. Verified 2026-10-29 against the ISO catalogue and committee pages for ISO/IEC 25023:2016 and ISO/IEC DIS 25000-23.2, and the publisher's preview of the 2016 edition (foreword, introduction, scope, conformance and table of contents).

A note on copyright: the standard is a paid, copyrighted document. This tutorial explains it in its own words and uses the names of characteristics, subcharacteristics and a few well-known measures, which are widely published. It does not reproduce the measure tables or formulas word for word. If you need exact definitions for a contract or a formal evaluation, buy the standard from ISO, IEC or your national standards body.

Who it is for: test managers and testers, quality assurance and quality engineering staff, software architects, product owners and business analysts who write non-functional requirements, acquirers and vendor managers, and evaluators who must give an evidence-based opinion on a software product. No earlier contact with ISO standards is assumed.

Prerequisites: none. It helps to know the idea of a product quality model (ISO/IEC 25010). If you do not, Lesson 1 gives you enough to follow the rest.

Learning objectives. After this course you will be able to:

Explain why ISO/IEC 25023 exists, what it covers and what it deliberately leaves out.
Use the core vocabulary correctly: quality measure, quality measure element, measurement function, internal and external measure, generic and specific measure.
Describe the structure of the standard and find the measures for any characteristic and subcharacteristic.
Select, tailor and document a set of measures for a product, with targets and a rationale for every exclusion.
Collect the data behind a measure, compute it, and interpret the result against agreed targets.
Explain what a conformance claim to ISO/IEC 25023 means and how measures feed product evaluations.
Place ISO/IEC 25023 among the SQuaRE standards and related measurement standards.

Lesson 1 – Origins, purpose and scope

The problem it solves

Most projects agree that their software should be "fast", "reliable" and "secure". Very few agree on what those words mean in a way that can be tested. One team calls a system reliable because it rarely crashes during demos; another means that it stays available for 99.9 % of scheduled hours; a third means that defects found in the field are fixed within a week. When each party carries a different meaning, contracts become arguments, test reports cannot be compared from one release to the next, and decisions about whether to release are made on gut feeling.

A quality model solves the first half of this problem by naming the qualities and splitting each into smaller, more precise pieces. ISO/IEC 25010 is that model for software and systems. But a model alone is still qualitative. You can say "time behaviour is important for us", but not yet "our time behaviour is good enough". ISO/IEC 25023 solves the second half: for each part of the product quality model it proposes measures with a defined way of calculating them, so that a quality can be expressed as a value, compared with a target, and tracked over time.

Where it comes from

The ideas are older than the current number. The ISO/IEC 9126 series, first published in 1991 and expanded in the early 2000s, combined a quality model (9126-1) with three technical reports of example metrics: external metrics (9126-2), internal metrics (9126-3) and quality-in-use metrics (9126-4). Practitioners found the metric reports useful but bulky; many metrics were duplicated between the internal and external lists, and some were rarely usable in practice.

From 2005 onward, ISO/IEC JTC 1/SC 7 reorganised this material into the SQuaRE series, the ISO/IEC 25000 family. The product quality model moved to ISO/IEC 25010 in 2011. The measures followed in 2016: ISO/IEC 25023 took over the internal and external metrics of 9126-2 and 9126-3, and ISO/IEC 25022 took over the quality-in-use metrics. During that move the committee reviewed every old metric and kept or dropped it according to its practical usefulness, added measures for the parts of the model that were new in 2011 (for example security as its own characteristic and compatibility), and merged the internal and external variants into a single table per subcharacteristic. The word "metric" was also replaced by "measure", in line with general measurement vocabulary.

Scope in plain words

The standard does three things. It defines a basic set of quality measures for each characteristic and subcharacteristic of the product quality model. It explains how such measures should be applied. And in informative annexes it adds practical considerations, the list of data items (quality measure elements) that the measures are built from, and a more detailed explanation of measurement types.

It also states clearly what it does not do. It does not tell you what value of a measure is good, acceptable or excellent. It assigns no rating levels and no grades of compliance, because a good value depends on the kind of software, its integrity level and what its users need. A response time that is excellent for a monthly report is unacceptable for a payment terminal. Targets are your job, usually through the quality requirements process of ISO/IEC 25030.

The intended users are named by activity rather than by job title: people who specify and evaluate quality during development (from requirements to acceptance), in quality management (assurance, control and certification work), in supply (proving quality at qualification test), in acquisition (selecting and accepting products) and in maintenance (improving a product using measurement). The measures are mainly meant for quality assurance and improvement during and after development.

Why the 2011 model matters

One detail is easy to miss and matters a lot today. ISO/IEC 25023:2016 is built on the 2011 edition of the product quality model, which it cites as a dated normative reference. That model has eight characteristics and 31 subcharacteristics. ISO/IEC 25010 was revised in 2023 and now has nine characteristics, with safety added and several names and groupings changed (for example usability became interaction capability, portability became flexibility, and maturity became faultlessness). So if your organisation has adopted the 2023 model, you will need a small mapping when you use the 2016 measures. The draft revision states its measures in terms of ISO/IEC 25010, whose current edition is the 2023 one, so the revised catalogue can be expected to follow the newer model; confirm the details once it is published.

Lesson 2 – Key concepts and vocabulary

Measurement standards are precise about words, because a sloppy word produces a sloppy number. The terms below come from ISO/IEC 25023 itself, from ISO/IEC 25000 (the SQuaRE guide and glossary) and from ISO/IEC 25021 and 25020. They are explained here in plain language, not quoted.

SQuaRE – Systems and software Quality Requirements and Evaluation: the ISO/IEC 25000 family of standards for quality models, measurement, requirements and evaluation.
Quality characteristic – a broad category of product quality, such as reliability or security, defined in the quality model (ISO/IEC 25010).
Quality subcharacteristic – a narrower part of a characteristic, such as availability within reliability. ISO/IEC 25023 organises its measures by subcharacteristic.
Property to quantify – a concrete, observable property of the product, its documentation or its behaviour that a measurement method can be applied to, for example the number of functions specified or the time taken to answer a request.
Quality measure element (QME) – a basic measured value, obtained by applying a measurement method to a property, that serves as an input to quality measures. "Number of failures observed" and "operation time" are QMEs. ISO/IEC 25021 defines how to specify them.
Measurement method – the logical sequence of operations used to obtain a QME, for example counting functions in a specification or reading response times from a load test tool.
Measurement function – the formula or algorithm that combines two or more QMEs into the value of a quality measure. Many in ISO/IEC 25023 are ratios.
Quality measure – a measure, defined as a measurement function of QMEs, that gives a value for a subcharacteristic, for example system availability or mean response time.
Internal measure – a measure of static attributes of the product, such as design documents or source code, obtained without executing the system.
External measure – a measure of how the system behaves when it runs, taken during testing or operation.
Quality in use measure – a measure of the outcomes users achieve in a real context of use (effectiveness, efficiency, satisfaction and so on). These are in ISO/IEC 25022, not 25023.
Generic measure (G) – a measure the standard expects to be used whenever its subcharacteristic is selected, unless you record a reason for leaving it out.
Specific measure (S) – an optional measure for particular situations, technologies or needs; you select it when it is relevant.
Target value – the value, or range of values, that a measure must reach for the requirement to be met. The standard supplies none; you set them.
Target entity – the thing being measured: a whole system, a software product, a component or an intermediate work product such as a design.

Internal and external are one family

In the old 9126 reports, internal and external metrics were separate lists. ISO/IEC 25023 presents one table per subcharacteristic, and the same idea can often be measured on a design or code (internal) and later on the running system (external). The two views are related but not identical. A classic illustration is fault and failure counting: counting faults in reviewed code is not the same as counting failures in tests, because tests never find every fault and one fault can produce several different failures. Internal measures are useful early, as predictors; external measures confirm what the running product really does.

Figure 1 – How a quality measure is built: measurement methods applied to properties give QMEs, and a measurement function combines them into a value for a subcharacteristic.

Figure 1 shows the measurement reference model that ISO/IEC 25023 relies on (the model itself is described in ISO/IEC 25020). Start at the top and follow the arrows down. You can observe properties of the product. A measurement method turns a property into a number, the QME. A measurement function combines QMEs into a quality measure. The measure tells you something about one subcharacteristic, which in turn is part of one characteristic. Every value in a quality report should be traceable down this chain; if you cannot say how a QME was obtained, the measure built on it cannot be trusted.

Lesson 3 – Structure of the standard

The clauses

The 2016 edition is short, about 45 pages, and most of it is the measure catalogue. Its clauses are:

Clause 1, Scope – what the standard covers and the activities it supports.
Clause 2, Conformance – what a requirement specification or evaluation must do to claim conformance (explained in Lesson 5).
Clause 3, Normative references – ISO/IEC 25000, ISO/IEC 25010:2011 and ISO/IEC 25021:2012.
Clause 4, Terms and definitions, and Clause 5, Abbreviated terms.
Clause 6, Use of system and software product quality measures – the measurement concepts and the overall approach to quality measurement.
Clause 7, Format used for documenting the quality measures – how every measure entry is laid out.
Clause 8, System and software product quality measures – the catalogue, one subclause per characteristic (8.2 to 8.9) and one sub-subclause per subcharacteristic.
Annex A (informative) – considerations for using quality measures in practice.
Annex B (informative) – the QMEs used to define the measures.
Annex C (informative) – a more detailed explanation of measurement types, such as measures based on counts, sizes and time.

Figure 2 – The structure of ISO/IEC 25023:2016, read top to bottom in document order: general clauses with guidance on use and format, the catalogue of measures by characteristic, and three informative annexes.

The catalogue follows the 2011 model

Clause 8 mirrors the eight characteristics of ISO/IEC 25010:2011, and inside each one the subcharacteristics of that model:

Clause	Characteristic	Subcharacteristics with measures
8.2	Functional suitability	Functional completeness, functional correctness, functional appropriateness
8.3	Performance efficiency	Time behaviour, resource utilization, capacity
8.4	Compatibility	Co-existence, interoperability
8.5	Usability	Appropriateness recognizability, learnability, operability, user error protection, user interface aesthetics, accessibility
8.6	Reliability	Maturity, availability, fault tolerance, recoverability
8.7	Security	Confidentiality, integrity, non-repudiation, accountability, authenticity
8.8	Maintainability	Modularity, reusability, analysability, modifiability, testability
8.9	Portability	Adaptability, installability, replaceability

This gives 31 subcharacteristics, each with a handful of measures. You do not read the catalogue cover to cover. You go to the subcharacteristics your quality requirements care about and pick from there.

What a measure entry contains

Every measure is documented in the same format, described in clause 7. In practice each entry tells you:

an identifier and a short name, so that the measure can be referred to unambiguously in requirements and reports;
whether it is a generic (G) or a specific (S) measure;
a description, usually phrased as the question the measure answers, such as "what proportion of the specified functions is implemented?";
the measurement function, with each input (QME) defined.

Many functions are ratios of the form "A divided by B", or "1 minus A divided by B", designed so that the result lies between 0 and 1 and a value closer to 1 is better. Others are averages or totals with physical units, such as seconds or transactions per minute. Because the direction and the unit differ from measure to measure, always read how a value should be interpreted before you set a target.

A tour of example measures

A few widely cited measures illustrate the style. They are described here in our own words.

Functional completeness: functional coverage looks at the functions that were specified and asks how many of them are missing from the implementation, giving a ratio near 1 when nearly everything is present.
Time behaviour: mean response time averages the measured time the system takes to respond to a user task or request; response time adequacy compares the measured mean with the specified target.
Capacity: transaction processing capacity asks how many transactions can be handled per unit of time; user access capacity asks how many users can be served at the same time.
Availability: system availability compares the time the system was actually in operation with the time it was scheduled to be; mean down time averages how long the system stays unavailable after a failure.
Maturity: mean time between failure and failure rate describe how often the system fails during a defined period of testing or operation.
Fault tolerance: failure avoidance asks what proportion of injected or observed fault patterns were brought under control without a failure, and redundancy of components looks at how many components are installed redundantly.
Accountability: user audit trail completeness asks how many of the actions that should be recorded actually appear in the audit trail.
Usability: error messages understandability and avoidance of user operation error look at whether messages explain the problem and whether the design prevents wrong operations.

The catalogue is deliberately not exhaustive. The introduction encourages users to refine the measures and, where needed, to define new ones using the QME format of ISO/IEC 25021.

Lesson 4 – How it works in practice

Measurement is a process, not a spreadsheet

Picking formulas is the easy part. Value comes from embedding measurement in a repeatable process: know why you measure, decide what to measure, measure it the same way every time, and act on the result. ISO/IEC 25023 sits inside that process. It is used together with ISO/IEC 25010 (what quality means), ISO/IEC 25030 (how to state quality requirements), ISO/IEC 25040 and 25041 (how to run an evaluation) and, more generally, the measurement process of ISO/IEC/IEEE 15939.

Figure 3 – Applying ISO/IEC 25023 measures: plan what and why to measure, then collect, compute, compare and act, and repeat for each release.

Step by step

1. Set the purpose and the target entity. Are you writing requirements for a new system, accepting a supplier's delivery, comparing two products before purchase, or tracking quality release by release? The purpose decides the precision you need and whether you can use internal measures (you need the code or design), external measures (you need a running system) or both. Name the entity exactly: the whole system, one service, or a specific build.

2. Select characteristics and subcharacteristics. Start from stakeholder needs and risks, not from the catalogue. A booking site under seasonal peaks cares about capacity and availability; a medical device data tool cares about integrity and functional correctness; a product line that will live for ten years cares about modifiability and testability. Record why each subcharacteristic is in or out.

3. Choose measures. For every selected subcharacteristic, take all of its generic measures by default. Drop one only with a written reason, for example "no user interface, so the operability measures do not apply to this batch component". Add specific measures where they fit your technology or risk. If no measure fits a need, define your own and document its QMEs.

4. Set target values and decision rules. For each measure, agree before measuring what value is required, what is acceptable with a concession and what blocks release. Write down which values are "must pass" and how several results are combined into a decision. Targets that are negotiated after the data arrive tend to move to wherever the data happen to be.

5. Define measurement methods and collect QMEs. For each input, say exactly where it comes from: which tool, which log, which test set, which time window, who counts and how. Decide what counts as a "function", a "failure" or a "user error" in your context, and keep that definition stable. Automate collection where you can; manual counting is slow and drifts between people.

6. Compute values. Apply the measurement function. Keep the raw QMEs, not only the final ratio, so that someone can check the calculation and so that you can recalculate if a definition changes.

7. Compare with targets. Interpret each value in its direction and unit. A ratio of 0.97 for functional coverage means three missing functions in a hundred; whether that is acceptable depends on which functions are missing.

8. Decide, report and improve. Report values with their targets, the measurement conditions and any limitations. Use the result: release, fix, renegotiate, or improve the process that produced the weakness. Then repeat with the next release, keeping methods stable so that the trend means something.

Roles and artefacts

The standard does not prescribe an organisation, but in practice a few roles recur. A requirements owner (product owner, business analyst or acquirer) decides which qualities matter and agrees targets. A measurement owner, often the test or quality lead, designs methods and keeps definitions stable. Testers, operations staff and tools produce the raw data. An evaluator, internal or independent, computes and interprets the results. Developers and architects use internal measures early to predict and steer.

The typical artefacts are a quality requirements specification that names measures and targets, a measurement plan that defines QMEs and methods, the collected data, and an evaluation or quality report. If you work in an agile team, the same content can live in a "definition of done", in non-functional acceptance criteria on backlog items and in a dashboard fed by the pipeline. What matters is that every number on the dashboard can be traced to its measure, function and data source.

Internal measures early, external measures later

The life cycle shapes what you can measure. In design and coding you cannot yet run load tests, but you can measure static properties: for example how coupled the modules are, how far the code follows agreed coding rules, or how many of the required functions are present in the design. During testing you can take external measures of behaviour under controlled conditions. In operation you can measure availability, mean down time and failure rates from real monitoring data. A mature programme uses internal measures as early warnings and checks them against external results later, to learn which internal signals actually predict trouble in your context.

Practical measurement quality

Annex A of the standard collects practical considerations for using measures. In the same spirit, here are a few questions you should be able to answer for every measure you use. Is it valid, meaning that it actually reflects the subcharacteristic you care about? Is it repeatable, so that the same person measuring the same product twice gets the same result, and reproducible, so that different people get the same result? Is the scale understood: can you meaningfully average it, or only rank it? Is the cost of collecting it justified by the decisions it supports? A measure that fails these questions is noise, however precise its formula looks.

Lesson 5 – Assessment, conformance and compliance

There is no ISO/IEC 25023 certificate

ISO/IEC 25023 is not a management system standard and there is no accredited certification scheme in which an organisation or a product becomes "ISO/IEC 25023 certified". Be cautious with any marketing that claims otherwise. The standard is a toolbox of measures. Its value appears when it is used inside requirements, contracts, test programmes and product evaluations.

What a conformance claim means

The conformance clause applies to two kinds of work product: a quality requirement specification and a quality evaluation. Either may claim conformance if, paraphrasing the clause, it:

selects the characteristics or subcharacteristics to be specified or evaluated from ISO/IEC 25010;
uses, for each selected subcharacteristic, all its generic measures from clause 8, giving a rationale for any that are excluded;
optionally adds any relevant specific measures;
gives a rationale whenever a measure from the standard is modified;
defines any additional measures and QMEs it needs, in the format of ISO/IEC 25021.

Notice what is not required. You do not have to measure every characteristic. You do not have to reach any particular value. You do have to be explicit and transparent about your choices. A reviewer checking a conformance claim therefore looks for traceability from each selected subcharacteristic to its generic measures, for documented reasons behind exclusions and modifications, and for proper definitions of any home-made measures.

Figure 4 – Tailoring a measure set so that a requirements specification or evaluation can claim conformance: generic by default, documented exclusions, optional specific measures and properly defined additions.

Who assesses, and how

Three kinds of party use the measures to form a judgement, matching the perspectives of ISO/IEC 25041: developers evaluating their own product, acquirers evaluating products they intend to buy or accept, and independent evaluators such as test laboratories working for either side or for a regulator. The evaluation process itself is covered by ISO/IEC 25040, whose 2024 edition is titled "Quality evaluation framework". Its first edition (2011) set out the classic sequence: establish the evaluation requirements, specify the evaluation (including measures and decision criteria), design it, execute it, and conclude it with a report. ISO/IEC 25023 supplies the measures used when the evaluation is specified and executed.

Some product certification and labelling schemes around the world test software against SQuaRE standards, most often ISO/IEC 25051 for ready-to-use software products, and they may use ISO/IEC 25023 measures in their test methods. The certificate in such a case belongs to that scheme and its rules, not to ISO/IEC 25023 itself. Check the scheme's own documentation before relying on it.

Using measures in contracts

Measures become powerful when they appear in acceptance criteria. "The system shall be highly available" is untestable. "System availability, as defined in ISO/IEC 25023 and measured from the monitoring tool over each calendar month, shall be at least 0.995 of the scheduled service hours" is testable, and both parties know how the number will be produced. Good contracts also fix the measurement conditions (test environment, data volume, load profile), the evidence to be handed over, and what happens when a value misses its target.

Honest interpretation

An assessor should report not just values but their limitations. How large was the test sample? Were the conditions representative? Were any QMEs estimated rather than measured? Was a generic measure excluded, and why? A score without context invites misuse, for example a manager comparing the "maintainability number" of two unrelated systems that were measured in different ways.

Lesson 6 – Worked example

The situation

RouteWell is a mid-sized logistics company. It is replacing its ageing dispatch system with a new web and mobile platform built by an external supplier. The platform assigns deliveries to drivers, shows live parcel status to customers and feeds invoicing. The previous system failed during the year-end peak, and the contract with the supplier now has to state quality in measurable terms. RouteWell's quality lead, Amara, uses ISO/IEC 25010 and ISO/IEC 25023 to do this. The numbers below are illustrative, not taken from the standard.

Step 1: purpose and target entity

Amara writes down two purposes: to specify quality requirements in the contract, and to evaluate each delivery at acceptance. The target entity is the dispatch platform (back-end services, web console and driver app), excluding the existing invoicing system it connects to. Because RouteWell will receive the source code at each milestone, both internal and external measures are possible.

Step 2: select subcharacteristics

In a workshop with dispatchers, operations, security and finance, the team ranks risks. The year-end failure makes capacity, time behaviour and availability top priorities. Customers and drivers must not see other people's parcels, so confidentiality and accountability matter. The platform will be extended for years, so modifiability and testability are included. Interoperability with the invoicing system is critical. User interface aesthetics and installability are judged low risk and recorded as out of scope, with the reason "covered by the supplier's design system; installation is via app stores and automated deployment".

Step 3: choose and tailor measures

For each selected subcharacteristic Amara lists every generic measure. Two examples of her tailoring decisions:

Time behaviour: she keeps mean response time, response time adequacy and the turnaround time measures, but excludes mean throughput for this subcharacteristic because throughput is already covered under capacity by transaction processing capacity. The rationale is written next to the exclusion.
Fault tolerance: she adds the specific measure redundancy of components, because the supplier's architecture relies on redundant instances, and it is cheap to check from the deployment description.

RouteWell also needs a measure the catalogue does not offer: the share of parcel status updates shown to customers within two minutes of a driver's scan. Amara defines it as a new measure with two QMEs (scans in the period and scans displayed within two minutes), documents the measurement method, and notes that it is a RouteWell-defined measure.

Step 4: targets and decision rules

Measure	Data source (QMEs)	Target	Decision rule
Functional coverage	Functions in the signed specification; functions missing at acceptance test	1.0 for "critical" functions, at least 0.95 overall	Must pass
Mean response time (dispatch screen)	Load test tool, peak profile of 1,200 active dispatch users	2.0 seconds or less	Must pass
Transaction processing capacity	Load test, parcels assigned per minute at stable error rate	At least 900 per minute	Must pass
System availability	Monitoring tool; agreed service calendar	At least 0.995 per month	Service credit if missed
User audit trail completeness	Sample of 200 sensitive actions compared with the audit log	1.0	Must pass
Modification efficiency	Effort recorded for a set of standard change requests	Trend reported each milestone	Monitor only

The team agrees that a release may go live only if every "must pass" target is met, and that "monitor only" measures inform the next milestone plan but do not block.

Step 5 and 6: collect and compute

At the first acceptance milestone the supplier runs the agreed load test in a production-like environment, with RouteWell observing. The tool records a mean response time of 1.7 seconds on the dispatch screen at the peak profile, and a capacity of 960 parcels assigned per minute. The acceptance test finds that 4 of the 160 specified functions are missing, one of them marked critical: coverage is 1 minus 4 divided by 160, or 0.975 overall, but below 1.0 for critical functions. The audit sample shows 196 of 200 sensitive actions logged, a completeness of 0.98, because address changes made through a back-office screen are not logged.

Step 7 and 8: compare, decide and improve

Performance targets are met. Two must-pass targets fail: one critical function is missing and audit completeness is below 1.0. The milestone is not accepted. Because the raw QMEs were kept, the supplier can see exactly which function and which screen are at fault, and fixes both within two weeks. Re-measurement shows coverage of 1.0 for critical functions and full audit completeness, and the milestone is accepted.

After go-live, availability is measured monthly from monitoring. In the first quarter it is 0.997, 0.993 and 0.998. The second month triggers a service credit and a root-cause review that leads the supplier to add a further redundant instance, which Amara records under the redundancy measure. Modification efficiency, tracked as a trend, worsens slightly over three milestones; the internal coupling measure on the code shows a matching rise, so the architects schedule refactoring before the next major feature.

What the example shows

The standard did not tell RouteWell what "good" was; the business did. ISO/IEC 25023 gave the team a shared vocabulary of measures, a disciplined way to choose and justify them, and formulas that both parties could recompute. Because its specification follows the conformance rules (characteristics from 25010, generic measures by default, recorded exclusions, an explicitly defined additional measure), RouteWell could state that its quality requirements conform to ISO/IEC 25023.

Lesson 7 – Relationship to other standards and frameworks

Inside SQuaRE

The SQuaRE family is organised in divisions: quality management (2500n), quality models (2501n), quality measurement (2502n), quality requirements (2503n), quality evaluation (2504n) and an extension division (25050 to 25099). ISO/IEC 25023 belongs to the measurement division. Its siblings there are ISO/IEC 25020 (the quality measurement framework, including the measurement reference model), ISO/IEC 25021 (quality measure elements), ISO/IEC 25022 (measures for quality in use) and ISO/IEC 25024 (measures for data quality). Each measurement standard serves one quality model: 25023 serves the product quality model of ISO/IEC 25010, 25022 the quality-in-use model and 25024 the data quality model of ISO/IEC 25012.

Figure 5 – Where ISO/IEC 25023 sits in SQuaRE: it turns the product quality model into measures that requirements and evaluations then use, alongside its sibling measurement standards.

Comparison table

Standard	What it provides	How it relates to ISO/IEC 25023
ISO/IEC 25010	Product quality model: characteristics and subcharacteristics	Defines what 25023 measures; the 2016 measures follow the 2011 model, the 2023 model needs a mapping
ISO/IEC 25021	Format for specifying quality measure elements, with examples	Building blocks of 25023 measures; new measures and QMEs must follow its format
ISO/IEC 25022	Measures for quality in use (outcomes for users in context)	Complements 25023, which measures the product itself rather than the outcomes of use
ISO/IEC 25030	Process and framework for quality requirements	Uses 25023 measures and targets to make quality requirements verifiable
ISO/IEC 25040 / 25041	Evaluation process and guidance for developers, acquirers and independent evaluators	Evaluations select 25023 measures and apply decision criteria to their results
ISO/IEC/IEEE 15939	General measurement process for systems and software	Gives the management process around measurement; 25023 supplies product quality content
ISO/IEC 5055	Automated measures of source code weaknesses for reliability, security, performance efficiency and maintainability	Offers tool-friendly internal measures that can complement or refine 25023 measures
ISO/IEC 9126 (withdrawn)	Former quality model and metric reports	Parts 2 and 3 were replaced by 25023 in 2016

Outside SQuaRE

Process frameworks such as CMMI, ISO/IEC 330xx process assessment or Automotive SPICE ask whether an organisation measures and controls its work; they do not supply product quality formulas. ISO/IEC 25023 can provide the product-side measures that such a measurement programme reports. Safety and security standards (for example IEC 61508 or ISO/IEC 27001) set their own requirements and evidence; 25023 measures can support that evidence, for instance by quantifying audit trail completeness, but they never replace the domain standard's requirements. Test standards such as the ISO/IEC/IEEE 29119 series describe test processes and techniques; many 25023 external measures are produced by exactly those tests.

Looking ahead: the revision

The draft that will replace the 2016 edition, ISO/IEC 25000-23, is titled "Measurement of product quality" and speaks of ICT products rather than only system and software products, in line with the widened scope of ISO/IEC 25010:2023. Its published scope still describes the same core purpose: quality measures and measurement methods for quantitatively assessing the characteristics and subcharacteristics of ISO/IEC 25010. Until it is published, details of its content are subject to change, so do not plan against draft content. The skills in this course (building measures from QMEs, tailoring with rationale, setting targets, interpreting honestly) carry over directly.

Common pitfalls and best practices

Common pitfalls

Measuring everything. Teams copy the entire catalogue into a dashboard. Collection costs explode, nobody reads the results and important signals drown. Select subcharacteristics from real needs first.
Expecting the standard to say what is good. People look for the "pass mark" for availability or response time in ISO/IEC 25023 and do not find it, because there is none. Targets come from stakeholders and context.
Silent tailoring. Generic measures are dropped or formulas changed without a written reason. The result can no longer claim conformance, and later readers cannot tell whether the change was deliberate.
Unstable definitions. What counts as a "function" or a "failure" changes from release to release, so trends become meaningless. Fix definitions in the measurement plan and version them.
Keeping only the final ratio. Without the raw QMEs nobody can check, recalculate or diagnose a result.
Comparing unlike numbers. Values measured under different conditions, on different products or with different methods are ranked against each other as if they were comparable.
Gaming the measure. When a single number is tied to bonuses or penalties, people improve the number rather than the quality, for instance by splitting functions to raise coverage.
Ignoring the model edition. Using 2016 measures with the 2023 model without mapping the renamed and moved subcharacteristics creates gaps, for example for the new safety characteristic.

Best practices

Start from decisions. For each measure, write down which decision it supports (accept, release, invest, refactor). If none, drop it.
Generic by default, exclusions in writing. Follow the conformance logic even if you do not claim conformance; it forces clear thinking.
Agree targets and decision rules before measuring. Record must-pass, acceptable and monitor-only values in advance.
Define every QME operationally. State source, tool, time window, counting rules and responsible person.
Automate collection. Feed QMEs from test tools, pipelines and monitoring so that measures are cheap and repeatable.
Pair internal and external measures. Use code and design measures as early warnings and check them against behaviour in test and operation.
Report context with every value. Show target, conditions, sample size and limitations next to the number.
Review the measure set regularly. Retire measures that no longer drive decisions and add ones for new risks, keeping a change history.

Summary and key takeaways

ISO/IEC 25023:2016 is the SQuaRE standard that supplies quality measures for the product quality model; it replaced ISO/IEC TR 9126-2 and 9126-3 and is now under revision as ISO/IEC 25000-23.
Measures are built from quality measure elements, obtained by measurement methods and combined by a measurement function; every value should be traceable down that chain.
The catalogue in clause 8 is organised by the eight characteristics and 31 subcharacteristics of ISO/IEC 25010:2011; using the 2023 model requires a mapping.
Internal measures look at static work products such as code and design; external measures look at the running system in test or operation; quality in use belongs to ISO/IEC 25022.
Generic measures are used by default for each selected subcharacteristic; exclusions and modifications need a rationale; specific measures are optional; new measures follow ISO/IEC 25021.
The standard sets no target values or ratings; targets and decision rules come from stakeholders and must be agreed before measuring.
There is no ISO/IEC 25023 certification; conformance applies to quality requirement specifications and evaluations, and the measures feed evaluations under ISO/IEC 25040 and 25041.
Good measurement is valid, repeatable, affordable and reported with its context.

Reflection questions

Pick a system you work on. Which three subcharacteristics carry the highest risk, and which generic measures would you start with for each of them?
For one of those measures, write the operational definition of each input: where the data come from, who collects it, over what period and with which counting rules.
Which generic measure in your chosen set would you exclude, and how would you word the rationale so that an auditor or a new team member understands it?
Your organisation has adopted the 2023 quality model, but the measures you use follow the 2011 model. How would you map them, and which needs (for example safety) would require new measures?
Think of a quality number in your organisation that is currently gamed or misunderstood. How would you change its definition, target or reporting to make it trustworthy?