What exactly is the method, and what criteria did it meet?

When scientists say that a method meets criteria, the phrase can sound simple outside the laboratory. In this article, 'method' refers to an analytical method, also called an analytical procedure: the complete workflow used to transform a sample into a result. It includes the sample, sample matrix (everything in the sample aside from the target being measured), sample preparation, instrument or assay, controls, software, calculations, acceptance criteria, and the rules used to determine whether the result is reliable.

That workflow can take many forms. It may be a separation method for impurities, a cell-based potency assay, a quantitative polymerase chain reaction (qPCR) test for nucleic acids, a quantitative nuclear magnetic resonance (qNMR), or another spectroscopic procedure. It could also be a dissolution method using high-performance liquid chromatography (HPLC) for the readout, a particle size measurement, or another analytical procedure used to support a scientific or regulatory decision. The risk is the same in each case: a method can meet its planned criteria and still miss the real question.

This is why a measurement workflow can pass every planned lifecycle checkpoint, from development challenges to qualification, validation activities, verification, transfer, and routine monitoring, yet still fail to answer the broader scientific question it was designed for. That doesn't mean qualification or validation failed; it means the planned evidence may have asked too narrow a question. A separation method can miss a relevant degradant. A potency assay can measure a clean reference material but not a stressed sample affected by matrix effects, which cause interference from other sample components. A qPCR method can amplify efficiently, but inhibition or extraction recovery can distort the result. A particle size measurement can be repeatable, but changes in sample handling can alter the population being measured.

The stakes are clear whenever a measurement leads to a decision: releasing a batch, accepting a stability claim, comparing a changed process, judging potency, investigating an unexpected signal, or deciding whether a result near a threshold is safe to trust. The article is not asking laboratories to collect more data for its own sake. It is asking whether the data already being collected are strong enough for the decision being made.

The uncomfortable gap

This is not an attack on validation. Qualification, validation, verification, transfer, and routine monitoring (watching performance over time) are all essential steps. The problem arises when any of these is seen as a substitute for scientific understanding. Meeting planned criteria shows that something is achieved, but it doesn't prove everything. The real question isn't, "Did the method meet the criteria?" but rather, "Were those criteria strong enough to support the decision?"

Procedural confidence is distinct from evidentiary confidence

Procedural confidence is the assurance that a laboratory can perform a measurement procedure and meet established criteria. Evidentiary confidence is the assurance that the resulting data support the scientific or regulatory decision. The first is essential. The second is the aim.

Analytical science sometimes blurs the distinction between these two types of confidence. A validation report can demonstrate that a procedure follows its protocol. A transfer report can show that two laboratories produced comparable results. A verification study can confirm that a compendial procedure functions correctly under actual use conditions. These are important pieces of evidence. However, they do not automatically prove that the procedure answers the right question in the right samples under the right conditions.

Table 1. Procedural confidence is not evidentiary confidence.

A modern case study: particle size is more than just a number

Particle-size distribution (PSD) provides a clear example of this issue. In a 2026 AAPS Open meeting report, Xu and colleagues summarized a workshop hosted by the FDA and the Center for Research on Complex Generics (CRCG) on particle-size analysis. The workshop focused on Dynamic Light Scattering (DLS) and Laser Diffraction (LD), used five pre-workshop materials, and assembled regulators, industry scientists, academics, and instrument vendors to discuss measurement principles, data interpretation, robustness, validation, and reporting [14].

Used carefully, this is not a story about one method failing. It is a case study in why a number is not automatically evidence. The report emphasizes that PSD data should not be interpreted in isolation because what is measured depends on model assumptions, sample conditions, and analytical purpose. It also highlights that methodological transparency can matter more than pursuing a single true particle size, especially when DLS and LD can report different physical views of the same complex formulation [14].

The workshop examples clarify the evidence-chain problem. In DLS, factors such as viscosity input, dilution, scattering angle, weighting mode, and sample integrity can alter the apparent hydrodynamic size. In LD, variables like refractive index, obscuration, dispersion conditions, bubbles, software smoothing, and model choice can shift the distribution. A method may produce repeatable, clear-looking size distributions, but still leave the decision under-supported if the measured state isn't the product-relevant state. The key question isn't just whether the PSD method met acceptance criteria, but whether the measurement genuinely supports the decision: formulation development, bioequivalence, release, stability, comparability, or process understanding [14].

The value of this example is that it links the philosophy of Deming and Juran to a practical regulatory science issue. The solution is not to require identical numbers from every instrument. Instead, it is to define the analytical goal, manage and document the sample history, question the model assumptions, use orthogonal or complementary evidence when needed, and connect the measurement to the function of the material. The number becomes evidence only when the measurement system is sufficiently understood to support the decision based on it [14].

Deming and Juran are valuable because they are not based on analytical methods

W. Edwards Deming and Joseph M. Juran did not create analytical procedures. That's exactly why they are valuable here. They steer the discussion away from the false comfort of accepted paperwork and back toward the fundamentals of learning, planning, control, and improvement.

Deming's Plan-Do-Study-Act cycle is not just a decorative quality diagram. It is a disciplined learning loop: plan by setting a goal or hypothesis and defining success metrics; do by implementing the plan; study by comparing results to predictions and learning whether the hypothesis holds; act by applying the learning and adjusting or scaling the method [1,2]. The crucial word is study. Deming's warning is not that laboratories fail to check. It is that they sometimes check without learning.

Juran's quality trilogy includes an additional discipline. Quality planning determines what need must be fulfilled. Quality control evaluates if the process stays stable during operation. Quality improvement aims to reduce or eliminate persistent weaknesses [3,4]. In analytical science, the core message is clear: define the decision, establish the analytical target profile (a brief statement of what the procedure must measure and its required performance), design the measurement based on that target, manage the risks, and improve when evidence indicates that the initial understanding was incomplete [5,6].

The lifecycle shift in analytical procedures

Modern guidance has followed a similar path. The International Council for Harmonization (ICH), which creates globally harmonized technical guidance used by regulators and industry, released ICH Q14 (Analytical Procedure Development) to outline science- and risk-based methods for developing and maintaining analytical procedures that are suitable for assessing the quality of drug substances and drug products [7]. ICH Q2(R2) (Validation of Analytical Procedures, Revision 2) indicates that validation proves an analytical procedure is fit for its intended purpose and is part of the analytical procedure lifecycle [8]. The United States Pharmacopeia (USP) chapter <1220> uses a comparable lifecycle structure: design and development, procedure performance qualification, and ongoing procedure performance verification [9,10].

This is not just regulatory housekeeping. In pharmaceutical quality, weak analytical evidence can impact batch release, stability claims, comparability decisions, process control, regulatory confidence, and ultimately patient trust [11]. The lifecycle shift should not be seen as a call for more paperwork. Its main message is that development knowledge, risk assessment, fit-for-purpose validation, verification, transfer, and ongoing monitoring should be integrated, not treated as separate documents. Evidence is not created all at once. It accumulates over time: it begins with defining the question, grows as the process is challenged during development, is demonstrated through qualification and validation activities, is expanded through verification and transfer, and is sustained by continuous monitoring.

Table 2. The analytical procedure lifecycle as an evidence chain.

Validation is one checkpoint, not the whole bridge

Lifecycle vocabulary is useful only if each activity is understood for the distinct evidence it can produce. Validation asks whether relevant performance characteristics have been demonstrated for the intended analytical application. Verification asks whether a procedure, often a compendial one (an official pharmacopeial procedure such as one published by the United States Pharmacopeia, or USP), is suitable under actual conditions of use [12]. Transfer asks whether another laboratory can perform the procedure with the needed knowledge and capability [13]. Ongoing procedure performance verification asks whether the procedure remains under control during routine use [9,10]. When these activities are applicable, none should be treated as a formality. But none is sufficient by itself. A strong analytical procedure is not a certificate. It is a chain of evidence.

The same evidence gap appears across analytical science

Consider several common analytical scenarios. A chromatographic impurity method validates against known standards but misses a coeluting degradant. A cell-based potency assay meets precision and range criteria but fails to reveal that a matrix component, which is another substance in the sample, suppresses the biological response. A qPCR assay provides efficient amplification, but extraction recovery or inhibition affects the apparent copy number. A qNMR procedure uses a clean diagnostic signal, but spectral overlap or solvent interference biases the result. A particle size measurement repeats reliably, but dilution or sample handling alters the population before analysis.

The lesson is not that these techniques are fragile or unreliable. The real lesson is that every measurement depends on context. The evidence is only as strong as the question asked during development, the samples used to challenge the method, the controls chosen for routine use, and the willingness to revise the process when real-world conditions demand it. A result close to a specification, reporting threshold, potency goal, or clinical decision boundary needs extra careful review because small analytical uncertainties can lead to significant decision risks.

This broader framing aligns with Q2(R2). The guidance addresses common uses such as assay, potency, purity, impurity, identity, and other quantitative or qualitative measurements. Examples include separation techniques, dissolution with HPLC, qNMR, biological assays, quantitative PCR, and particle size measurements [8]. In other words, the concern is not chromatography but the reliability of any measurement used to support a significant decision.

Table 3. How evidentiary gaps appear across analytical modalities.

Controls are a philosophy made visible

System suitability (a predefined check that the measurement system is performing acceptably) is often viewed as just a routine gate before starting the actual work. That perspective is too narrow and too focused on chromatography. In lifecycle thinking, controls are where the scientist demonstrates what they believe could go wrong.

In a separation method, the control may monitor critical resolution or peak shape. In a potency assay, it may observe reference standard behavior, cell health, reagent lot, and curve similarity. In qPCR, it may track extraction controls, inhibition controls, negative controls, copy standards, and primer or probe specificity. In qNMR or spectroscopy, it may oversee signal selection, solvent interference, calibration, and model space. In particle measurement, it may track dilution, dispersion, aggregation, adsorption, and the assumptions used to convert signals into sizes or distributions.

A good control strategy should not just ask, "Can the procedure run today?" It should also ask, "Can this measurement system detect the failures that would make this result untrustworthy?" The more challenging question is the scientific one.

Table 4. Controls should reveal the failure modes that matter.

Where this argument can be abused

There are two simple ways to misuse this argument. The first is anti-validation rhetoric, which is wrong because validation is essential. The second is lifecycle theater, which is more common and more dangerous: attractive lifecycle documents that hide fragile procedures, risk assessments that list hazards without testing them, and monitoring programs that focus on trending convenient numbers rather than actual risks.

Deming and Juran do not fix weak science. A Plan-Do-Study-Act wheel does not improve a measurement process. A quality trilogy slide does not produce quality planning. A lifecycle table does not demonstrate understanding of a procedure. These concepts only earn their place when they lead to better experiments, controls, transfer, monitoring, and decisions.

Table 5. Evidence-chain review checklist for analytical leaders.

What should change

For analytical leaders, the practical change is not to demand more tests indiscriminately. It is to make the evidence chain explicit before the result is used: define the analytical target profile, test realistic matrices, choose controls that reveal plausible failures, treat transfer as knowledge transfer, review uncertainty near decision thresholds, and use routine monitoring as a learning system. For regulatory and quality professionals, the same principle applies in another language: do not ask only whether the package is complete. Ask whether the package explains why the measurement warrants the decision.

What analytical method quality really means

The quality of an analytical method is not determined by the absence of failed validation runs. It's not about having a successful transfer package. It's not solely based on a system suitability result that meets criteria. While these are important pieces of evidence, they do not form the entire case. A measurement procedure gains trust when the organization can explain what the measurement is intended to demonstrate, the remaining uncertainty about the result, the failure modes tested, the controls that reveal these failures, the independent or complementary evidence supporting the conclusion, and what evidence would signal the need for improvement.

Deming assigns analytical science the discipline of study, while Juran emphasizes planning, control, and improvement. Regulations such as ICH Q14, ICH Q2(R2), USP <1220>, USP <1224> (Transfer of Analytical Procedures), USP <1226> (Verification of Compendial Procedures), and guidance from the U.S. Food and Drug Administration (FDA) establish the regulatory framework for this discipline [7-13]. An experienced scientist should not only ask whether the method met the criteria. The more challenging question is, What did it prove, what did it not prove, and is that evidence strong enough to support the decision? That question may be uncomfortable, but it marks the start of true method quality.

References

[1] W. Edwards Deming Institute. PDSA Cycle. Describes Plan-Do-Study-Act as a learning cycle with emphasis on Study rather than Check. Deming Institute PDSA

[2] Deming, W. Edwards. Out of the Crisis. MIT Press. Bibliographic record. MIT Press record

[3] Juran Institute. The Juran Trilogy. Summarizes quality planning, quality control, and quality improvement. Juran Trilogy page

[4] Juran, J. M. Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services. Free Press, 1992. Bibliographic record. WorldCat record

[5] Jackson, P., Borman, P., Campa, C., et al. Using the Analytical Target Profile to Drive the Analytical Method Lifecycle. Analytical Chemistry 2019, 91, 2577-2585. DOI: 10.1021/acs.analchem.8b04596

[6] Borman, P., Campa, C., Delpierre, G., et al. Selection of Analytical Technology and Development of Analytical Procedures Using the Analytical Target Profile. Analytical Chemistry 2022, 94, 559-570. DOI: 10.1021/acs.analchem.1c03854

[7] International Council for Harmonisation. Q14 Analytical Procedure Development. Step 4 adopted November 2023; FDA guidance issued March 2024. ICH Q14 PDF; FDA Q14 guidance page

[8] International Council for Harmonisation. Q2(R2) Validation of Analytical Procedures. Step 4 adopted November 2023; FDA guidance issued March 2024. ICH Q2(R2) PDF; FDA Q2(R2) guidance page

[9] United States Pharmacopeia. <1220> Analytical Procedure Life Cycle. USP-NF official preview page. USP <1220>

[10] Borman, P. J., Guiraldelli, A. M., Weitzel, J., et al. Ongoing Analytical Procedure Performance Verification Using a Risk-Based Approach to Determine Performance Monitoring Requirements. Analytical Chemistry 2024, 96, 966-979. DOI: 10.1021/acs.analchem.3c03708

[11] U.S. Food and Drug Administration. Analytical Procedures and Methods Validation for Drugs and Biologics: Guidance for Industry. July 2015. FDA guidance PDF

[12] United States Pharmacopeia. <1226> Verification of Compendial Procedures. USP-NF official preview page. USP <1226>

[13] United States Pharmacopeia. <1224> Transfer of Analytical Procedures. USP-NF official preview page. USP <1224>

[14] Xu, X., Smith, W. C., Polli, J. E., et al. Mastering particle size analysis: lessons, challenges, and future directions from the FDA-CRCG workshop. AAPS Open 2026, 12, 11. DOI: 10.1186/s41120-026-00148-4