Ministry of Health (revised) procedure for approval of a new test for drinking-water sample analysis

Ministry of Health (revised) procedure for approval of a new test for drinking-water sample analysis

The Ministry of Health has revised the procedure for the approval of new test(s) for drinking-water sample analyses. The procedure describes the Ministry of Health's requirements for drinking-water analytical laboratories carrying out compliance testing and needing to access the Ministry's Laboratory Register.

The Ministry of Health (revised) procedure for approval of a new test for drinking-water sample analysis (Word, 77 kB)
The Ministry of Health (revised) procedure for approval of a new test for drinking-water sample analysis (PDF, 85 kB)

New (revised) statistical validation criteria for drinking-water microbiological methods
A report by NIWA for the Ministry of Health

Date of publication: May 2005

Methods for establishing equivalence between a “gold standard method” and an alternative candidate method for enumeration of micro-organisms in drinking-water are discussed.

The focus is on presence/absence data, because “Maximum Acceptable Values” in current Drinking-Water Standards for New Zealand define a breach of standard as the presence of at least one micro-organism in a given tested volume.

A simple test is proposed for establishing the equivalence of an analytical method with the referee method for E.coli prescribed in the new (revised) Drinking-Water Standards for New Zealand 2005 (Appendix 3). First one calculates the lower one-sided 95% confidence limit for Cohen’s kappa (being a measure of chance-corrected agreement between the methods). If that limit exceeds 0.6 it can be inferred that the alternative method is equivalent to the standard method. (In more formal technical language this is a one-sided precautionary 5% level test of the hypothesis that the true value of Cohen’s kappa statistic is less than 0.6).

It is also concluded that the minimum number of samples to be used in equivalence testing is 50, with the proviso that if the alternative candidate method is borderline for equivalence with that many samples, one should increase this to 150 samples and assess equivalence again.

Some discussion of appropriate methodology for enumeration data is given. The concordance correlation coefficient (first proposed by Lawrence Lin in 1989, as corrected by him in 2000) is recommended for use with continuous data and MPN data from “QuantiTray” techniques. “Strength-of-agreement” criteria are proposed for that coefficient.

Document availability

This document is not available in hard copy, it is only available on this website as a PDF file.

Statistical validation criteria for drinking-water microbiological methods (PDF, 518 kB)

The NIWA concordance calculator

Read our copyright information

Go to information about ordering and downloading publications