Manufacturers are increasingly being asked by regulatory authorities to report data on their new products. For example, the United States Food and Drug Administration (FDA) requires that these data are used to compare a new tobacco product to a predicate tobacco product in order to prove substantial equivalence. In 2013, we showed^[1] the importance firstly to have validated and standardised methods with known precision, and secondly to use the appropriate statistical methods (critical differences as recommended by the ISO 5725 part 6) to compare results from different laboratories in order to avoid misleading conclusions. However, if the comparison of products involves several analytes simultaneously, then the use of critical differences separately for each analyte, using the same level of significance (p<0.05), could lead to wrong conclusions. This issue, well known in statistics, is still a very active topic of research with many challenges to be taken into account, such as the notion of independence and statistical power.

In this presentation we will define the multiple testing issue and concepts and we will introduce some existing methods for addressing multiple testing.

[1] B. Teillet, X. Cahours, T. Verron, S. Colard, S. Purkis. Comparison of Smoke Yield Data Collected from Different Laboratories. Beitr. Tabakforsch. Int. 25 (2013) 663-670.
[2] ISO5725-6, 2001. Accuracy (trueness and precision) of measurement methods and results. Part 6. Use in practice of accuracy values.