Resampling-Based Multiple Testing

Resampling-Based Multiple Testing
Author: Peter H. Westfall
Publisher: John Wiley & Sons
Total Pages: 382
Release: 1993-01-12
Genre: Mathematics
ISBN: 9780471557616

Combines recent developments in resampling technology (including the bootstrap) with new methods for multiple testing that are easy to use, convenient to report and widely applicable. Software from SAS Institute is available to execute many of the methods and programming is straightforward for other applications. Explains how to summarize results using adjusted p-values which do not necessitate cumbersome table look-ups. Demonstrates how to incorporate logical constraints among hypotheses, further improving power.

Multiple Testing and False Discovery Rate Control

Multiple Testing and False Discovery Rate Control
Author: Shiyun Chen
Publisher:
Total Pages: 142
Release: 2019
Genre:
ISBN:

Multiple testing, a situation where multiple hypothesis tests are performed simultaneously, is a core research topic in statistics that arises in almost every scientific field. When more hypotheses are tested, more errors are bound to occur. Controlling the false discovery rate (FDR) [BH95], which is the expected proportion of falsely rejected null hypotheses among all rejections, is an important challenge for making meaningful inferences. Throughout the dissertation, we analyze the asymptotic performance of several FDR-controlling procedures under different multiple testing settings. In Chapter 1, we study the famous Benjamini-Hochberg (BH) method [BH95] which often serves as benchmark among FDR-controlling procedures, and show that it is asymptotic optimal in a stylized setting. We then prove that a distribution-free FDR control method of Barber and Candès [FBC15], which only requires the (unknown) null distribution to be symmetric, can achieve the same asymptotic performance as the BH method, thus is also optimal. Chapter 2 proposes an interval-type procedure which identifies the longest interval with the estimated FDR under a given level and rejects the corresponding hypotheses with P-values lying inside the interval. Unlike the threshold approaches, this procedure scans over all intervals with the left point not necessary being zero. We show that this scan procedure provides strong control of the asymptotic false discovery rate. In addition, we investigate its asymptotic false non-discovery rate (FNR), deriving conditions under which it outperforms the BH procedure. In Chapter 3, we consider an online multiple testing problem where the hypotheses arrive sequentially in a stream, and investigate two procedures proposed by Javanmard and Montanari [JM15] which control FDR in an online manner. We quantify their asymptotic performance in the same location models as in Chapter 1 and compare their power with the (static) BH method. In Chapter 4, we propose a new class of powerful online testing procedures which incorporates the available contextual information, and prove that any rule in this class controls the online FDR under some standard assumptions. We also derive a practical algorithm that can make more empirical discoveries in an online fashion, compared to the state-of-the-art procedures.

Multiple Testing Problems in Pharmaceutical Statistics

Multiple Testing Problems in Pharmaceutical Statistics
Author: Alex Dmitrienko
Publisher: CRC Press
Total Pages: 323
Release: 2009-12-08
Genre: Mathematics
ISBN: 1584889853

Useful Statistical Approaches for Addressing Multiplicity IssuesIncludes practical examples from recent trials Bringing together leading statisticians, scientists, and clinicians from the pharmaceutical industry, academia, and regulatory agencies, Multiple Testing Problems in Pharmaceutical Statistics explores the rapidly growing area of multiple c

Statistics Done Wrong

Statistics Done Wrong
Author: Alex Reinhart
Publisher: No Starch Press
Total Pages: 177
Release: 2015-03-01
Genre: Mathematics
ISBN: 1593276206

Scientific progress depends on good research, and good research needs good statistics. But statistical analysis is tricky to get right, even for the best and brightest of us. You'd be surprised how many scientists are doing it wrong. Statistics Done Wrong is a pithy, essential guide to statistical blunders in modern science that will show you how to keep your research blunder-free. You'll examine embarrassing errors and omissions in recent research, learn about the misconceptions and scientific politics that allow these mistakes to happen, and begin your quest to reform the way you and your peers do statistics. You'll find advice on: –Asking the right question, designing the right experiment, choosing the right statistical analysis, and sticking to the plan –How to think about p values, significance, insignificance, confidence intervals, and regression –Choosing the right sample size and avoiding false positives –Reporting your analysis and publishing your data and source code –Procedures to follow, precautions to take, and analytical software that can help Scientists: Read this concise, powerful guide to help you produce statistically sound research. Statisticians: Give this book to everyone you know. The first step toward statistics done right is Statistics Done Wrong.

Multiple Comparison Procedures

Multiple Comparison Procedures
Author: Yosef Hochberg
Publisher:
Total Pages: 482
Release: 1987-10-05
Genre: Mathematics
ISBN:

Offering a balanced, up-to-date view of multiple comparison procedures, this book refutes the belief held by some statisticians that such procedures have no place in data analysis. With equal emphasis on theory and applications, it establishes the advantages of multiple comparison techniques in reducing error rates and in ensuring the validity of statistical inferences. Provides detailed descriptions of the derivation and implementation of a variety of procedures, paying particular attention to classical approaches and confidence estimation procedures. Also discusses the benefits and drawbacks of other methods. Numerous examples and tables for implementing procedures are included, making this work both practical and informative.

Controlling Error Rates with Multiple Positively-dependent Tests

Controlling Error Rates with Multiple Positively-dependent Tests
Author: Abdullah Al Masud
Publisher:
Total Pages: 75
Release: 2011
Genre:
ISBN:

It is a typical feature of high dimensional data analysis, for example a microarray study, that a researcher allows thousands of statistical tests at a time. All inferences for the tests are determined using the p-values; a smaller p-value than the alpha-level It is a typical feature of high dimensional data analysis, for example a microarray study, that a researcher allows thousands of statistical tests at a time. All inferences for the tests are determined using the p-values; a smaller p-value than the -level of the test signifies a statistically significant test. As the number of tests increases, the chance of observing some small p-values is very high even when all null hypotheses are true. Consequently, we make wrong conclusions on the hypotheses. This type of potential problem frequently happens when we test several hypotheses simultaneously, i.e., the multiple testing problem. Adjustment of the p-values can redress the problem that arises in multiple hypothesis testing. P-value adjustment methods control error rates [type I error (i.e. false positive) and type II error (i.e. false negative)] for each hypothesis in order to achieve high statistical power while keeping the overall Family Wise Error Rate (FWER) no larger than, where is most often set to 0.05. However, researchers also consider the False Discovery Rate (FDR), or Positive False Discovery Rate (pFDR) instead of the type I error in multiple comparison problems for microarray studies. The methods involved in controlling the FDR always provide higher statistical power than the methods involved in controlling the type I error rate while keeping the type II error rate low. In practice, microarray studies involve dependent test statistics (or p-values) because genes can be fully dependent on each other in a complicated biological structure. However, some of the p-value adjustment methods only deal with independent test statistics. Thus, we carry out a simulation study with several methods involved in multiple hypothesis testing. Our result suggests a suitable method given that the test statistics are dependent with a particular covariance structure while allowing different values of the underlying parameters in the alternative hypotheses.

Statistical Analysis of Gene Expression Microarray Data

Statistical Analysis of Gene Expression Microarray Data
Author: Terry Speed
Publisher: CRC Press
Total Pages: 237
Release: 2003-03-26
Genre: Mathematics
ISBN: 0203011236

Although less than a decade old, the field of microarray data analysis is now thriving and growing at a remarkable pace. Biologists, geneticists, and computer scientists as well as statisticians all need an accessible, systematic treatment of the techniques used for analyzing the vast amounts of data generated by large-scale gene expression studies

Large-Scale Inference

Large-Scale Inference
Author: Bradley Efron
Publisher: Cambridge University Press
Total Pages:
Release: 2012-11-29
Genre: Mathematics
ISBN: 1139492136

We live in a new age for statistical inference, where modern scientific technology such as microarrays and fMRI machines routinely produce thousands and sometimes millions of parallel data sets, each with its own estimation or testing problem. Doing thousands of problems at once is more than repeated application of classical methods. Taking an empirical Bayes approach, Bradley Efron, inventor of the bootstrap, shows how information accrues across problems in a way that combines Bayesian and frequentist ideas. Estimation, testing and prediction blend in this framework, producing opportunities for new methodologies of increased power. New difficulties also arise, easily leading to flawed inferences. This book takes a careful look at both the promise and pitfalls of large-scale statistical inference, with particular attention to false discovery rates, the most successful of the new statistical techniques. Emphasis is on the inferential ideas underlying technical developments, illustrated using a large number of real examples.

The Analysis of Variance

The Analysis of Variance
Author: Henry Scheffé
Publisher: John Wiley & Sons
Total Pages: 500
Release: 1999-03-05
Genre: Mathematics
ISBN: 9780471345053

Originally published in 1959, this classic volume has had a major impact on generations of statisticians. Newly issued in the Wiley Classics Series, the book examines the basic theory of analysis of variance by considering several different mathematical models. Part I looks at the theory of fixed-effects models with independent observations of equal variance, while Part II begins to explore the analysis of variance in the case of other models.