Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) is a widely used, computerized neuropsychological test battery for the diagnosis and management of sport-related concussions (SRC). As SRC is known to affect neurocognitive performance, athletes provide pre-injury, baseline ImPACT scores, to which post-injury scores can be compared in the event of SRC. However, if an athlete's baseline scores are not fully representative of his abilities, the utility of post-injury score comparison is diminished. Return-to-play release may be granted prematurely, putting the athlete at risk for second injury or long-term consequences. For this reason, the ImPACT includes low score thresholds on five validity indices to identify insufficient effort, though evidence of these indices' performance is limited. The present study compares existing ImPACT validity indices, as well as three proposed indices not currently being used to inform protocol validity, to external validity measures. The ImPACT, Word Memory Test (WMT) and Minnesota Multiphasic Personality Inventory - 2- Restructured Form (MMPI-2-RF) were administered to 242 undergraduate students. Participants were instructed to either give full effort on testing or to simulate SRC. The ImPACT demonstrated significantly higher specificity (0.94) and lower sensitivity (0.42) as compared to the WMT and MMPI-2-RF. Alternative score thresholds for the existing ImPACT validity indices may maintain high specificity while improving sensitivity. The three proposed indices showed higher sensitivities than the existing ImPACT indices, though lower specificities. The existing ImPACT indices' high specificity at the expense of lower sensitivity compared to external validity measures raises concern, as unidentified poor-effort may result in premature return-to-play decisions for athletes with SRC. Improvements or additions to the existing indices may raise sensitivity while maintaining acceptable specificity, aiding in the protection of athletes and safe athletic participation.