Impacts of Title I Supplemental Educational Services on Student Achievement. NCEE 2012-4053

Impacts of Title I Supplemental Educational Services on Student Achievement. NCEE 2012-4053
Author: John Deke
Publisher:
Total Pages: 200
Release: 2012
Genre:
ISBN:

This report presents the findings of an evaluation sponsored by the Institute of Education Sciences (IES) at the U.S. Department of Education (ED) and conducted by Mathematica Policy Research (Mathematica) that uses a regression discontinuity (RD) design to assess the potential benefits of offering SES in districts that have unmet need. Specifically, the study focuses on six school districts in which more eligible students applied for SES than could be served with available funds (i.e., oversubscribed districts), and which therefore allocated scarce SES spaces by giving priority to lower-achieving students among the eligible applicants. The answers to the key questions addressed in the study are as follows: (1) "What is the average impact of offering SES to eligible applicants who are on the cusp of having access to services, in school districts where services are oversubscribed?" Across the six oversubscribed districts, the authors find no evidence of impacts of offering SES to students near the cut point for an offer. For students in these oversubscribed districts in grades 3-8 at the cusp of receiving an offer of services, they find no statistically significant impact of "offering" SES on student achievement in reading or in mathematics. The point estimate of the average impact on reading is -0.03 standard deviations, and that for mathematics is 0.05 standard deviations. Furthermore, there is no evidence of potential benefits for at-risk subgroups of students. Similarly, they find no statistically significant impact of "participating" in SES on student achievement in reading or mathematics. The estimated impact of participating (which involved an average of 21 hours of services) is -0.10 standard deviations for reading and 0.11 standard deviations for math (again estimated for students in grades 3-8 near the cutoff for an offer, in these oversubscribed districts); (2) "What are the characteristics of SES provided to students in oversubscribed districts?" Across districts participating in this study, services averaged 21.2 hours per student for the school year (standard deviation of 8.8), with over a third of the students (36 percent) receiving tutoring in both reading and math, 55 percent receiving tutoring in only reading, and 9 percent receiving tutoring in only math. For students receiving reading services, the mean was 17.2 hours of tutoring (standard deviation of 9.2). For students receiving math services, the mean was 12.5 hours of tutoring (standard deviation of 8.2). In the study districts, most providers (70 percent) were for-profit firms. On average, 60 percent of providers' instructional staff were regular schoolteachers working in the local district. Most providers (64 percent) offered services at the schools of their students. Providers reported that group sizes of 2-5 students were most frequently used, with most other sessions provided individually (in one-on-one sessions). On average, 44 percent of provider services were in groups of 2-5 students, 34 percent in one-on-one sessions, and 21 percent in groups of 6-10; and (3) "Are the characteristics of Supplemental Educational Services, providers, or practices in host school districts correlated with the estimated impacts?" There is variation across providers in the average number of hours of math and reading services received by students, with average hours in services focused on math ranging from 0 to 27 across providers and average hours in services focused on reading ranging from 0 to 43. However, the intensity of services is not significantly related to the estimated size of impacts on math or reading act (for these eligible students near the cutoff for having been offered SES). They also found no evidence that any other observed provider characteristics were significantly associated with stronger impacts. Appended are: (1) SES Assignment and Participation; (2) Estimation Methods; (3) Diagnostic Analyses; (4) Exploratory Analyses; (5) Assessing Robustness of Impact Estimates; (6) Impact Estimates and Graphical Analyses by Mini-Study; (7) SES Survey Data Collection Methods; and (8) Supplemental Materials to Chapter III: SES Provider Characteristics and Students' SES Experiences. (Contains 57 tables, 72 figures and 30 footnotes.) [For "Impacts of Title I Supplemental Educational Services on Student Achievement. Executive Summary. NCEE 2012-4054," see ED532017.].

Impacts of Title I Supplemental Educational Services on Student Achievement. Executive Summary. NCEE 2012-4054

Impacts of Title I Supplemental Educational Services on Student Achievement. Executive Summary. NCEE 2012-4054
Author: John Deke
Publisher:
Total Pages: 12
Release: 2012
Genre:
ISBN:

This paper presents an executive summary of a study that uses a regression discontinuity (RD) design to assess the potential benefits of offering SES in districts that have unmet need. Specifically, the study focuses on six school districts in which more eligible students applied for SES than could be served with available funds (i.e., oversubscribed districts), and which therefore allocated scarce SES spaces by giving priority to lower-achieving students among the eligible applicants. Findings from this evaluation are based on six, nonrepresentative school districts in three states (Connecticut, Ohio, and Florida) where more eligible students applied for SES than could be served with available funds. Across the six oversubscribed districts included in our study, there were 50,843 applicants to SES, of which 30,673 were in study grades (3-8). Among these applicants, 24,113 were assigned to services based on a measure of prior achievement. These 24,113 constitute the study population. Among the students in the study population, 19,750 students (82 percent of the applicants) were offered SES based on their prior scores while 4,363 students were not offered SES, and 16,954 (86 percent of those offered services) participated in SES (i.e., reported any tutoring services). Almost all participating students (98 percent) were offered services with their first choice providers, with most students being served by a few dominant providers. (Contains 2 footnotes.) [For the full report, "Impacts of Title I Supplemental Educational Services on Student Achievement. NCEE 2012-4053," see ED532016.].

State and Local Implementation of the "No Child Left Behind Act." Volume I

State and Local Implementation of the
Author: Ron Zimmer
Publisher:
Total Pages: 62
Release: 2007
Genre:
ISBN:

This report presents findings about the relationship between participation in the Title I school choice and supplemental educational services options and student achievement from the National Longitudinal Study of "No Child Left Behind" (NLS-"NCLB"). A key component of the "No Child Left Behind Act of 2001" ("NCLB") was to provide options to parents whose children had been attending Title I schools identified for improvement, corrective action, or restructuring due to failure to achieve adequate yearly progress toward meeting state standards for two or more years. Under "NCLB," parents have the option of: (1) transferring their children to another school in the district that is not in need of improvement; or (2) enrolling their children in supplemental education services (e.g., tutoring, remediation, or other academic instruction) in addition to instruction provided during the school day. This study used data from nine large, urban school districts to examine the characteristics of students participating in the two options and the resulting impact on student achievement. The study found the following: that participation was highest in elementary grades; that African-American students had the highest participation rates of all racial and ethnic groups; that participating students had lower achievement levels than eligible but nonparticipating students; that students who transferred tended to transfer to higher-achieving, racially balanced schools; and that there was no statistically significant (positive or negative) effect on achievement among students participating in the two options. In sum, although participation rates were not high, the users of the two Title I parental options came from the disadvantaged populations that "NCLB" is intended to target. Appendix A provides a description of the nine-district data set. Appendix B presents the full results of the alternative analyses of the school choice option, comparing achievement gains of current and future choosers. Appendix C presents a meta-analysis of effects of Title I school choice and supplemental educational services. The information in this report was provided through the congressionally mandated National Longitudinal Study of "No Child Left Behind" (NLS-"NCLB"), which was conducted by the RAND Corporation and the American Institutes for Research. (Contains 23 exhibits.) [This report was prepared for Policy and Program Studies Service, Office of Planning, Evaluation and Policy Development, US Department of Education.].

Measuring the Impact of Supplemental Educational Services on Academic Achievement

Measuring the Impact of Supplemental Educational Services on Academic Achievement
Author: Katharine A. Salling
Publisher:
Total Pages: 170
Release: 2012
Genre:
ISBN:

This thesis examines the Supplemental Educational Services (SES) component of the No Child Left Behind Act of 2001 (NCLB) and focuses on the effect of this program improvement intervention on the academic performance of low-achieving students who participate compared to the academic performance of low-achieving students who are eligible but do not participate in the program. Public schools and districts that serve a high percentage of students from low-income families receive federal funding under Title I of the Elementary and Secondary Education Act of 1965, reauthorized in 2001 as the No Child Left Behind Act (NCLB). Under NCLB, schools that receive Title I funds are required to test students annually to assess proficiency of basic skills in English-language arts and mathematics. Schools must demonstrate Adequate Yearly Progress (AYP) by increasing the margin of students who are academically proficient each year. Schools that fail to achieve the margin of proficiency after two years must reserve 15 percent of the Title I grant to provide Supplemental Educational Services (SES) for low-achieving, low-income students. The SES program is tutoring offered to students before- or after-school by private organizations not affiliated with the public school or district. Approved SES providers receive payment for the tutoring services from the district's Title I funds that are reserved pursuant to the program improvement sanction. NCLB requires the states to approve and authorize tutoring organizations or individuals to provide the SES program, and California requires firms to apply annually to be authorized and to submit accountability reports annually to maintain eligibility as a provider. In addition to information regarding providers' business operations and credential qualifications, SES providers must provide descriptive information regarding the tutoring services it delivers to individual students. However, the state or districts do not systematically evaluate SES providers to determine the statistical effect of the tutoring on students' academic achievement. Whereas NCLB is a standards-based education reform policy, the absence of research-based evidence of the SES program effect creates ambiguity regarding the intent of the intervention strategy and provokes questions as to whether the policy meets the test of Pareto improvement. This thesis uses a linear regression analysis and administrator interviews to evaluate the estimated effect of SES participation on the academic achievement of students attending two program improvement schools in a small suburban school district located in Northern California. The regression results indicate that a student's SES participation did not have a statistically significant effect on the student's test score for either English-language arts or mathematics. The conclusions drawn from the interviews with school district administrators indicate that school officials are uncertain and doubtful about the effectiveness of the SES intervention for improving student's academic performance on standardized test scores, particularly in comparison to program improvement strategies implemented in the classroom. With regard to implementation of the SES program, the interviews revealed that the district office assumes the role of ensuring compliance with the NCLB mandates and the delivery of the SES program. The district conforms to the administrative recommendations and requirements of the state for managing the program but does not engage in additional monitoring or statistical evaluation of the effectiveness of the SES program or providers. School officials are generally satisfied with the way in which the district administers the SES program, yet they are hesitant to endorse the program as an effective use of resources in the absence of research-based evidence. Although the district does not conduct any formal survey to assess the level of satisfaction with the SES program among participants, the administrators report that informal communications with parents and students reveal generally positive opinions about the program. I recommend a systematic approach for auditing the performance of the independent SES providers for evidence that the tutoring programs are effectively improving academic proficiency among the students who are eligible and participate in the entitlement program.

Supplemental Education Services Under No Child Left Behind: Who Signs Up, and What Do They Gain?

Supplemental Education Services Under No Child Left Behind: Who Signs Up, and What Do They Gain?
Author: Carolyn J. Heinrich
Publisher:
Total Pages: 60
Release: 2009
Genre:
ISBN:

Schools that have not made adequate yearly progress in increasing student academic achievement are required, under No Child Left Behind (NCLB), to offer children in low-income families the opportunity to receive supplemental educational services (SES). In research conducted in Milwaukee Public Schools, the authors explore whether parents and students are aware of their eligibility and options for extra tutoring under NCLB, and who among eligible students registers for SES. Using the best information available to school districts, the authors estimate the effects of SES in increasing students' reading and math achievement. They find no average impacts of SES attendance on student achievement gains and use qualitative research to explore possible explanations for the lack of observed effects. Appendices include: (1) Supplemental Education Services Evaluation Focus Group Protocol; and (2) Milwaukee Public Schools 2006-2007 Supplemental Education Services Student Survey. (Contains 8 tables and 19 notes.) [This paper was presented at the "NCLB: Emerging Findings Research Conference" at the Urban Institute, Washington, D.C. on August 12, 2009.].