Evaluation of Urban Advantage by the NYU Institute of Educational and Social Policy
Since 2008, researchers at New York University have studied the impact of Urban Advantage (UA) on students’ achievement in science. Findings are reported in white papers and published in peer-reviewed journal articles.
Urban Advantage Interim Report
Citation:
Weinstein, M., Debraggio, E., Schwartz, A. E., Leos-Urbel, J., Nazar, L. (2010). Urban Advantage Interim Report. IESP Report, NYU Wagner.
Description:
The main findings from this evaluation paper include: (1) attending a UA school increases students’ performance on the Intermediate Level Science (ILS) exam by approximately 0.05 standard deviations, (2) effects were larger for black students, special education students and male students, and (3) small positive effects on the likelihood that a student takes a science Regents exam in the eighth or ninth grade, but no effect on the probability of scoring proficient on the exam.
Successful schools: How school-level factors influence success with Urban Advantage
Citation:
Weinstein, M., Whitesell, E. R., Leardo, M. (2014). Successful schools: How school-level factors influence success with Urban Advantage. IESP Report, NYU Steinhardt.
Description:
This analysis uses fixed-effects model to estimate school effects. This paper also identifies 9 school case studies and examines what promotes uptake of UA practices at these schools? In addition, they isolated the unique contribution of schools to science achievement, above and beyond school participation in UA and student characteristics. They find that UA schools do not, on average, have higher school effects than non-UA schools, suggesting that positive impacts of UA reflect program effects and not overall school quality. The case study analysis identifies collaboration as an important theme—schools that had higher collaboration reported more success with UA.
Museum, zoos, and gardens: How formal-informal partnerships can impact urban students’ performance in science
Citation:
Weinstein, M., Whitesell, E.R., & Schwartz, A.E. (2014). Museum, zoos, and gardens: How formal-informal partnerships can impact urban students’ performance in science. Evaluation Review, 38(6), 514-545.
Description:
The main findings from this evaluation paper include: (1) attending a UA school increases students’ performance on the Intermediate Level Science (ILS) exam by approximately 0.05 standard deviations, (2) effects were larger for black students, special education students and male students, and (3) small positive effects on the likelihood that a student takes a science Regents exam in the eighth or ninth grade, but no effect on the probability of scoring proficient on the exam.
Rolling out and scaling up: What happens when a program is no longer new
Citation:
Weinstein, M. & Whitesell, E. R. (2015). Rolling out and scaling up: What happens when a program is no longer new. A paper prepared for the XXIV Meeting of the Economics of Education Association.
Description:
This analysis uses a longitudinal data set that spans 2005-2014. This data set includes individual student characteristics linked to school, but not teacher. They found that the effects of UA on students’ achievement is greatest for schools in the second-to-lowest quartile. There are small difference in effect between schools with low, medium, and high concentrations of UA teachers—higher concentration of UA teachers are associated with higher achievement. There is an effect of longer time in UA on proficiency but not z-score. There is a significant effect of repeated exposure. Students that experience two years of UA are 6 percentage points more likely to be proficient on the eighth grade test than students in non-UA schools. However, participation for a single year is not associated with a significant difference in achievement.
A day at the museum: The impact of field trips on middle school science achievement.
Citation:
Whitesell, E.R. (2016). A day at the museum: The impact of field trips on middle school science achievement. Journal of Research in Science Teaching, 53(7), 1036–1054. [view article]
Description:
This analysis, published in the premier journal in science education the Journal of Research in Science Teaching, uses 6 years of student-level data to estimate impact of field trip visits to informal science institutions by UA students. The study finds small positive effects of exposure to field trips on students’ science test scores and proficiency. Effects were largest for Hispanic students and those who qualified for free or reduced-price lunch.
The urban advantage: The impact of informal science collaborations on student achievement
Citation:
Weinstein, M.G. & Shiferaw, M. (2017a). The urban advantage: The impact of informal science collaborations on student achievement. Institute for Education and Social Policy Working Paper. New York University.
Description:
This analysis uses unique student-teacher linkage data that allowed researchers to identify students taught by a UA teacher. The comparison group consists of schools that are not participating in UA in that year. Researchers use Propensity Score Matching (PSM) to obtain a matched set of treatment and control schools that have the same propensity to participate in UA. The main finding reported is that having a UA teacher improves students’ scores on the intermediate-level science test. Using models comparing students with and without a UA teacher in the same school, they find that the program increases scores on the 8th grade ILS exam by 0.08 standard deviations (moves participating students from 64th to 67th percentile of the distribution). PSM results are attenuated (0.03 standard deviations).
The urban advantage: Do informal science collaborations improve teacher retention?
Citation:
Weinstein, M.G. & Shiferaw, M. (2017b). The urban advantage: Do informal science collaborations improve teacher retention? Institute for Education and Social Policy Working Paper. New York University.
Description:
The main finding of this analysis is that UA participation increases teacher retention. UA science teachers are, on average, three percentage points more likely than non-UA science teachers to remain teaching at their school the following year. This impact is substantially higher for teachers with 3-5 years of teaching experience (16 percentage points).
ESSENTIAL INFO:
For more information, contact: [email protected]
Education Research Profile: Anna MacPherson