Gains achieved by students who exceeded the previous year: PPS compared to ACCESS

    It is well established that high-achieving students make lower gains than other students.  This has been found in districts across the country and it is also true for Portland Public Schools, where students who exceed benchmarks consistently make lower gains than other students.  See the results for this year and earlier years here.  I have often heard that this was a result of students hitting the "ceiling" on the tests and did not reflect the student's learning or instruction. 

Under a very silly provision of No Child Left Behind, state tests must measure students against grade level expectations, not against higher expectations.  This does put a lid on how much students can demonstrate that they know when they have been working above grade level, but the Oregon OAKS, as a computer-adaptive test, has a somewhat higher "ceiling" than paper and pencil tests.

I was curious to see whether the testing ceiling was in fact making it impossible for high-achieving students to make gains, so I compared the gains made by ACCESS students with those of other students in Portland who "exceeded" benchmarks.  ACCESS admits students in the top 1% of achievement--if there was a ceiling they should be hitting it harder than other students.  They might have higher scores to begin with, but they would not have higher gains. If, on the other hand, the difference is a reflection of real differences in instruction, then ACCESS students might continue to outperform other students in PPS who also exceeded benchmarks.

PPS only recently began posting ACCESS scores separately from those of the other students in its shared building.

The results from this year are below.  They confirm the results for the previous year which was the first year that PPS began reporting these results separately.  I have included the mean achievement levels of the same group of students from the previous spring, so it is evident that ACCESS students who exceeded benchmarks are beginning the school year with higher scores than their classmates in other schools who also exceeded benchmarks. If they followed the normal pattern, the higher their starting score, the lower their gains would be.   However, this was not the case.  ACCESS students who exceeded continued to make higher gains than their classmates in other schools who exceeded, in both reading and math with the exception of a loss in sixth grade reading.  (Last year, there was a loss in fifth grade reading).   In particular, ACCESS students are making higher gains in math, outperforming in their gains not only other students who exceeded benchmarks, but other students who met them.

Because ACCESS is a small program, there is likely to be greater variability in test performance averages than in the district as a whole.  The scores should be compared over a period of several years before we can be confident about the story they tell.  Nevertheless, the results cast doubt on the common argument that regression to the mean, margins of error and ceiling effects inevitably depress the gains that can be obtained by high-achieving students.

This spread reflects the gains that other "exceeds" students could be making in PPS if they had access to appropriate curriculum and instruction.  It is a learning opportunity that has been taken from them.  In fact, it is a very minimal estimate because, as noted above, the tests that are being used don't truly measure what these students know.  Out-of-level testing for all students who exceeded would reveal a greater spread and would also significantly reduce the margin of error on the tests.

2010-11 exceeds gains, PPS and Access (the test score means for

 the "exceeds" students from the previous year are included)

READING
		PPS	PPS	ACCESS	ACCESS
	Grade	09-10 mean	Exceeds	09-10 mean	exceeds
	4	226.9	5.1	227.9	6.1
	5	231.8	2.3	235.8	2.8
	6	236.3	3.8	238.3	3.8
	7	240.8	5.4	243.6	5.9
	8	246.3	-1.4	250.7	-3.5
MATHEMATICS
		PPS	PPS	ACCESS	ACCESS
	Grade	09-10 mean	Exceeds	09-10 mean	exceeds
	4	223.9	8.7	225.3	17.9
	5	232.1	5.9	234.3	5.9
	6	236.4	3.4	239.0	6.2
	7	240.6	4.7	245.3	7.2
	8	246.6	3.9	250.8	8.2

 

 

 

 Gains by PPS Students who met benchmarks compared to ACCESS students who exceeded