Common Core Pilot Tests Expose Glitches, IT Weaknesses

Technology and tests get a workout as school districts nationwide pilot Common Core assessments from the Smarter Balanced Assessment Consortium.

by / March 25, 2013 0
Wooden block toy image via Shutterstock.

The first large-scale pilot of online assessments that measure students against the Common Core State Standards is underway — and the more than 6,000 participating schools are figuring out how to deal with minor test glitches and older IT equipment.

The Common Core State Standards were developed for K-12 students in English and math by the National Governors Association Center for Best Practices and the Council of Chief State School Officers, and 45 states and the District of Columbia have chosen to adopt them.

To develop assessments for these standards, two state consortia received grants from the U.S. Department of Education, and by the 2014-15 school year, both will finalize computer-based assessments for schools to use.

But one of the consortia, the Smarter Balanced Assessment Consortium, already has launched its first major pilot test for schools, occurring from February through May 2013. The consortium developed this pilot test, in part, from the more than 500 schools that participated in its small-scale trials in fall 2012.

For two Southern California school districts that participated in the smaller trials in the fall and are now going through the pilot test phase, the larger pilot test has been fairly smooth comparatively. The October 2012 trial was rough because many procedures and processes weren't really nailed down, said Jim Klein, director of information services and technology for Saugus Union School District in Santa Clarita.

Assessment challenges

In this larger-scale pilot, the phrasing of questions, minor interface glitches and a long test time have caused challenges.

"The wording is challenging and complicated," Klein said. "It's not necessarily been put together by people who understand computers, and has been put together by multiple people, so you get different takes on different things."

For example, some questions used the word "select" when they really meant "highlight." And one question gave students two columns of words and asked them to draw lines between the words that go together -- something that's not possible on a computerized test. 

Also in need of some attention are Interface glitches. If students scroll to the bottom of a question, they can scroll back up partially, but can't get all the way to the top. Though the tech team figured out a few workarounds, they are by no means true solutions. The first requires the student to press the "tab" key numerous times as they scroll or pause the test, which requires that the student log out and log in again with his or her 11-digit state student ID number.

The system only allows 20 minutes of pause time total before it automatically submits a student's test. But some of the tests take an hour and a quarter to finish -- which is too long for students to sit without more break time, Klein said. Their 20 minutes disappears quickly because logging in and out to fix technical issues or to take bathroom and water breaks eats into that time.

Klein advises the consortium members to have people who did not write the questions go over them to eliminate confusion and also to address the test time issue.

"They need to do a little bit better time management," Klein said. "They need to understand that 85 minutes is way too long to be staring at a computer screen, and there needs to be a way to take a break in the middle of the test, which we really don't have the capacity to do right now the way it's set up."

On the hardware and bandwidth side of the pilot, Saugus Union School District has figured out a way to make do. It had no trouble with its Windows computers in the lab or its Linux laptops as far as performance.

The only issue with the Windows computers was installing the secure browser that the consortium provides for the test and getting it to run. The Windows machines run Open Office and have Windows Live Messenger. The secure browser detects that these programs are running in the background and forces users to stop them before it will work. The students don't know how to close little tray icons and apps to get the browser to open, so that takes some hands-on help.

Bandwidth and hardware challenges

Even though the district has a relatively slow Internet connection of 5 megabits, it hasn't caused a problem because the district caches Web pages to boost performance. This summer, the district will upgrade to 20 megabits.

But slow Internet connections and old network hardware have caused some challenges for another district 50 miles southeast of Santa Clarita, Calif. Whittier City School District has a 30 megabit Internet connection for its whole district, and for the current pilot, picked a middle school with older network hardware equipment to see whether everything would work correctly, said Rolland Kornblau, director of technology. 

Because the district picked a middle school that had newer laptops for the fall 2012 trial, however, that test went much more smoothly. The major issues that it experienced then were low bandwidth and network dropages, which happened frequently because so many students consistently connected to the Internet. If a connection did drop, the browser froze, and students would have to log back in.

In this pilot, the district has had a tough time with the network hardware and wireless.

"We felt a lot of our weaknesses were brought to light basically because of the Apple air ports that we're using," Kornblau said. "We're having some issues sustaining more than 30 connections like they're rated for.

Whittier City School District used a combination of desktop Mac 10.7's, Asus netbooks running ubermix and a limited trial of iPads. This combination gave them more flexibility in price and equipment, especially at a price tag of $260 for the netbook with ubermix, a no-charge operating system based on Linux.

Because of California's budget challenges over the last six or seven years, the school district hasn't been able to update its hardware and wireless -- items it needs to handle the Common Core assessments.

"If we were to open this up and have to test a whole school from 9-12 weeks, we'd have a lot of issues, mostly again because of hardware and wireless capacity," Kornblau said.

His district qualifies for 81 percent of its broadband and related costs to be covered by the federal e-rate program, and a district bond has helped upgrade hardware at seven out of 12 locations. But it still has a ways to go before it's prepared.

What's next?

At the end of the pilot in May, the Smarter Balanced Assessment Consortia will collect feedback from participating schools and use data from the scientific sample of about 6,000 schools to program the adaptive test engine that isn't in the online tests yet. Adaptive tests adjust to students' knowledge level to give a more accurate picture of how students match up to the standards. For example, a student who answers "easy" questions correctly will get tougher questions next and vice versa.

Despite the challenges, the assessment process is running well and shows promise for school districts.

"At this point, the technical issues are pretty well worked out," Klein said. "Now it's more the human issues that have to be addressed — minor interface glitches and the wording and those sorts of things we just need to work through. I'm confident that they'll get that sorted out."

Tanya Roscorla Former Managing Editor

Tanya Roscorla covered ed tech from 2009-2017.