5 Education Data Obstacles States Face

As more states place student learning results in schools' hands, they deal with challenges and share how they overcome them.

by / February 6, 2012 0

As the federal government emphasizes tracking educator and student progress through data, states build the capacity to collect and act on education information.

But on the road to a successful outcome, states are facing roadblocks that threaten to hinder their progress. Along the way, they are learning some valuable lessons.

The technical challenges

One of the biggest challenges Oregon faces is determining what information educators need. By starting the Oregon Direct Access to Achievement Project with a federal grant, Oregon answered this question by training educators to make data-driven decisions.

Although most of the technical challenges have been resolved, there’s at least one remaining that’s plaguing several states, and that’s identifying students among K-12, post-secondary and workforce systems, said Ben Passmore, assistant vice chancellor of the University System of Maryland. For example, Oregon doesn’t collect Social Security numbers at the K-12 level, and that’s what the workforce needs to link the two systems together.

"Now it's a question of people and processes, and those are in a lot of ways much more difficult challenges because they will actually force us to change how we do business."

States need educator feedback

The way Georgia does business does not involve state mandates or laws that require school districts to use the longitudinal data system. As a result, no one spent time arguing about whether they should or shouldn’t have to use it. Instead, they figured out how to use the system.

"The key is making sure that your user base has confidence in what you're doing and that they have complete control," said Bob Swiggum, CIO of the Georgia Department of Education. "If they're not right there buying into the process and giving you all the feedback you need to develop a good system, you're not going to have a system that anybody uses."

The Education Department built a tunnel between each district's student information system and the state longitudinal data system. Through tunneling, teachers and administrators log in to an environment they're comfortable in with their local ID and password. A separate portal shows parents how their students are performing.

The importance of role-based data access

Of 10 state policy actions the Data Quality Campaign – a collaborative effort of over 50 organizations to improve education data quality –– recommends, two of them are proving particularly difficult to achieve.

New Hampshire and Arkansas completed every component of action No. 5, which balances access to information with student privacy.

Six or seven years ago, the New Hampshire Legislature required the state’s Education Department to start gathering student-level data.

"This isn't about collecting the data for the state to do reporting," said department consultant Mike Schwartz. "It's not about collecting data for the feds. It's about helping schools improve and empowering schools to be more excellent."

Through a Performance Tracker assessment and reporting tool, New Hampshire showed educators what each assessment measured and what conclusions they could draw from it, said Ginny Clifford of the department's Bureau of Credentialing.

Oregon finished some of the components. For now, though, the state doesn't plan to change its model of having a security administrator in each district manage data system access.

"Our architecture has really been local control, and we don't want to get into providing access at a state level to students and teachers and parents because we can't manage the roles," said Doug Kosty, assistant superintendent of the Oregon Education Department.

But role-based access is one of three steps Arkansas recommends to provide quality data to educators and policymakers.

"Make it current, make it useful, and make sure that you can do that on a role-based system," said Jim Boardman, assistant commissioner of the Arkansas Department of Education's Division of Research and Technology.

With the online community "hive," Arkansas educators and policymakers are comparing aggregate results of classes and schools on standardized tests. They post scatter plots, bar charts and bubble charts of the results and comment on what others are posting.

How 2 states prepare, evaluate educators

Only three states completed every element of the Data Quality Campaign’s No. 9 action: North Carolina, South Carolina and Florida. These states incorporated effective data use into professional development, credential requirements and feedback to colleges of education.

North Carolina

Through the Education Value-Added Assessment System from North Carolina-based company SAS, districts see student test results over the past three years and identify students at risk of dropping out. Professional development — including SAS training, professional learning community discussions and monthly regional roundtables — prepares educators to evaluate student data.

Instead of making decisions based on end-of-year tests, policymakers should include student work samples, formative assessments and interviews with students, cautioned Elissa Brown, director of teacher and leader preparation programs for the North Carolina Department of Public Instruction.

"While data is quantitative in nature, the nature of the teaching and learning enterprise is much more quasi-experimental and much more complex. And data can tell you quite a few things, but it's not the whole picture."

South Carolina

In South Carolina's certification requirements, every educator needs to know how to analyze and use data to change their instruction. The state education department set 10 performance standards each for educators and principals, including effective data use.

From college to career, professors, mentors and schools measure educators against performance standards. These standards give educators a target to hit and show colleges of education and schools how they're doing, said Mark Bounds, deputy superintendent for South Carolina’s Division of School Effectiveness.

By analyzing data, teachers can improve instruction, and more importantly, improve learning.

"I think this is just the beginning of how we're going to use data to inform instruction and make sure that we have the best quality teachers in our classrooms," Bounds said.

Tanya Roscorla Former Managing Editor

Tanya Roscorla covered ed tech from 2009-2017.