Minnesota Online State Testing - Not Fair to Students

So, Minnesota's high-stakes online student testing (known as MCA's) kicked off on March 9th, but because of Spring breaks and the desire to have as many instructional days as possible before testing, most districts wait until later in the 8-week testing window (which is ridiculously long from a statistical point of view) to administer the tests. This week was the first "heavy" week of testing, though some started earlier. And, as many predicted, the online systems received a failing grade, but it's the students who really lose.

A talented technology coordinator from Milaca, Steve Bistrup, wrote a message to his administration about their experiences with testing thus far. Here is a sample of what he wrote - warning, it is a bit long, but I think well worth the read and sharing with others!!


Long Wait #1 During the mornings in particular, but not exclusively, students have to wait an unacceptably long time for Pearson's main "Self Registration" page to load. Staff are now having to come to work extra early to "pre-load" Pearson's main server page so that students don't have to wait the 5-20 minutes for the login page. This is extremely difficult when we potentially will be testing six sessions simultaneously. (This slowness is NOT a result of our network infrastructure or speeds... these have been ruled out conclusively by technicians and Pearson support).

Long Wait #2 Once students enter their credentials and test code there is another "randomly" long waiting period. We have seen students wait as long as 45 minutes for "the blue bar" to finally load their test questions. (Again, this is NOT a result of our caching server or internal network)

Two to four adults needed to start each session Because we are always having to restart the browser, try a different computer, empty the java cache, reset the browser, etc. until we can finally get a student logged in to the test, an unacceptable number of adults are needed in order to reduce the amount of time students are sitting in front of a spinning wheel while the person next to them is 20 minutes into their test. I can't imagine the amount of anxiety this creates in the students. It certainly creates a high degree of anxiety and frustration in those that desperately want the students to have a fair shot and do their best work. I recall one student who anxiously waited for his test to load....worried that if he had to wait too long he would miss outside recess. The teacher and I had to both reassure him to take his time once he was logged in.

Constant browser plug-ins updates and Pearson's sporadic warnings On 4/14/15 Oracle released a new version of Java. Because Apple computers and TestNAV require you to be on the latest version of Java, staff needed to stay long into the evening that night in order to update Java on every computer that students would be testing on the next morning. For Milaca this is 120 laptops and 90 lab computers. There was also a new version of Flash on this date. Pearson warned us about the Java update but not the Flash update. Luckily we caught it. I can't begin to count how many times Java and Flash have had updates since school started. Some of them Pearson informs us of...more often then not they don't.

Math Test Calculator Issues Pearson acknowledged this as a known issue during the OLPA session. We have now had the same issue during the MCA math sessions. When a student uses the built-in calculator during a math test it randomly locks up their test session. Pearson's only answer has been to "force quit the test and have the student log back into the test." *see points one and two why this is more than inconvenient. Worse, a student has to shut down and restart their testing mindset and momentum.

Server outage and support On 4/14/15 Pearson Access server crashed or "became degraded" across the entire State of Minnesota resulting in testing administrators being unable to manage test sessions. When I called I was put on hold and twice was disconnected with a busy signal. When Pearson eventually sent out a formal message the verbiage was disingenuous. The message made a point to emphasize that "TestNAV and Testing were not effected." Yes, for those kids that were currently in their test they were able to continue. However, no other session could be started or resumed resulting in entire testing sessions being missed or postponed.

Not a fair assessment for kids I have been involved with online testing since the very beginning and I can say that no testing period has been approached with more trepidation by those managing the testing. Unfortunately we have now seen those fears played out. These past two weeks have been frustrating and unfair to kids. If the testing mechanism is supposed to be as transparent as a paper booklet and as technologically easy as working a #2 Dixon Ticonderoga then this has been a colossal failure. I've watched kids put their heads down in frustration as they wait. I've seen students lose their focus as a result of the glitches. I've seen the worry on an instructors face as they watch all the preparation and build-up deflated in a matter of minutes.


Hi - I'm back! I have examples from many other districts too, but Steve says it well, so I wanted to share those experiences. (Congratulations, by the way if you've reached this part of the post!) I can say that not all schools have experienced all of the same issues, but that just goes to reinforce the idea that this is not a fair assessment for kids. In the MN Department of Education Assessment Update - released today 4/15/15 - They included the following message:

To reduce testing disruptions for students, take appropriate actions to reduce noise, such as limiting the use of alarms (not fire alarms), bells, and announcements. Districts should avoid scheduling test sessions during known interruptions, such as during lunch breaks or pre-planned fire or tornado drills.

What about the disruptions caused by all the issues Steve points out? If a district encounters those kinds of issues (NOT a result of anything they did or didn't do) can we really validate that data? Is it a fair comparison to previous years' results if the same issues were not present then? Can anyone feel good about evaluating the success of teachers, students and schools based on such inconsistent testing environments? Heaven forbid we compare districts to each other based on these results. Yet, that is exactly what happens.

The MN Department of Education publishes their "Minnesota School Report Card" on their website each year. Designed for the public to look at progress and demographic information about schools - and built to provide side-by-side comparisons. Front and center on each schools "report card" is proficiency data for the MCA tests for the past 5 years. No where does it indicate that you may be comparing a school where online testing worked flawlessly and one where students were subjected to the kinds of distractions and frustrations that Steve noted. Yes, I firmly believe there would be statistically significant differences. Not to mention you are looking at results from year to year when entirely different groups of students were tested - I'm sure there are no variables there that could influence validity! Plus (though I'll admit this is noted if you look carefully) standards have changed in all three tested areas in the past 5 years, which again, invalidates comparisons from year to year.

I'll stop now - you get the point about how unfair this situation is. Kudos to you if you made it this far! I'll probably break this up, condense it and repackage it in some messages to folks at MDE, legislators, etc. Feel free to share it with anyone you think has the attention span to read it!! And... stay tuned for more updates and even some thoughts about how to improve it.

Rural Broadband and Education - Part 3

Took a few days off - this takes more time than I'd like it too!

I said after the last post that this time, I would talk about how increased bandwidth changed the environment in a school. In May 2006, I took a job as the technology director for ISD 15 in St. Francis, MN. At the time, the school district of nearly 6,000 students, with six school buildings and several other service facilities was served by two T-1 circuits. That is the equivalent of about 3mbps. Now, 2006 was quite some time ago, but at the same time, the 10 school districts in ECMECC shared a gigabit fiber network and 200mbps of Internet bandwidth.

The fist thing I did in my new district was to go on a listening tour to hear from all district administrators, department chairs and program directors. I wanted to know what was great about technology in their school and what they saw as issues. You can probably guess that I heard about many more issues than I did positives. Over and again, I heard about all the things they couldn't do because the technology department told them there wasn't enough bandwidth to run the student information system, finance/payroll systems and many educational tools at the same time. The district had a subscription to a popular streaming video service but staff was told they couldn't stream the videos during class, instead having to schedule videos to download after hours. The service hardly ever was used. They were told that they should not expect to use Internet resources with a class in a computer lab as they would probably not work. Teachers complained about slow access to Internet resources and they could not use the electronic grade book and other features that were part of their student information system. Technology committee meetings, I heard, were frustrating exercises and the technology department and staff seemed constantly at odds.

Within a few months, the St. Francis district was connected to the ECMECC network and had access to as much Internet bandwidth as they could use. During the first part of the 06-07 school year, the bandwidth use climbed to nearly 10mbps - which doesn't sound like much by today's standard, but was over 3 times what they had access to only months earlier. While not all technology issues were solved by this, the conversations with staff and administration changed almost overnight. Technology planning went into full swing and the technology committee talked about what they could do. We implemented use of streaming video in the classroom and for athletic teams. Teachers began using Internet resources with their classes in labs. Projectors and whiteboards started showing up so resources could be used in classrooms. Discussion began about wireless access. Everyone was excited about technology in the school, a far cry from where they had been only a few months earlier.

I left the district only two years later, but the district is now a full member of ECMECC and has numerous technology initiatives that rival that of most school districts. For example, the district has a "bring-your-own-device" policy that sees thousands of mobile devices attached to their network every day. Students use the devices for course work and research. Technology goals set during the technology planning process are reached. The district is using interactive video conferencing to take students on virtual trips around the world to participate in interactive learning experiences. See this link for a story about IVC at St. Francis from WCCO-TV. Implementing unlimited bandwidth was a simple change that fundamentally altered the educational technology conversation throughout the district.

Up Next: This is an example of how things changed for a school district once they had no data cap (in this case related to speed). I believe data caps in place for many of the choices rural residents and business have for broadband create a similar situation where conversations are more about what can't be done than what can. We'll take a look at that issue next time. - Stay warm!

A little late to the party... but here we are.

One of my favorite episodes of the 1980's (and early 90's) sitcom, "Cheers" is when main character Sam Malone, the "once was" major league pitcher now tending bar in a small watering hole in Boston "where everybody knows your name," is given the opportunity to fill in as a TV sports reporter. Part of his duties are to do a nightly commentary called "I on sports." He proceeds to provide captivating discourse on such controversial topics as... "cheering for the home team" and "natural grass or artificial turf." His ticket back to bar tending is validated when he turns to a rap and ventriloquism on back to back broadcasts.

Why do I mention this? Well, I'm certainly late to the blog party. Hey... despite years of work in education technology, I'm not always an early adopter! Despite the tardiness, I hope NOT to be one of those people who thinks they have something to say but, in reality, has no business, "doing the sports news on TV." What I DO hope is that I can provide engaging and perhaps, once in a while, entertaining information about technology in the K-12 schools. I may cover the role of traditional IT in the school environment, new and interesting instructional applications, administrative issues such as funding or staffing, and maybe an opinion or two.

I expect to invite some "guests" to write at times and I invite comments and feedback of all kinds. We'll get this party going full steam next week. In the meantime, enjoy this set of clips from Cheers - and, "Cheers"!!