Minnesota MCA Testing Woes Continue

After issues on April 14th and 15th, glitches cropped up again today (4/21) in Minnesota's high-stakes online MCA tests. This time, nearly every district in the state experienced some issues. At least one local news organization has picked up on the problems. I have feeling more will follow. (And they have - Pioneer Press - WCCO - In fact, since I started writing this post, MDE has suspended all testing until further notice)

The MN Department of Education (MDE) and the testing vendor, Pearson held a joint conference call on Monday (4/20). During the Q & A, dozens of MN districts reported various issues that caused, at minimum, frustrating delays in starting test sessions for students and at worst, lost data. What was clear was that these issues were not isolated to specific tests or geographic location of the schools and they were not usually caused by local school district network or setup problems.

So, bring on today (Tuesday 4/21) and by 8:45 this morning, Pearson and MDE acknowledged that some of the systems were in a "degraded" state resulting in delays. For about 2.5 hours in the middle of the day, the systems were reportedly working, before going back into a degraded state and eventually becoming unresponsive. (my guess is it worked during the lunch hours when few testing sessions were being held.)

If you've read my previous posts, you know that I have questions about the validity of a testing system that introduces variables of anxiety and frustration. I've conveyed that message to many people including officials at MDE and as you saw above, MDE has now suspended testing citing some of the same concerns I and many others have pointed out. Also of note is that these issues have affected testing in other states as well.

I did find it interesting that one of the responses I received from an MDE official pointed to me to a study that had been done after the 2013 Spring testing season when similar issues occurred with the previous testing vendor, American Institutes for Research (AIR). After taking a brief look, I recalled that this was used to calm the concerns over validity as the study generally indicated that there was no significant effect to students who experienced the issues on those days in April 2013. However, there are significant problems with the study and it so narrowly examined ONLY the specific issue that occurred on those days that it certainly has NO correlation to what we are experiencing now. I'll probably do a separate post about that study - and I have a general post about testing coming up too. Check back soon if you think you might be interested in either of those ramblings!

Minnesota Online State Testing - Not Fair to Students

So, Minnesota's high-stakes online student testing (known as MCA's) kicked off on March 9th, but because of Spring breaks and the desire to have as many instructional days as possible before testing, most districts wait until later in the 8-week testing window (which is ridiculously long from a statistical point of view) to administer the tests. This week was the first "heavy" week of testing, though some started earlier. And, as many predicted, the online systems received a failing grade, but it's the students who really lose.

A talented technology coordinator from Milaca, Steve Bistrup, wrote a message to his administration about their experiences with testing thus far. Here is a sample of what he wrote - warning, it is a bit long, but I think well worth the read and sharing with others!!


Long Wait #1 During the mornings in particular, but not exclusively, students have to wait an unacceptably long time for Pearson's main "Self Registration" page to load. Staff are now having to come to work extra early to "pre-load" Pearson's main server page so that students don't have to wait the 5-20 minutes for the login page. This is extremely difficult when we potentially will be testing six sessions simultaneously. (This slowness is NOT a result of our network infrastructure or speeds... these have been ruled out conclusively by technicians and Pearson support).

Long Wait #2 Once students enter their credentials and test code there is another "randomly" long waiting period. We have seen students wait as long as 45 minutes for "the blue bar" to finally load their test questions. (Again, this is NOT a result of our caching server or internal network)

Two to four adults needed to start each session Because we are always having to restart the browser, try a different computer, empty the java cache, reset the browser, etc. until we can finally get a student logged in to the test, an unacceptable number of adults are needed in order to reduce the amount of time students are sitting in front of a spinning wheel while the person next to them is 20 minutes into their test. I can't imagine the amount of anxiety this creates in the students. It certainly creates a high degree of anxiety and frustration in those that desperately want the students to have a fair shot and do their best work. I recall one student who anxiously waited for his test to load....worried that if he had to wait too long he would miss outside recess. The teacher and I had to both reassure him to take his time once he was logged in.

Constant browser plug-ins updates and Pearson's sporadic warnings On 4/14/15 Oracle released a new version of Java. Because Apple computers and TestNAV require you to be on the latest version of Java, staff needed to stay long into the evening that night in order to update Java on every computer that students would be testing on the next morning. For Milaca this is 120 laptops and 90 lab computers. There was also a new version of Flash on this date. Pearson warned us about the Java update but not the Flash update. Luckily we caught it. I can't begin to count how many times Java and Flash have had updates since school started. Some of them Pearson informs us of...more often then not they don't.

Math Test Calculator Issues Pearson acknowledged this as a known issue during the OLPA session. We have now had the same issue during the MCA math sessions. When a student uses the built-in calculator during a math test it randomly locks up their test session. Pearson's only answer has been to "force quit the test and have the student log back into the test." *see points one and two why this is more than inconvenient. Worse, a student has to shut down and restart their testing mindset and momentum.

Server outage and support On 4/14/15 Pearson Access server crashed or "became degraded" across the entire State of Minnesota resulting in testing administrators being unable to manage test sessions. When I called I was put on hold and twice was disconnected with a busy signal. When Pearson eventually sent out a formal message the verbiage was disingenuous. The message made a point to emphasize that "TestNAV and Testing were not effected." Yes, for those kids that were currently in their test they were able to continue. However, no other session could be started or resumed resulting in entire testing sessions being missed or postponed.

Not a fair assessment for kids I have been involved with online testing since the very beginning and I can say that no testing period has been approached with more trepidation by those managing the testing. Unfortunately we have now seen those fears played out. These past two weeks have been frustrating and unfair to kids. If the testing mechanism is supposed to be as transparent as a paper booklet and as technologically easy as working a #2 Dixon Ticonderoga then this has been a colossal failure. I've watched kids put their heads down in frustration as they wait. I've seen students lose their focus as a result of the glitches. I've seen the worry on an instructors face as they watch all the preparation and build-up deflated in a matter of minutes.


Hi - I'm back! I have examples from many other districts too, but Steve says it well, so I wanted to share those experiences. (Congratulations, by the way if you've reached this part of the post!) I can say that not all schools have experienced all of the same issues, but that just goes to reinforce the idea that this is not a fair assessment for kids. In the MN Department of Education Assessment Update - released today 4/15/15 - They included the following message:

To reduce testing disruptions for students, take appropriate actions to reduce noise, such as limiting the use of alarms (not fire alarms), bells, and announcements. Districts should avoid scheduling test sessions during known interruptions, such as during lunch breaks or pre-planned fire or tornado drills.

What about the disruptions caused by all the issues Steve points out? If a district encounters those kinds of issues (NOT a result of anything they did or didn't do) can we really validate that data? Is it a fair comparison to previous years' results if the same issues were not present then? Can anyone feel good about evaluating the success of teachers, students and schools based on such inconsistent testing environments? Heaven forbid we compare districts to each other based on these results. Yet, that is exactly what happens.

The MN Department of Education publishes their "Minnesota School Report Card" on their website each year. Designed for the public to look at progress and demographic information about schools - and built to provide side-by-side comparisons. Front and center on each schools "report card" is proficiency data for the MCA tests for the past 5 years. No where does it indicate that you may be comparing a school where online testing worked flawlessly and one where students were subjected to the kinds of distractions and frustrations that Steve noted. Yes, I firmly believe there would be statistically significant differences. Not to mention you are looking at results from year to year when entirely different groups of students were tested - I'm sure there are no variables there that could influence validity! Plus (though I'll admit this is noted if you look carefully) standards have changed in all three tested areas in the past 5 years, which again, invalidates comparisons from year to year.

I'll stop now - you get the point about how unfair this situation is. Kudos to you if you made it this far! I'll probably break this up, condense it and repackage it in some messages to folks at MDE, legislators, etc. Feel free to share it with anyone you think has the attention span to read it!! And... stay tuned for more updates and even some thoughts about how to improve it.

Rural Broadband and Education - Part 4

So... how does a lack of affordable broadband access in rural areas affect a typical family? Let me give you an example (based on real people and experiences)...

This "rural" family lives only a few miles outside of a community of over 3,000 people. The family is composed of two tech-savvy parents and two high school students with four smartphones, a tablet device, and two laptop computers. Statistically, a fairly average situation. Available options and approximate costs (real rates and NOT special introductory pricing) for this family to consider include:

• Cellular data access available from two carriers, one offering 4G technology at about 8mbps download speed. Pricing is based on data usage (data caps) with an average of $10 per GB/mo with a $15 per GB overage charge. Additional monthly costs are incurred for each connected device and/or a shared hotspot device.
• Fixed wireless service offering speeds up to 5mbps down. Costs include equipment (about $300), setup (about $150), router (about $75) and a monthly charge of up to $100 for 5mbps speed (slower options are available at a lower monthly rate). 
• Satellite service offering speeds up to 12mbps down. Costs include setup fee (about $150), equipment rental (about $10 per month) a two-year service agreement and a monthly charge of $50 (with a 10GB data cap) to $130 (with a 25GB data cap).
• DSL service from the local telephone company offering a speed of up to 1.5mbps (however, in their location it is more likely to be about 768kbps) with a monthly charge of $35 month with equipment rental and a landline (more if no phone line is included)
• Dialup service is available, but doesn't even come close to the lowest definition of broadband.

Given those options, what would your choice be and what would the conversations in your household sound like? How would you feel knowing that if you lived in town 3 miles away, you would have access to 30mbps speeds with no data cap for less than $50/mo or 7mbps with no data cap for less than $40/mo?

This family does not live "in the middle of nowhere." With the options they have, they cannot reliably stream any kind of video, be it for entertainment or educational purposes, without the risk of high cost for higher data consumption or overages. They are forced to carefully monitor their Internet usage and plan around time at work and school to update apps on mobile devices, run updates on computers, and do other bandwidth intensive activities. Most options don't provide reliable use of communications technologies such as Skype or Google+ due to high latency or slow speed. All of this forces the discussion to be about what can't be done rather than focusing on what can (or needs to) be done.

Why does this matter? We make choices when we decide to live where we do, right? Consider the increasing use of laptop computers and mobile devices in schools. How useful is your computer, tablet or smartphone when it isn't connected to a high speed, Wi-Fi network? What goes through your mind when you are away from home and staying in a hotel with poor wireless access? Students access class work online through web portals and learning management systems. Homework involves accessing information on these sites and communicating with teachers and classmates online. Interactive electronic textbooks and other online learning resources are fast becoming the norm in education. Is it fair to put students without this access at an immediate disadvantage, forcing the students and teachers to figure out accommodations? Would it be fair to all students NOT to use these technologies if they make education better?

Well, this only affects a small population, right? Overall, a majority of people in this nation have access to high-speed broadband, but that doesn't mean it is affordable. Costs can vary wildly, which adds to the divide we are creating. Providers lure customers in with artificially low (loss-leading) introductory pricing and then costs rise dramatically after a year or two. Additionally, consider that nearly 75% of the population in Kanabec County (Minnesota) lives outside of the counties communities and, therefore, has very similar options to those above. There are many other parts of this country that are similar.

Broadband access is beginning to define the haves and have-nots. It is the new digital divide. We will not stop the movement of voice, video and print resources online. Soon, land-line phones, DVDs (and Blu-ray), CDs, books, magazines and newspapers will be a thing of the past just like video tapes, records, and encyclopedias. If we don't become concerned about how to make affordable high-speed access ubiquitous, we will put a whole subset of our nation's population further behind.

I'll be taking a break from writing about this topic (but keep reading for other ramblings!) until after the "Boarder to Boarder Broadband: A Call to Action" conference coming up February 4-5, 2014 in St. Paul, MN. I'm hopeful that there will be concrete action items that come out of the conference that I can write about. If you are interested in attending, find more information at:  https://www.regonline.com/builder/site/Default.aspx?EventID=1360663 

Rural Broadband and Education - Part 2

How much bandwidth is enough?

This is a graph showing the average amount of Internet bandwidth consumed in an average week by the 42,000+ students and 2,000+ instructional and administrative staff in the 14 ECMECC member school districts that I work with.

If you look closely, you can see that we are using about 1.0gbps (that's 1000 megabits per second) during peak times. Eight years ago, we served most of these same schools with 10mbps - that's 1% of what we are using now -- and is similar to the speed of a "basic" DSL or Cable connection in a non-metro community. Today, our schools use 100 times the bandwidth they did eight years ago on a daily basis. And... the number continues to climb. So, what are students and staff doing with all this bandwidth, and what is enough?

There are people who will have us believe that kids are watching videos, playing games and engaging in social media activities while in school. I would say that they are absolutely right, students, are, indeed, doing all of those things. They would say this is a waste of time and bandwidth. I would say it is the way students learn today. Recently, a letter to the editor appeared in our local paper from a writer who believed we should eliminate computers (and presumably mobile devices - though they didn't say that specifically) from our schools since they detract from what they believe is the best way to learn. At a recent meeting I attended organized by Senator Matt Schmit, a participant suggested that people in rural areas and communities don't need "that much" bandwidth since their kids will just stream movies, play games and spend all their time on "that Facebook." Once again, I contend that this is now how people access this kind of content and communicate. It would be like going back 20 years and saying some people don't need a VCR or a phone since they will just play movies and talk.

Think back 10 years... How many places could you find in your area to rent a DVD (or buy one)? Today, how many are there? While rental stores have been replaced to some degree by Netflix (mail-order) and services like Redbox, there is no disputing that the options for renting and buying physical content (DVDs) are diminishing and will likely vanish in the not-to-distant future. The same is true for music (CDs). Now, music and video content is streamed online, and it takes a robust Internet connection to get good quality. Of course, that's entertainment. What about education?

Again, take yourself back 20 years or so and think about how you "learned." Current events information may have come from newspapers, broadcast television or local radio. Remember the trip to the public library or a bookstore to find a book about something that you wanted to learn about? Remember when you needed a manual to figure out how to replace the headlight in your car? In school, you used textbooks that may have been published as long as 10 years or more prior to your use and films that were produced in years gone by.

Today, the number of daily newspapers being published in the United States is down 25% from 20 years ago and readership numbers have fallen off exponentially. Broadcast television and radio is struggling to keep viewers and listeners and has less and less unbiased "news" content. Physical books and manuals have been replaced by ebooks and videos. Heck, most encyclopedia companies don't make encyclopedias anymore (though World Book still does!). We don't access information or learn like that anymore. All of this content has shifted online and will continue to do so.

Accessing that information takes bandwidth in larger and larger amounts. There is a seemingly endless amount of rubbish on YouTube, but it is also the first place I go now when I want to know how to fix or do something. News organizations use YouTube to post content and there are just as many educational videos as cat videos (OK, maybe almost as many as cat videos!) See the Khan Academy for just one example of educational content on YouTube. Without high-speed, affordable Internet access students and families are immediately at a disadvantage. What is enough? I don't know, but I do know that a large number of Minnesotan's (and I'm sure others in the US) don't have adequate access to learn and live in the world of Internet delivered content. People living in high-density urban areas keep getting improved Internet speeds while those in rural areas pay more for less and see little speed improvement. As I said in my previous post, it is our new digital divide and we need to find a solution.

Up Next: How increased bandwidth fundamentally changed the education environment of a school district and how it could change a families life.

Rural Broadband and Education - Part 1

Starting a multi-part series of posts about some of the issues around broadband access in rural areas and education. I've been working for several years with a variety of rural broadband initiatives including the Kanabec Broadband Initiative (KBI) and the East Central Broadband Initiative (ECBI). These groups, and others like them, continue to educate the general public about the importance of broadband, and work to bring more broadband options and affordability to rural parts of Minnesota.

It is no secret that with the proliferation of mobile devices, 1-1 initiatives and migration to cloud based services for productivity and curriculum resources, schools rely on broadband more than ever, and there is no looking back. Unfortunately, many of our students, especially those living in rural areas outside of communities, do not have adequate, affordable broadband access. This puts them at an immediate disadvantage in the new technology-laden education environment. We have a new divide to conquer. Of course, the lack of quality, affordable access in rural areas poses a number of problems from economic development to quality of life to healthcare, to name a few. My focus, however, continues to be on education.

To that end, I have participated in some community meetings recently, arranged by MN Senator Matt Schmit and sponsored by various community organizations. Senator Schmit has taken on the issue of rural broadband and spent time in November and January touring the state to hear about the needs of rural Minnesotans. I had the opportunity to attend two of the meetings in Mora and North Branch to represent the schools in the area. In both cases, I shared some of the initiatives going on in area schools and talked about how schools are limited to what they can expect from students using technology outside of the school buildings. Our schools have very high speed access and do not have speed or data usage caps. I shared that this has changed the way schools talk about technology. Our schools do not talk about what they can't do (because of low bandwidth), they talk about what they CAN do. Sadly, that is not the case with many families living in rural areas. With slow speeds, data caps and expensive overage charges, these families still talk about what they CAN'T do.

I will bring this message to the Border to Border Broadband conference on February 4th and 5th in St. Paul. Senator Schmit has invited me to participate with him on a panel to share the messages he heard around the state during his listening tour. I invite anyone interested in the rural broadband divide to attend this conference supported by the Blandin Foundation.

Up Next: I'll share some of the comments heard from the meetings around the idea of "how much bandwidth (or speed) is enough."

A little late to the party... but here we are.

One of my favorite episodes of the 1980's (and early 90's) sitcom, "Cheers" is when main character Sam Malone, the "once was" major league pitcher now tending bar in a small watering hole in Boston "where everybody knows your name," is given the opportunity to fill in as a TV sports reporter. Part of his duties are to do a nightly commentary called "I on sports." He proceeds to provide captivating discourse on such controversial topics as... "cheering for the home team" and "natural grass or artificial turf." His ticket back to bar tending is validated when he turns to a rap and ventriloquism on back to back broadcasts.

Why do I mention this? Well, I'm certainly late to the blog party. Hey... despite years of work in education technology, I'm not always an early adopter! Despite the tardiness, I hope NOT to be one of those people who thinks they have something to say but, in reality, has no business, "doing the sports news on TV." What I DO hope is that I can provide engaging and perhaps, once in a while, entertaining information about technology in the K-12 schools. I may cover the role of traditional IT in the school environment, new and interesting instructional applications, administrative issues such as funding or staffing, and maybe an opinion or two.

I expect to invite some "guests" to write at times and I invite comments and feedback of all kinds. We'll get this party going full steam next week. In the meantime, enjoy this set of clips from Cheers - and, "Cheers"!!