Being Clear on “High Quality”

Some readers are misinterpreting my critique of the phrase “high quality” as it’s used with regard to textbooks and other educational resources. Let me reiterate my points and then provide a new example that hopefully sheds more light on what I’m criticizing and what I’m not.

My problems with the phrase “high quality” are two-fold: (1) how the phrase gets equated with a single authoring process to the exclusion of all other authoring processes, and (2) how that usage distracts us from the efficacy conversation.

I do not have any issues with the traditional authoring process itself. As I wrote yesterday, “I fully believe that resources created through the ‘traditional process’ can effectively supporting learning.” Not only do I not have an issue with the traditional process itself, I don’t have an issue with the people or organizations who employ the traditional process – in fact I count some of them as good friends.

To be clear, my first issue is with the way “high quality” is often equated with the traditional process and that process only. According to this usage, if you don’t follow the traditional authoring process it is literally impossible for you to create “high quality” materials. This restrictive usage serves to lock out alternative processes from competing in the marketplace. The usage makes it impossible, for example, for commons-based peer production models to result in “high quality” resources (because “high quality” was previously defined as “adhering to the traditional process”). That’s just plain wrong. We need to create more openness to alternative authoring models in the minds of faculty. The traditional process can create effective learning materials, but so can other processes. These alternative processes likely share some common functions with the traditional process – like quality reviews – but the specifics of how those functions are performed can differ significantly across processes (e.g., focused review by a smaller number of experts versus ongoing review by a much larger number of peers) and still result in effective educational materials.

My second issue is the way this misuse of “high quality” omits any consideration of results, and tends to distract us from the efficacy conversation we need to be having. As this series of posts demonstrates, “high quality” can mean many things. But for the sake of our students, the one thing we most desperately need it to mean is “effective.” It’s quite simple – in order to be considered “high quality” educational materials must be effective. We need to help faculty understand that the phrase “high quality” is often disconnected from measures of efficacy when used by publishers to describe textbooks and other instructional materials, and we need them to help bring effectiveness back into the conversation.

Let me give an example to clarify exactly what I’m calling out and exactly what I’m not.

Take OpenStax as an example. These guys are doing God’s own work, creating a catalog of open textbooks for high enrolling courses using the most open of all the Creative Commons licenses. They seem to use a fairly traditional authoring process, and that’s absolutely fine. Again, the process that any person or organization uses is entirely outside the scope of this critique.

What is absolutely relevant to the critique is this – I’ve never heard OpenStax badmouth OER that were created using other processes solely because they were created using alternative processes. Even though they appear to use a fairly traditional process internally, they don’t try to perpetuate the “high quality if and only if traditional process” myth. As you would expect from leaders in the open educational resources community, they’re open minded about the possibility of other authoring processes resulting in effective materials.

The second point to make in the context of my critique of the way “high quality” gets misused is that OpenStax actually cares about efficacy. In fact right now there’s a huge banner on their homepage inviting people to participate in an efficacy study being conducted by a highly respected third party (the OER Research Hub). OpenStax aren’t trying to distract people from the efficacy conversation; on the contrary, they’re dedicating homepage real estate to trying to get people more involved in the conversation.

I provide the OpenStax example in order to clarify that (1) I’m not criticizing the traditional authoring process itself, and (2) just because an organization uses a more traditional authoring process doesn’t mean that they’re making the mistakes that I think the field at large needs to stop making. My critique of “high quality” is simply that it is being defined far too narrowly – unnecessarily excluding alternate authoring processes and omitting effectiveness. Perhaps rather than needing to stop using the phrase “high quality” we instead need to engage in a campaign to “redefine high quality” so that the phrase emphasizes efficacy and is more accepting of diversity in authoring processes.

2 thoughts on “Being Clear on “High Quality””

  1. I’m personally glad David is able to read my mind and tell me what I mean when I refer to educational materials as “high quality.” Or that he is enlightened enough to lump my intentions with those of commercial publishers who also use the term. And I’m sure that these posts are in no way attempts to control the language and, therefore, control the argument. 🙂

    While I agree that “high quality” is a vague reference akin to “middle class,” “effective” is only slightly more clear. To some, a resource is effective if it reaches underprivileged students who haven’t had such resources in the past. To others, an effective resource is one that yields similar outcomes, but costs much less. But, even in these cases, “effective” is only meaningful in contrast to more or less effective resources. Ergo, a resource cannot be “effective” in isolation.

    In the end, I wonder if this discussion isn’t just kicking up dust, and then complaining that we can’t see.

  2. Thanks David. Perhaps some readers who don’t have personal experience working within the “the traditional authoring process” will find this information graphic, “What goes into making a textbook”, an insightful frame of reference _ it describes the contributions in talent & time invested in the creative and production processes of a widely recognized “high quality” resource _ quoting “Bruce Hildebrand, executive director for higher education at the Association of American Publishers, noted that more than 10 man-years of labor [30,000 hours] went into publishing the 7th edition of Biology by Neil Campbell and Jane Reece. This doesn’t include the work of producing the supplemental materials,” _ Source: The Text and Academic Authors Association [TAA] debunks the top 7 myths regarding textbook costs. _ http://blog.taaonline.net/wp-content/uploads/2013/12/TAAmythsflyer.pdf __

    I recent read that OpenStax has been contracting-out work to educational resources developers [Words&Numbers, I believe], and their positive experience will likely prompt others in the field to seek additional funding for their OER project budgets that will enable outsourcing editorial-creative work to professionals. FYI, I found the link you pointed to on _ https://openstaxcollege.org/ _ in now closed to the OER Research Hub survey.

Comments are closed.