D.10: Critiquing Institutional Narratives
“Constructing an Ethical Core: Centering Teachers Interpretations of Automated Assessment and the (In)flexibility of State Standards,” J.W. Hammond
“You Want It Done Right? Do It Yourself: Publishing a Digital Textbook to Engage Students,” Laural Adams
“Hammond will address the surprisingly little research exploring classroom teacher beliefs about the ethical significance of state standards and large-scale automated assessment, drawing on qualitative interview data to show that teachers rhetorically (re)interpret the meaning of large-scale standards and automated assessment in ways that align with local values and needs. Adams will discuss the process of publishing a tutorial-based, writing-intensive textbook designed to overcome challenges in digital learning environments that plague digital textbooks, including student poor cognitive processing, reduced effort, cheating, lack of fit with curricular goals, and student disengagement.”
Beginning the panel, Hammond, from the University of Michigan, discussed his research on standardized testing and automated scoring in K12 education. In his study, Hammond conducted semi-structured interviews with four student teachers, three mentor teachers, and three field instructors. Each of the interviews was focused around the question: What do you understand to be the promises and perils of the CCSS and its assessment? The interview data demonstrated that while teachers were all broadly concerned with the CCSS themselves, they were not necessarily all worried about the machine scoring for writing prompts. The perspectives on the machine scoring of writing prompts demonstrated a misalignment between expectations of K12 and expectations of postsecondary teachers. While postsecondary teachers were considered with the ethics of machine scoring itself, K12 teachers expressed more concern over the ethics of the medium of the tests. Hammond suggests that moving forward from these results, it is essential that we take multiple steps:
- First, we need to reframe automated essay scoring ecologically.
- Next, we need to continue research on the ethical perceptions of automated scoring.
- Finally, we need to build durable collaborations and professional partnerships between K12 and postsecondary teachers to resolve the misalignment.
Adams portion of the panel focused on her experience with publishing a digital textbook. She shared the issues that she, and likely many other instructors, have with implementing a pre-designed curriculum with a set textbook. After dealing with textbooks that never seemed to fit her needs or best serve her students, Adams decided to take a leap and publish her own textbook. In designing her textbook, she kept multiple aims in mind:
- Avoid policing the textbook work
- Avoid the lecture approach
- Devote time to designing classroom experiences
- Generate student engagement
- Assist instructors in developing course material to maintain ethical concerns of workload
- Create a semi-automated experienced for assessment of non-writing assignments
Adams also explained that she wanted to keep her textbook relevant to her student population, and project-, tutorial-, and practice-based.
Delivered completely online, Adams textbook allows students to learn through a flipped-classroom design. By reading, practicing, and articulating their findings at home, they are able to spend more time in class discussing and working on projects.
After implementing her textbook, Adams found many positive impacts and also a few places for improvement. Concerning positive impacts, Adams explained that students’ effort (with no feedback on written work) improved; students loved walking toward a finished deliverable with the project-based, tutorial approach; the assignments had real-world impact; guidance can be specific to them in the text; there was greater consistency in course content and strong student work in outcomes across sections; and there was a better understanding of the genre approach to communication. Unfortunately, Adams also found that most students still weren’t reading the textbook, the students sometimes viewed the activities as ‘busy work’, and that there are limits to what the publisher can do. Additionally, an interactive online textbook requires extensive technical support.
Final Thoughts
Leaving the panel, audience members had much to consider. Specifically, I am left with many questions about the implications of the presenters’ research in our work at postsecondary institutions:
- First, how does Hammond’s research shed light on the debate over standardized scoring for placement within postsecondary writing programs; should universities appoint placement readers to personally review each individual essay, or can ACT or SAT scores accurately represent a student’s writing abilities?
- Next, what message do we send my utilizing automated scoring within writing programs?
- Finally, Adams discusses the need to maintain a “vision” when publishing a text. How do we, as instructors, maintain our own “visions” when creating and implementing a curriculum with designated course texts? How do we know when our vision no longer stands or needs to be adjusted to better fit our students’ needs?
