Presenters
Mike McLeod, Michigan State University
Ann Shivers-McNair, University of Washington
Jeff Grabill, Michigan State University
Review
Mike McLeod, Ann Shivers-McNair, and Jeff Grabill talked and asked for feedback about their work with Eli Review in their “Tools for Writing Researchers by Writing Researchers” panel at the 2016 Computers and Writing Conference.
Shivers-McNair presented first, describing her need for a tool like Eli Review. For instance, she talked about how she realized she needed to better understand the peer review process in order to better communicate its purposes to students. Using Eli Review as a tool for collecting students’ assignments, she began to ask the question “what if Eli could systematically study peer review data?” and brought those kinds of questions to the Eli Review staff. This feedback contributed to the development of the option in Eli Review for data export.
Grabill talked next. He focused on the business side of Eli Review and the decisions concerning access that were made. He used this time to elaborate on some of the ideas he had talked about in his Friday evening keynote. One such idea was the stupidity of trying to separate the technology from pedagogy. Pedagogy and technology, Grabill reminded, if situated in a positive relationship, cannot and should not be separated; good technology should necessarily influence pedagogy and an open pedagogy should drive technological needs. Grabill, too, spent some time talking about access. He noted that Eli Review’s data is not open access. Proposals for research must be submitted and reviewed. This approval process also means, though, that those at Eli Review can tailor data exports to specific research projects.
Grabill also was sure to talk about the ownership of the students’ work and the protection of anonymity. He noted that Eli still has important ethical questions to answer. Since Eli is already being used in high schools and middle schools, for instance, what happens if middle school students identify their location in their writing projects by using name of school in writing? Shivers-McNair came back into the conversation to talk about Eli’s attempt to begin to answer those ethical questions. Those at Eli looked to existing corpora (MICUSP, PSU C-SAW, USE, and BAWE) and how those corpora dealt with informed consent. Shivers-McNair also talked about asking several linguists about what Eli should be called: is it a corpus or a repository?
Then, discussion turned to incentivizing the use of Eli. How can Eli both compel instructors to use the program and also compel students to share their writing? Existing corpora have different methods for providing that incentive. Some offer students modest payment, others offer something useful to the student in exchange for their writing, and some even share any research that is done that uses corpus data.
In his presentation, Mike McLeod (Eli’s head of product) discussed similar practical considerations about Eli’s use and gave the data about how often Eli is already being used. He noted that since 2012 (when Eli was last at the Computers and Writing Conference), Eli has seen 28,799 task prompts in the program, 242,944 “composed in Eli” submissions, 95,654 uploaded files, and over 1 million comments. McLeod then talked about the three primary challenges for an opt-in corpus. One is primarily a hardware question: “How do we [Eli] isolate and protect the shared data?” Going along with this challenge, though, is one about IRB-compliance. If Eli wants to be used for research, after all, it needs to be compliant to researchers’ home IRBs. The final challenge is one of interface: “How do we [Eli] persuade people to contribute?”
Before using the session as a time to get feedback from instructors on the interface, McLeod described the thought process that went into creating the current interface. He noted how important student comfort was, describing student restraint to share less-than-exceptional work with Eli. Recognizing that discomfort, McLeod talked about the importance of immediately recognizing the students’ wishes: “If they [students]say ‘no’ once [to sharing work with Eli], we won’t bug them anymore.”
After this talk about the thought behind the interface, session attendees were asked to view the interface and then complete a survey to get real-time, immediate feedback. McLeod, Grabill, and Shivers-McNair then talked with attendees about the results of the survey. They noted that this kind of interaction is precisely the kind that Eli is trying to achieve–immediate feedback for instructors and researchers about Eli Review.
