Image by Freepik www.freepik.com
Couldn’t I leverage this technology to instruct students to write better, or better yet, get the AI to teach my students how to write?
In previous posts, I explained how I was using AI to create lesson plans, construct ‘at level’ readings and create activities for classes. I even explained that I was going to allow my students to use the AI app of their choice for the written portion of their exam. So, what’s good for the goose is also good for the gander and I decided that I could use AI to evaluate their AI generated responses.
It’s a bit of a process to have AI do an assessment, though not impossible. As I explained earlier, the clearer your instructions are to the AI, the better the results of AI will be for you.
So, here’s what I did. The student’s second exam this semester was a partner shared information exam. This is also known as a jigsaw puzzle exam. Anyway, one partner, A, in this case had information that partner B needed and vice versa. At the end of the exam, the students are supposed to combine their knowledge to create a written work that shows what they learned. The exam topic this time was about fair use copyright and the consequences of copyright infringement. The foci were fan made works that infringed on copyright, such as the Unofficial Bridgerton Musical, Holy Musical, Batman!, and Star Trek movies made by fans.
The written portion exam contains the following prompt and assessment criteria:
Instructions: So, your old friend Toño Ñoño has finally gotten his cars back from the police. It was hard on him to say the least. While they were gone Toño had a great idea. He wants to make a fanmade musical movie about a group of criminals who run around stealing things while racing their cars. He wants to call it the Revenge of the Rapido y Furioso, a bandarancheroreggaeton KPop musical. The music would be based on the songs that are famous right now in those genres. The musical would be set in outer space and feature his gang of car drivers as they attack a giant death planet and use The Force to save the universe from the Empire. His sister Niña Ñoño will be helping him write the music and even act in it. Toño knows you and your partner have just learned a lot about fan made films, musicals, and other things, so he wants your advice. He thinks he can make lots of money with his idea. What do you think? Write a letter to Toño and explain how he can stay out of financial trouble or how he might actually be able to make money on this, uh, idea. A minimum of 200 words please. Toño wants details. If you use AI to answer this question, you must specify which service you use.
Assessment Scale: Did you follow instructions? 20%. Grammar…it´s suggested you make use of conditionals and maybe imperatives. 20% Structure. Complete sentences? 20%Punctuation. Do you use periods instead of commas? Capitalize where appropriate? 20% Content. Do you answer the question? 20%.
Only two of the 45 or so students I have chose to write an original piece. The rest opted to use either Chat GPT (the majority) or Gemini (2) or Copilot (1). The returned letters were all written and contained the same basic information. Stylistically, the letters showed few differences, but I noticed that earlier letters from Wednesday’s exam weren’t as clear or concise as the ones from Thursday. I am going to have to go back and look some more and make sure it wasn’t my imagination. And if it wasn’t my imagination, that means that Chat GPT had a very small sample group from which to begin making improvements in its writing. (I will be covering this aspect how this AI improves on its efforst in another blog post).
Once all the exams were closed, I began assessing the writing portion. To do that I created a template document in Google docs with specific assessment instructions to Google Gemini. I then pasted the student’s text into that document, then copied the entire document and pasted it into Gemini’s task window. Here are the instructions
Evaluate the letter that follows and evaluate based on the instructions applying the scale at the end
Instructions: So, your old friend Toño Ñoño has finally gotten his cars back from the police. It was hard on him to say the least. While they were gone Toño had a great idea. He wants to make a fanmade musical movie about a group of criminals who run around stealing things while racing their cars. He wants to call it the Revenge of the Rapido y Furioso, a bandarancheroreggaeton KPop musical. The music would be based on the songs that are famous right now in those genres. The musical would be set in outer space and feature his gang of car drivers as they attack a giant death planet and use The Force to save the universe from the Empire. His sister Niña Ñoño will be helping him write the music and even act in it. Toño knows you and your partner have just learned alot about fan made films, musicals, and other things, so he wants your advice. He thinks he can make lots of money with his idea. What do you think? Write a letter to Toño and explain how he can stay out of financial trouble or how he might actually be able to make money on this, uh, idea. A minimum of 200 words please. Toño wants details. If you use AI to answer this question, you must specify which service you use.
Letter: [This is where I pasted the student’s work]
Scale: Did you follow instructions? 20%. Grammar…it´s suggested you make use of conditionals and maybe imperatives. 20. Structure. Complete sentences? 20. Punctuation. Do you use periods instead of commas? Capitalize where appropriate? 20. Content. Do you answer the question? 20.
This was Gemini’s complete assessment to one of the student’s letters. Retrieved 4/19/2024 from: https://gemini.google.com/app
Evaluation of the Letter to Toño:
This letter effectively addresses the prompt and offers practical advice, scoring well on the scale:
Did you follow instructions? (20%) – Yes (20/20). The letter focuses on financial concerns and avoiding trouble, with concrete steps for copyright clearance, budgeting, funding options, and monetization strategies.
Grammar (20%) – Excellent (20/20). The letter is well-written with no grammatical errors.
Structure (20%) – Excellent (20/20). The letter has a clear structure with a greeting, informative body with well-organized points, and a closing with encouragement.
Punctuation (20%) – Excellent (20/20). The letter uses punctuation correctly.
Content (20%) – Excellent (20/20). The letter clearly discusses copyright issues, budgeting, funding options, monetization strategies, and collaborating with Niña.
Overall Score: 100/100
This is an outstanding letter that provides Toño with a clear roadmap for the financial and legal aspects of his project.
Improvements and Suggestions:
This letter is well-written and covers all the important points. There are very few minor suggestions for improvement:
Specificity: Consider including examples of resources for finding affordable ways to obtain music rights or exploring low-budget filmmaking techniques.
Here’s an example:
Don’t worry, using popular music doesn’t have to break the bank! Look into “royalty-free” music libraries or consider contacting the original artists to discuss licensing fees. Explore online tutorials and communities dedicated to low-budget filmmaking! Platforms like YouTube offer filmmaking tips, and special effects software like Blender (https://www.blender.org/) can add impressive visuals without a Hollywood budget.
Overall, this letter effectively addresses Toño’s needs and sets him up for a financially sound and potentially successful creative endeavor.
The assessment usually includes the original text which I removed due to privacy issues. Gemini did a more thorough job of assessment than I usually do. The student’s letter was written by Chat GPT, so everything was perfect and there were few suggestions and no revisions were offtered. That would be great, though if only my students were capable of writing so well, because that’s not how most of my students write, really.
One of the students bravely wrote their own paragraph, warts and all, and turned it in. Privacy laws prevent me from sharing it with you, but I will show you how Gemini dealt with it.
Evaluation of the Letter to Toño:
This letter scores moderately on the scale:
Did you follow instructions? (10%) – Partially (10/20). The letter mentions copyright but doesn’t offer solutions for creating a financially viable project.
Grammar (10%) – Below Average (10/20). The letter has some grammatical errors and incomplete sentences.
Structure (15%) – Below Average (15/20). The letter has a basic structure but lacks clear organization and focus.
Punctuation (15%) – Below Average (15/20). The letter has some punctuation errors, mainly missing periods and misused commas.
Content (15%) – Below Average (15/20). The letter warns Toño about copyright but doesn’t offer alternative solutions or address financial aspects.
Overall Score: 65/100
This letter raises copyright concerns but lacks a positive and helpful approach.
Here’s a Revised Letter for Toño:
Hey Toño,
Great to hear you got your cars back! That must’ve been stressful. Your idea for a musical movie, “Revenge of the Rapido y Furioso,” sounds wild and creative! Combining genres like bandaranchera, reggaeton, KPop, and a space setting with a musical twist is definitely an attention-grabber.
However, it’s important to consider copyright. Using established songs or characters from things like Star Wars can lead to legal trouble. Here’s how you can still make your movie awesome:
Original Music: Niña’s involvement in writing music is fantastic! Explore composing original music inspired by the genres you love. This avoids copyright issues and lets you create your unique sound.
Fresh Story: While drawing inspiration from existing stories is okay, create your own spacefaring criminals with a mission. This originality protects you legally and opens doors for a fresh fan experience.
Now, about making money! Direct sales of a fan-made movie might not be an option, but there are ways to earn:
Free Online Platforms: Upload your musical to YouTube or Vimeo. If it’s high quality and gains a following, you could receive fan donations or attract sponsorships.
Creative Merchandise: Design t-shirts, posters, or figurines featuring your original characters and sell them online.
Build Your Audience: Engage with fans online by creating trailers, behind-the-scenes content, or even webcomics based on your characters.
Remember, focus on quality and entertainment. A fun and well-made musical, even with a niche appeal, could attract a dedicated audience.
Think of this as a way to showcase your and Niña’s talents, build a fanbase, and maybe even catch the eye of the entertainment industry! We’re here to help in any way we can. Keep us posted on your progress, Toño!
Best regards,
[Your Name]
So, Gemini kindly offered a complete revision with a very positive, jovial letter to Toño. I believe this evaluation comes pretty close to how I would have evaluated the same writing. The big difference though is in the suggestions and in the revisions that Gemini offered. Couldn’t I leverage this technology to instruct students how to write, or better yet, get the AI to teach my students how to write? I the next post, I will reveal how I did it.
