Image Source: https://openclipart.org/detail/296505/crane

Thanks so much to fellow blogger, Terry Freedman (ICT & Computing in Education, Eclecticism), for his comment on my blog entry. He quotes this selection from my blog entry, Critical Thinking About AI Results:

“One supposes that teaching critical thinking, problem solving, and creativity skills was essential WITHOUT a focus on AI or even needing to bring AI into the equation.” 

Then, Terry says:

Exactly. When I started teaching there was not the degree of tech we have now, let alone Ai, and I emphasised the need to question all info sources, including me!

I’m not sure what happened to information problem-solving. It was always a bit difficult of a sell in schools, and ended up being relegated to posters on the wall. Too many competing perspectives, one supposes.

Terry’s been experimenting with AI, as well. You can read his pieces here:

• Blogging With Artificial Intelligence Part 1

• Blogging With Artificial Intelligence Part 2

• Blogging With Artificial Intelligence Part 3
• Blogging with Artificial Intelligence Part 4
• Experiment In Style — With AI

The Real Issue: Assessment

The issue many educators seem to have with AI isn’t that critical thinking will fail. We don’t have to wait for AI adoption everywhere to see the rise of fantastical rationalizing to appreciate that fact. The real challenge for educators appears to be centered around assessment.

I’m guessing that the issue is assessing student work that has been generated by AI. Further below, I explore that. In the meantime, check out what Terry says.

AI Assessment of Student Work

Terry makes this point in one of his blog entries, Artificial Intelligence in the Classroom-Revisited:

One of the challenges teachers face when trying to work out whether a pupil knows something is balancing the requirement for lots of evidence with the need to reduce workload. Perhaps in the future similar systems will be available in education. Imagine being able to present the computer with a pupil’s e-portfolio, essays, project notes and annotated programs, and be rewarded with an instant analysis of her ‘computing capability’.

At the very least, AI could be used to suggest the student’s weak and strong points – of course, how to convey that to the student and scaffold appropriate learning activities would still be up to you, the teacher.

At the very least. Now, we know better. Terry wrote that article in 2019. Today, we see that the potential for AI to provide feedback on student work is much greater. For example, if we asked AI to analyze a grade 6 reading short constructed response (Texas Grades 6-8 Reading Language Arts), what would that look like?

I haven’t the faintest clue. So I checked.

In A Classroom

A response would have at least:

• At least one explanation about the author’s main message in an article
• One piece of supporting evidence, used accurately, from the text

How would AI do in response to this Time for Kids article (you may want to skim it first)? Here’s what ChatGPT did with it:

Life as an astronaut on the International Space Station is unpredictable. Things can break and objects can come flying at the station. Astronauts have to be ready for anything, so they go through tough training. They learn about the Soyuz capsule and how to handle problems in space.

Astronauts also have to relearn basic tasks because of the weightlessness. Safety is a big deal and they have to work well with their crew. Becoming an astronaut is hard and they have to have a degree and pass exams. Then they train for two years at NASA.

I asked ChatGPT to tell me what to say, as a sixth grade teacher. It’s response was quite long:

As a 6th grade Texas teacher, here is some feedback on the provided text:

1. The text provides a good introduction to the topic – life as an astronaut on the International Space Station.

2. It is important to clarify the intended audience of the text. Is it targeted towards students, adults, or a specific age group? This will help determine if the language and content are appropriate.

3. Consider expanding on some of the concepts mentioned. For example, when discussing “things can break and objects can come flying at the station,” it would be helpful to provide specific examples or incidents that have occurred. This would make the information more engaging and memorable for students.

4. The sentence “Astronauts have to be ready for anything, so they go through tough training” is a bit vague. Explain what type of tough training they go through to prepare for the challenges they may face in space. Are there specific simulations, physical exercises, or mental preparations that astronauts undertake?

5. When mentioning the Soyuz capsule and handling problems in space, provide more details on the training, simulations, or specific procedures astronauts are taught to manage these situations effectively. This will help students visualize the complexity of their training.

6. Elaborate on the phrase “relearn basic tasks because of the weightlessness.” Explain what basic tasks are impacted, how they are affected by weightlessness, and how astronauts prepare or adapt to perform them in space.

7. Consider mentioning some examples or specific incidents showcasing how safety is prioritized and how astronauts work well within their crew for the international cooperation aspect. This would make the topic more relatable and interesting for students.

8. Provide additional information about the degree and exams astronauts need to pass. Highlight some of the key subjects or skills that are necessary for aspiring astronauts.

9. Add more details about the training process at NASA. Are there specific modules, exercises, or challenges astronauts undergo during the two years of training? Mention some of the key concepts that are covered or skills that are developed during this period.

10. Consider concluding the text with an inspiring sentence or two that relates back to the unpredictable nature of life as an astronaut but emphasizes the importance and rewards of their work.

Remember to keep the language accessible to 6th grade students, provide examples to support the information provided, and use clear, concise and engaging language throughout the text.

I can’t really speak to whether this is quality feedback or not. The Grade 6 Informational Writing Rubric looks like this:

What do you think? Does that “original response” from a sixth grade student (a la ChatGPT) and the feedback ChatGPT match up? So I fed it the rubric and a piece of writing:

Overall, this piece of writing meets the criteria of the rubric to some extent. However, there are areas where improvements can be made.

1) Controlling Idea/Thesis is clear and fully developed: The controlling idea is vaguely stated but lacks development. It would be beneficial for the writer to provide a more specific thesis statement that clearly conveys the main point of the essay.

2) Organization is effective: The organization of the writing is fairly straightforward and follows a logical progression. However, it lacks a clear introduction and conclusion, which can help to better frame the content and tie the main points together.

3) Evidence is specific, well-chosen, and relevant: The evidence provided is limited and does not go into sufficient detail. It would be helpful to include specific examples or anecdotes that support the statements made. This would add credibility and make the writing more engaging.

4) Expression of ideas is clear and effective: The expression of ideas is clear but quite simplistic. There is room for improvement in terms of using more varied vocabulary and sentence structures to enhance the overall readability and impact of the writing.

In conclusion, while the piece of writing does address the rubric criteria to some degree, there are areas that could be improved upon. Adding a clearer thesis statement, providing more specific evidence, and utilizing more advanced language and sentence structures would help to strengthen the writing overall. Additionally, including an introduction and conclusion can help to establish context and provide a sense of closure.

What If

Then, it occurred to me, what if I gave it the same instructions but asked it to analyze a piece of writing the Texas Education Agency had already analyzed? Would the results be the same?

Here’s a student response that scored a “2:”

 I think the maim message the auther is trying to construct is “Working on the ISS is
diffucult and is a struggle for people even if they already know alot of science and
technolege. It takes alot of time and effert for the people who really believe they want to
be an astonaught.”
Supported sentences from artical.
Paraghraph 1, “ife can be very unpredictble for astronauts on the ISS”
 “part of the station might stop working at any time.”
Paraghraph 5 “ Mission prep begins but two years befr liftoff”
Paraghraph 14 “ 12 out of 18000 applicants were aprooved by NASA’s” 

 Here’s the analysis from TEA:

The writer states the author’s main message is (“Working on the ISS is diffucult and is a
struggle for people even if they already know alot of science and technolege”). This is an
acceptable answer. The writer includes relevant evidence to support the author’s main
message (“ife can be very unpredictble for astronauts on the ISS”) and (“part of the station
might stop working at any time”). Additional quotes from the article provide supplementary
evidence to the author’s main message (“ Mission prep begins but two years befr liftoff” and
“ 12 out of 18000 applicants were aprooved by NASA’s”).

 So I prompted ChatGPT in this way:

Provide feedback on this piece of writing that takes into account these rubric areas. Rubric areas include 1) Controlling Idea/Thesis is clear and fully developed; 2) Organization is effective; 3) evidence is specific, well-chosen, and relevant; 4) expression of ideas is clear and effective.

Piece of writing to be assessed by the criteria above is:

 [student response from above goes here]

Here’s ChatGPT’s analysis:

Overall, the writing meets the criteria of the rubric to some extent, but there are areas for improvement.

1) Controlling Idea/Thesis is clear and fully developed:
The controlling idea is somewhat clear – that working on the ISS is difficult. However, it could be more developed to include more information. For example, it could mention the challenges faced by astronauts and the skills required for the role.

2) Organization is effective:
The organization of the writing is not clear. It lacks proper paragraph structure and cohesiveness. It would be better to separate the supporting sentences into individual paragraphs, each focusing on a different point or piece of evidence.

3) Evidence is specific, well-chosen, and relevant:
The evidence provided is specific and relevant to the main idea. However, it would benefit from being expanded upon and explained further. For example, instead of just mentioning that part of the station may stop working, the writer could elaborate on the potential implications and challenges this presents for astronauts.

4) Expression of ideas is clear and effective:
The expression of ideas in the writing is somewhat clear, but it could be improved. Sentence structure and grammar need to be corrected. Additionally, the writing could benefit from proper punctuation and capitalization.

In summary, the piece of writing addresses some of the rubric areas, but there is room for improvement. The controlling idea could be developed more fully and the organization of the piece should be adjusted to better present the evidence. Additionally, the expression of ideas needs improvement in terms of sentence structure, grammar, punctuation, and capitalization.

Not too shabby.