A few weeks ago (what? 2?) I shared that I’d stolen the Massachusett’s Technology Self-Assessment Tool (TSAT) and adapted it for use in a local school district. To be honest, I was more interested in exploring the use of the Questionnaire Module for Moodle, so when someone said, “Gee, wouldn’t it be great to do an online assessment of administrators so that we can know what professional development they need?” I was already 3/4 of the way done…or so I thought. Then, the email came that had me re-investigating Filemaker Pro as a way to manage data, exporting from the Questionnaire Module (thank goodness, it was easy) and then using my non-existent artistic design skills–I’m a wordsmith, not an artist!–to come up with a usable district level and individual respondent report.

I know, this is terrible. I shouldn’t be admitting this or even sharing it with you but when you spend a few hours of a day working on a project, you want other people to know, to suffer with…so, while I wish that the process above was what I followed, here’s the process I actually followed (although, come to think of it, I don’t think I was THAT far off):

1) I stole MA Technology Self-Assessment Tool. I liked it because it boiled the Technology Standards for School Administrators (TSSA) down to 3 domains, which it characterized as:

• Basic Technology Skills
• Leadership Skills
• Social, Legal and Ethical Issues

While I don’t care for the first domain–basic technology skills–they are pretty accurate…and needed by administrators. For example, how do you save your email into Personal Folders in Outlook to save space on the mail server? That’s important for administrators because they have 100s of emails coming at them, some with large attachments but they only have 50 megs of email space (why don’t they switch to GoogleApps for Education? 6 gigs of space!). Yet, a lot of them may not know how to accomplish this. Another point–I don’t see these domains as having to be mastered one at a time, one before the other. I see them all as co-existing and valuable but you don’t have to do the basics before getting the leadership or ethical issues nailed down.

Then, with all that in mind, I made some modifications to the instrument. Remember, the goal of this assessment was to help administrators understand what professional development they needed to participate in. That means, professional development that I would have to offer or contract out with someone to provide. But, I didn’t think that far ahead when I was dropping this stuff into Questionnaire Module…I was having too much fun learning a new tool…has that ever happened to you?

I was hoping to point administrators in the general direction of the new NETS-S, especially creativity and collaboration, digital citizenship, etc. It didn’t occur to me at this point 2 weeks or so ago that I would need to craft workshops aligned to the questions.

2) After swiping the MA Technology Self-Assessment Tool, I eagerly installed and dropped the content into the Questionnaire Module in Moodle. What an awesome tool. Yes, I was carried away by the excitement of the tool without thinking clearly about its uses. So what? I had fun, I just didn’t realize 44 questions later and 44 responses later, what would be involved in analyzing the data.

In the meantime, I enjoyed the output of the reports that displayed simple YES/NO responses. Please, tell me how bad it was for me to assume, “Yes, you either know something,” or “No, you don’t know it.” I like the honesty of such an admission, the ownership of reducing complex knowledge, skills and strategies to a simple “YES” or “NO”. Blank responses–such as to questions about budget–were labelled as “NO”. And, in reality, the answers were 1s and 0s. The irony of binary responses was lost on me until this very moment.

Here are a few of the questions (the first in each domain):

I did make some modifications to the questions to reflect my fascination with GoogleDocs, blogs, etc:

2) Analyzing the data collected. District administrators responsible for campus administrators set a requirement of about a week and a half for staff to complete the assessment. Then, came the question that I should have anticipated but neglected to anticipate:

How soon can you analyze the results of the technology survey? Will you have the capability to analyze results by Area, by campus level, or only by District? Pending access to survey responses and identification of patterns of needs, we would like you to present a needs-based topic to principals at the next principals’ meeting. The time would be approximately 30 minutes….

30 minutes for a variety of topics. But how to export the data? Fortunately, Questionnaire Module in Moodle makes it easy.

But the real challenge was analyzing the data, making it understandable, because what follows isn’t all that helpful:

unless you import it. So, that’s what I did…right into Filemaker Pro. And, that helped me understand the data a lot more than just a bunch of 1s and 0s. But I still wasn’t happy…I wanted to provide something more, and frankly, I just couldn’t see the patterns in the 1s and 0s (I’m not Neo from The Matrix).

So I started playing with Filemaker and how I could represent the data. It started to shape up but the fact is, I’m not sure what I shaped up. Here’s what it ended up looking like:

Page 1:

3) Creating individualized reports–a customized learning plan–based on the data. This was particularly difficult. Not the concept of doing so, but that I’d have to design courses to fill the gaps. Of course, difficult and fun! But a lot more work needs to be done. If I’m not way over the line in interpreting assessment data (I suspect I’m so far over I’ll be condemned, how’s that for transparency?), I’ll be even more way out there for what is to come.

What appears below is Page 2, which is an individual administrator’s report (in 3 sections). Each administrator gets their own custom report. You’ll note that the raw scores appear (Yes=1, No=0). Every “0” is tied to a specific workshop you can take to address your “deficit” or need as an administrator. This is where I really took a leap of something and where I’m most nervous about this assessment and its validity (no, probably none).

The recommended workshops again reflect specific needs. Most of the Web 2.0, or Read/Write Web, stuff, well, you can see that it’s what principals were shown to need. How these align to the Technology Standards for School Administrators (TSSA), that’s the next part of the job…making that connection as clear as possible, or at least, trying to make the connection. Again, I’m honestly not sure if this is possible but with only a few hours work to turn this around (that sounds like an excuse), this is what I came up with.

I am also worried about the design. Not the artistic design of the report, but rather, the design of the instrument itself. What fundamental rules have I broken here? The rest of the recommended workshops are not as easy. I noticed that as soon as I departed from the how-to in the realms of leadership and ethics, I was immediately struggling to come up with workshops that would address the needs. I definitely need to spend more time reflecting on what I know that fits, as well as what I don’t know that could fit (gee, your help would be great here!).

Going through this has taught me a few things, and made me aware of the gulf of what I don’t know. As a speaker and author, I get to pick what I’m presenting on and share it with a wide audience. I venture out, quite confident, about a particular topic or subject but the job of assessing (which is probably best left to folks like Dr. Chris Moersch with his LOTI, or enGauge, or the others out there) and adapting workshops…well, I’m not as confident. This highlights for me a whole area of professional learning that I need to engage in because what I thought I knew, I don’t. And, what I wish I knew, well, I better get started learning now. The problem is, what do I do with the principals who will get this assessment report and then dedicate time to live up to it?

That’s the part that bothers me the most. Will I have given them a goal worthy of them and what I’ll teach for both of us to live up to? I don’t know. Thoughts?

---

---

Subscribe to Around the Corner-MGuhlin.org

---

Be sure to visit the ShareMore! Wiki.