Evaluating, Implemeting, and Managing Instructional Programs and Projects

1. An evaluation model that I found when searching for evaluation models is the Flashlight Triad Model which helps to give focus and to carry out technology-oriented evaluations.

This model has 5 distinct steps:

1) Overview & Confronting the Blob--In this step, brainstorming takes place to help identify all of the various elements of a lesson, course, or program.

2)From Blob to Issue--determining the purpose for the evaluation (i.e. summative or formative). In this step it is also decided who the intended audience for the evaluation is. The following questions should be answered--a) What are the three most crucial things you fear might happen as a result of using technology? b)What are the three most crucial things you hope will occur?--then one or two of the top issues from these responses should be chosen for further study.

3) From Issue to Triad--For each issue selected, create several "triads". A triad includes a possible technology to use (T), a specific activity that the technology enables (A), and expected outcomes from the activity (O). Select the triads on which you wish to focus.

4) From Triad to Data--Generate questions to gather data about the triads (these can be administered as surveys, interviews, focus groups, etc.). At least five types of questions can be created: a) technology, b)interaction of technology and activity, c) activity, d) interaction of activity and outcomes, and e) outcomes.

5) From Data to Next Steps--Based on evaluation data, decisions can be made about a lesson/program, etc. (such as necessary modifications).


Another evaluation model I found is the Discrepancy Evaluation Model. This model serves as an ongoing narrative of the project's activities related to it original goals and objectives, as well as a roadmap that evaluators can use to analyze the status of the process and summative evaluators. In discrepancy evaluation, performance is compared to the standards (i.e. what is is compared to what should be). If a difference is found between the performance and the standard, this is considered a discrepancy, and a discrepancy can be either positive (performance is better that the standard) or negative (performance is lower than the standard).

The Discrepancy Evaluation Model has four stages:

1) Pre-Design Evaluation--determines the appropriateness of the simulation to be chosen as a method of instruction for a particular lesson.

2) Design Evaluation--seeks to determine that the chosen simulation is appropriate and that the simulation has validity.

3) Input/Process Evaluation--examines the situation in which the simulation is being tried.

4) Output Evaluation--the simulation is assessed to determine which, if any objectives were not met.

It sounds to me like it is possible to use both of these evaluation models with the same lesson. The Flashlight Triad Model seems to lend itself to evaluating the planning aspect of a lesson, whereas the Discrepancy Model might work well during the instruction process. With the Triad Model, I would brainstorm an idea for a lesson (such as using a wiki for group work and peer revision). My fears for this type of activity (step 2) might be having the students use the editing features of a wiki to alter classmates' assignments in non-productive ways, the wiki idea might be too far fetched for my students to truly appreciate (beyond their skill level), or technical difficulties might impede the lesson and activity. The crucial elements I hope would take place might be the students' increased feelings of ownership towards a lesson, increased success in peer-tutoring opportunities, enough success to warrant another activity completed in a similar manner. An example of a triad for this lesson (step 3) might look like this: Technology (T)= electronic communication tools, email, and discussion boards; Activity (A)= student critique of each other's course assignments; Outcome (O)= increased course satisfaction and student collaboration. Some questions I might ask regarding the triad I create for this activity could be (step 4): How will the students use the wiki to aid in the completion of this assignment? What is the overall goal of using a wiki for this type of assignment? How will this technology better benefit my students in successfully completing this assignment? Finally, I will be able to make any necessary modifications and/or revisions (step 5) to this lesson or another like it before using it with students again.

Using the Discrepancy Model, I would decide that using a wiki for students to complete group work and peer revisions is an appropriate way to complete this task (step 1) and has validity(step 2). Then I would observe and assist my students as they proceeded to complete the assignment requirements through use of the wiki (step 3). Finally, I would assess the entire experience in order to determine if all the objectives of the task were met (step 4).



2. The most recent innovation that has been introduced to our home is Direct TV with DVR capabilities. Prior to Direct TV we were using Dish Network with little complaint (in 2.5 years time we have had 3 different boxes for the television because the boxes kept corrupting themselves and erasing the shows we had set to record, but other than that we were content), but when the World Series came around we decided to make the change to Direct TV because of the problems that were going on between Dish Network and FOX (wouldn't you know that an agreement was reached between the two companies on THE DAY we had Direct TV installed...figures). So...here we are with the new cable provider and we're locked in for the next two years, but I digress...
The relative advantage of Direct TV over Dish Network is unobservable. Both allow the user to record live television for later viewing; however, with Direct TV we can record up to 4 shows simultaneously (this is possible with Dish Network also, and we only have this feature with Direct TV because of their failure to better explain the various packages they had to offer). Direct TV is compatible to our needs because it allows us to record multiple shows. The complexity of this new cable server is greater than that of Dish Network, but this could be largely because it is new to us. Maybe with time we will find our way around the new system and our opinions of its complexity will change. Unfortunately we were not given a trial period for Direct TV or we might have decided to stay with Dish Network (not only because an agreement had been reached but also because we like the ease of use with Dish Network). Observable benefits of Direct TV...I like that it provides a folder where numerous episodes of the same show can be found (Dish Network used to do this, too...I don't know why they stopped). Overall, I am not satisfied with Direct TV. It's functions and capabilities seem much more limited than that of Dish Network. It's possible these features that I am missing in our new cable provider are actually available and I just haven't found them yet, but if that's the case then I'm not pleased. It shouldn't be this hard. I can't wait until our 2 year contract expires so that we can go running back to Dish Network.



3. In Phase 1 of my professional development technology sessions, I would be large and in charge. I could use this opportunity to give a detailed description of what the learners should expect during the course of the training--objectives and goals, steps to be taken to reach completion of the training and an understanding of the technology being taught, etc. The learners would get the opportunity to begin working with the new technology to gain a better understanding of its capabilities while I would provide direction, guidance, and technical support.
In Phase 2 the learners will be gaining ground and feeling more confident in their new found skills (I am one happy teacher!). At this point I can provide encouragement and begin helping others to focus on aspects of their own classroom where the technology being learned might be helpful in teaching their own students.
In Phase 3 the learners are moving right along in their learning. As the instructor, I will be able to focus on what the learners are creating and get excited with them. Their projects are looking great and they are excited about taking their new skills back to the classroom.
In Phase 4 I am observing more than leading. The learners are humming along in their work, quite pleased with their accomplishments, as am I. Mission accomplished on all fronts.