Section 3: Evaluating, Implementing and Managing Instructional Programs and Projects

                In order to determine if an instructional design is effective, there must be some type of evaluation of the instruction. Two popular evaluation models are the CIPP and Kirkpatrick models. Each model has steps to flow through while evaluating the success of an instructional design. There are different types of evaluation models: summative and formative (Reiser and Dempsey, 2007).[1] Barbara Flagg (1990) developed a formative evaluation system for instructional design.

Flagg’s evaluation design includes:

Evaluation Design	Example
Needs Assessment	Review Experts
Pre-Production	Design
Production Formative Evaluation	Feedback
Implementation Formative Evaluation	Results from field testing [2]

This evaluation design focuses on the delivery of the information and not the summative. Its focus is the design process before the delivery of the instruction to a large audience. An example of a summative assessment evaluation model is one developed by Smith and Ragen (1999).

Smith and Ragen’s evaluation design includes:

Evaluation Design	Example
Determine Goals	What questions need to be answered?
Select Indicators	Learning evidence
Select Orientation of Evaluation	Qualitative or Quantitative data
Select Design Evaluation	Data Validity
Design or Select Evaluation	Implantation
Collect Data	Schedule data collection
Analyze Data	Statistics
Report Results	Include information from all of the above[3]

                The evaluation design models of Flagg and Smith and Ragen, can be transferred into a K-12 educational setting. However, Flagg’s evaluation model is not as viable as an option simply for the fact that there is really no test group in a classroom setting, unless you consider each year a test group to improve instruction upon for the next year. Gathering feedback from students could be an effective evaluation tactic in instructional design. The Smith and Ragen model is summative and could enable educators to evaluate a program. Starting off by selecting the goals is one of the most important tasks an instructional designer can do. Without the goals, how does one know what to work towards? Asking what do you want the students to be able to answer at the end of the instruction is crucial. In education, many say that an educator should start with the end of the lesson (outcome) when designing a lesson and then work backwards. Collecting data is part of the evaluation design. Data is important in determining if the instructional design is effective. This author will often start with a higher order thinking question that students should be able to discuss/debate at the end of the instructional lesson. While keeping that question in mind, the lesson is more focused, data is collected during the lesson, and adjustments are made to ensure students have a deep understanding of the instructional goal. There are other evaluation models available that are more suitable for a corporate setting. For instance, the PERT (Program Evaluation and Review Teaching) is an evaluation model focusing on time management.[4] This model is about sequence and creating a path using estimating time of the instructional design. Also, there is the CPM model (Critical Path Method) that focuses on sequencing, time, and following a critical path. [5] An instructional designer must not only select the proper instructional design to use, but must also determine how to evaluate the instructional design. This requires planning, sequencing, and time management. As an educator, this author feels that the use of formative and summative evaluations is imperative to determine if students are understanding objectives. In order to make that determination, data must be collected and evaluated.

        With the new trends in education, integrating technology into the classroom is becoming increasingly important. The state of Texas has implemented technology standards and there is also the movement of 21^st century learning. The use of promethean boards is one technological innovation that has been introduced into the classroom in this author’s school recently. The school purchased a promethean board for each classroom in hope that the teachers will use them on a daily basis. Promethean boards are a way to get students interacting with the technology while learning required information. The use of promethean boards has a high relative advantage in the classroom. This movement of using promethean boards has been progressive. It began with the simple overhead projector and pull down white screen that teachers used to sit in front of with an erasable marker. Then, as computers and projectors became more common, that progressed to PowerPoint presentations. Now, with the introduction of smart boards, student interaction and flipcharts are becoming more popular. The student engagement and interaction is valuable in the instructional design. Students are used to interacting with information outside of school (video and computer games) in today’s age; so why not use what they like to do in the classroom. Compared with older versions of the overhead and pull down screen, promethean boards have a high relative advantage.

                The promethean board is very compatible with what educators are doing in the classroom. It does not go against the values of education to interact with smart board. Promethean boards create active learners out of students rather than passive. Flipcharts can be easily created from lessons in the curriculum to ensure that the promethean board remains compatible with the current educational standards. Even though there was not a trial ability of the use of promethean boards before the adaption of them to the school, the school did provide training on its usage and instructional ideas for integration.  The trial is going on in the classroom. To be honest, there are many teachers who have been teaching for a while who have been slower at adapting to the technology. It could be that it is not compatibile with what they are used to doing and are still trying to understand what possibilities it could have in the classroom. In other words, the trial is happening while educators are becoming comfortable with using the technology. This author has been teaching for seven years and received a promethean board two years ago. Trying to integrate the technology into the curriculum has not always been easy, however, the more lessons that are created that use the promethean board, the easier it gets. Observablility is always occurring (Reiser and Dempsey, 2007). To measure if student learning is really increasing as a result of the promethean board is an ongoing process. For the most part, students are engaged, participating, and enjoying the learning experience while interacting with the promethean board. Therefore, an educator using the promethean board can see the benefits in the classroom. The perceived attributes of using the technology will lead to a positive outcome. Using the promethean boards is not complex and with a little bit of training can become a highly effective tool.

                Part of designing an instructional design project is being an effective manager. For instance, if this author was assigned to develop a series of professional development sessions on using a new technology in the classroom, like promethean boards, the situational leadership models from Hersey and Blanchard would be good to use (Reiser and Dempsey, 2007). This model contains three phases of management: guidance, amount of socio-emotional support, and readiness level of followers. As the manager, the overall authority of the project must rest in that position. The manager must be able to communicate clearly and precisely. They must motivate the team in coming together and completing the project. Also, resolving conflicts and keeping the team on schedule is a vital part of being an effective manager.

As noted by Reiser and Dempsey (2007), the manager will have to provide more guidance at the start of the project (phase 1) where one must be very direct and demanding. Next, under phase 2, as the team gains more confidence in their work and they know their role in the project, the manager can take on a less demanding role. Finally, under phase 3, the team is able to function under a less direct role and the manager provides guidance while monitoring the project. Basically, after the initial stage, the manager gradually loosens up control and offers support rather than direct commands on details of the project. The situational leadership model would be very effective if designing a professional development series. From this model, a manager reminds this author of being an effective teacher.  For instance, an effective math teacher teaches a new concept directly and makes sure students understand how to do the math, the proper way to set it up, and if doing group work, their roles in the group. As students become more confident in their abilities in doing the math and working as a group, the teacher can take on a role monitoring the groups rather than direct teaching. The model below represents the Situational Leadership Model.

[1] Reiser, R. A., & Dempsey, J. V. (2007). Trends and issues in instructional design and technology. (2nd ed.). Upper Saddle River: Prentice Hall.

[2] Dabbagh, N. (date retrieved). The Instructional Design Knowledge Base. Retrieved February 2, 2012 from Nada Dabbagh's Homepage, George Mason University, Instructional Technology Program. Website: http://classweb.gmu.edu/ndabbagh/Resources/IDKB/index.htm

[3] Dabbagh, N.

[4] Pocock, J. W. (1962). PERT AS AN ANALYTICAL AID FOR PROGRAM PLANNING--ITS PAYOFF AND PROBLEMS. Operations Research, 10(6), 893-903.

[5] Net MBA Business Knowledge. (2010). Retrieved from http://www.netmba.com/operations/project/pert/.

---