The End |
We also look at at the component of test taking , and looking at referral back approach, which is a skill that some test takers use. Touched on text anxiety and ways you can help a test taker diminish anxiety, an example was: Have a quiz versus a test, it helps handle test anxiety, it helps the test taker feel more comfortable and have more confidence taking a quiz versus a test. It may also help the instructor see the areas where the tester is really struggling and help the tester and instructor revisit the problem areas and successful learn the material.
Saturday class also had a lot of revisiting involved, touching on the Levels of Evaluations again, Criterion Reference and Normal (Bell Curve) Reference. We also touched on the five levels of Evaluation, in the review, the question was What does the levels mean? The five levels were listed and discussed in class. These levels are:
1. Reaction
2.Learning
3.Transfer
4.Impact
5.ROI
Levels 1-4 recognized person of interest for these levels is Donald Kirkpatrick, for level 5 Jack Phillips
Why use these levels? Answer: it fine tunes training.
it was stated in the classroom discussion that only 5% of organizations use level 5 of the evaluation scale, reason being , it is Hard to measure, it takes a lot of thinking and a lot of people are unable to do this because you have to quantify. The difference was also noted between levels 4 and 5 which is (Money) ROI is $ Business/Impact (Numbers)
EVALUATION LEVELS
Level | Measurement Focus | Examples | |
1. | Reaction | Participant feelings about a program | "Smile sheets" Interviews, Discussions |
2. | Learning | Changes in knowledge, skills, and attitudes | Written pre/post tests Skill laboratories Simulations Projects; Oral exams |
3. | Transfer | Changes in on-the-job behavior (application) | Performance checklists Performance appraisals Critical incident analysis Self-appraisal |
4. | Business Impact | Changes in business of an organization | Production indices-- (Cost, scrap, accuracy; rework, schedule compliance, output, referrals, quality) Grievances, Law suits Absent/Tardy rates Medical reports/leaves Turnover Accident rates Employee suggestions Customer complaints |
5. | Return on Investment (ROI) | Compares program benefits to the costs. | Comparative Monetary Values (Productivity converted to $&c; cost avoidance (e.g. law suits); cost per time or error reduction; retaining vs losing customers (both internal and external) cost per complaint reduced or prevented; cost of absenteeism, tardiness, accident; safety violation |
Another exercise was done using the table above, the exercise was located in Wiki on the right hand side under Evaluation levels, the exercise completed and discussed in class was the Hairstylist scenario evaluation level practice. In this exercise the 5 levels of evaluation were examine to see which levels were present and which were not, at the end of the exercise, levels 1-4 was present, you needed more info ans the present of $ amt in order to have reach level 5 the ROI and the data, and information of the money made was not presented in the scenario. During the discussion of the scenario, classmates including myself, gave a gender to the salon manager or the stylist, it was noted by the professor to keep material that is written "gender neutral."
We discussed the difference between Education, Presentation and Training
Education: General Background (Nice to know)
Presentation: Give out the information (passive approach)
Training: must include practice and feedback
We touched on REWORK (James Harrington) Harrington we spend 40% of time on reworking ,Total quality management (TQM.) Professor Ackerman, has a design called the kiss hug design, which due to copyright reasons i will not post it into this blogger it is her own design, and i wouldn't want someone else taking credit for her work, but in simple terms it shows the difference of treatment and observation, using post test, pre or post testing, post testing only, having a control group and pre-testing only.
We also talked about a website called merlot which is an open source site where you can find online learning material to include in training sessions. MIT developed the open source idea, open source means open to the general public for viewing
The class did an AHA moment before breaking for lunch on what was learned during the morning session of class. During the evening portion of the class the Final exam was to create a quiz item and critique, what was different about creating a quiz item this time around was the fact that the class had experience with this assignment, the quiz item created we better developed, and not simple low standard question, but very well thought of and developed questions, what also improved was the critiquing, the first time this assignment was accomplished there was a great deal of time spend on critiquing the quiz items, this time around, hardly any issues with the item and less critiquing took place, due to better understanding of using guidelines, developing higher level questions,being precise and accurate.
Saturday class ended with 90 second reflection of WED463, there were a lot of great insight from each student and the reflection portion of class showed how much everyone learned and gave each student an idea of what the other took away from the course.I learned a lot more about formative evaluation, and being more discipline using webtools such as Skype, Zunal and Wiki.
As for Sunday class we have to finish up all assignments, wiki post, blogs, catch up work because it is all due on Monday 6th at 8am.I enjoyed this course and took some great learning tools away from it. IT WAS FUN!!!!!!!!