October 10th 2014
Greetings,
Due to the unexpected change in our class schedule last week I was forced to cut class short however I did include a lot of videos to complete our schedule lesson. Please be sure to review them as many times as you may require.
Please note that these videos will not compensate for you actually reading the chapters on your own.
For this week.
1. Be sure that you have read up to chapter 5.
2. Class discussion on chapter 4-5.
3. Watch the following videos to improve understanding and be prepared to contribute in our class discussion.
4. Blog post- respond to the discussion and comment on your classmates response.
Discuss: Which, if either, of the two approaches to bias detection discussed in the chapter
(judgemental or empirical) do you think yields the most compelling evidence of assessment bias? Why?
Hint! you need to read the chapter, then respond!
Quiz 3 - Chapter 3 - online October 9th 2014 (Read the chapter and you will be fine.)
Test 1 - Chapter 1-5 October 17th 2014 - Format 50 MC/TF (TBA if online or hardcopy)
Enjoy your week and don't forget to read.
Judgmental vs. Empirical
ReplyDeleteEffective teachers should provide assessments that will measure their students learning based on their weekly or daily objectives. When we assess it can become bias to some extent because we can incorporate what we think they should know compare to what they actually know. You have to meet the students at the level where they are. If you are being bias in your assessment then you are not an effective teacher. Judgmental and empirical are the two approaches of bias detection that I will discuss. In the empirical method we separate the constituent parts. We tend to use different items and ways to test. For example; multiple choice, match, true and false, fill in the blanks and answer in complete sentences. We will then eliminate the act of bias assessment in this case. I must say that I don’t believe that the empirical approach actually yield towards assessment bias. When we search other references an alternative ways to retrieve information we are using the empirical method. The bias and ways of expressing very personal feelings or opinion can be the judgmental approach. The tests are being looked at closely. Everyone is different; we grade and make tests differently. As an example if I am teaching a “B” class and the teacher from the “C” class made a test; maybe the test would be too easy for my students to do. I will get a lot of hundreds; even though I personally don’t agree with streaming kids. So I may use some of the problems or questions but not all. I may think that the test will not be a challenge for my students. Then again it would not be fully reliable.
The Judgmental approach can lead to debates. These groups can encounter serious conventional issues. I am geared towards the judgmental approach because it enhances evidence that validates assessment bias. The judgmental approach is not based on analysis but on how the individual thinks about the assessment or whatever topic is being discussed. In this case they are liable to be bias.
The degree to which students’ performance on a test accurately reflect the quality of instruction specifically provided to promote students’ mastery of what is being
ReplyDeleteassessed (Popham, 2006) Hence assessments can measure the effects of previous teaching, and they can also be used as outcome measures to evaluate instruction, as well as to identify students who need additional instruction. Thus Judgmental and Emipircal are two approaches that teachers commonly commit whether knowingly or unknowlingly. When we refer about being judgmental we are talking about the procedures that are provided by yourself, your colleagues and your students. Each of the three judges will follow different procedures. The first judge would be you, judging your own assessments. The five criteria procedures for judging your own assessments are; first, adhere to item-specific guidelines and general item-writing commandments, second, contributions to score-based inference, third, accuracy of content, fourth, absence of content lacunae or gaps, and fifth, fairness. The second judge would be a collegial judge. The collegial judge would be provided the same five criteria used by yourself as well as a brief description of each of the five criteria. The third judge would be the students. The students are often overlooked as a source of test analysis, but whom better and most experienced than the test takers. There are five improvement questions students can be asked after an assessment is administered. The first is, if any of the items seemed confusing, and which ones were they? Second, did any items have more than one correct answer? If so which ones? Third, did any item have no correct answers? If so which ones? Fourth, were there words in any items that confused you? If so, which ones? Fifth, were the directions for the test or for particular subsections unclear? If so which ones? Note that these questions can be altered slightly for constructed response assessments, performance assessments, or portfolio assessments. Ultimately the teacher needs to be the final decision maker for any needed changes. By doing this, it allows for personal feelings to com into play when considering allocating the grades to another section without any rubric or prior criteria. On the other hand, when using the empirical methods several different types of item analysis and discrimination practices can be used by applying various formulas. Ultimately the assessment creator needs to apply good techniques and have a willingness to spend some time to improve their assessments. For example Negative discriminating items are a sign that something about the test is not good. This is so because the item is being missed more often by students who are performing well on the test in general and not being missed by students who are not doing well on the test. Ebel and Frisbie (1991) apply the following table.
.40 and above = Very good items
.30 - .39 = Reasonably good possible improvement needed
.20 - .29 = Marginal items should improve these items
.19 and below = Poor items – remove or revise items
Another method for the empirical strategy is the distracter analysis. A distracter analysis is when we see how high and low groups are responding to the items distracters. This analysis is used to dig deeper on the basis of p-value or discrimination index’s needing revision.
To conclude both strategies could be considered as bias and as such our wording of our questions should not reflect gender bias, race, religion etc but should be geared at having the student understand the instruction for execution and problem solving.
Reliability and Validity
ReplyDeleteReliability as defined is the consistency or precision in measurement, while Validity encompasses the entire experimental concept and establishes whether the results obtained meet all of the requirements of the scientific research method. As a teacher I am faced with implementing and practicing reliability and validity on a daily basis. The practice of reliability and validity is present in every aspect of life especially in the life of a teacher. Students and teachers have a strong bond, because as teachers we have to guide and direct, children in the path they want to go. Reliability is important to implement in lessons, as it provides consistency and transparency in measurements of what ever aspect of teaching. This may be done using quizzes, questionnaires, test, etc. Math being a compulsory subject adapts and uses the practices of reliability and validity, when it comes to a precise length or distance. It is not only used in relation to Math, but also in surveys, reports, grading, assessing, etc. Factors to be considered to affect one's reliability and validity are systematic and unsystematic errors.Its very important to make sure there are no systematic or unsystematic errors because they cause inconsistency in the measurement to be done.
e.g. Throughout the entire school year, James passed all his assessment, quizzes and test very high with all A's. On his end of term exam he did not do as good. He failed the end of term exam badly. You might say that James is not as smart as he seems or he cheated to do so good on his test, quizzes, assessment, etc. But what your must put into consideration is that assumption is not reliable. May be the things on the exam where different from what was thought in class or James had not have much experience in the exam sector and gets real nervous and agitated or made errors due to unprepared.
Controlling errors is basically controlling inconsistencies. Methods of multiple judging is another form of reliability that basically uses multiple people judging something, whether it be athletic performances, CXC's, P.S.E, Olympics, etc. this practice is very essential and good for teachers to assess students as it decreases bias or favoritism.
to conclude both reliability and validity are very influential strategies that help promote consistency in and less errors. Also it gives students a better understanding on how the follow instructions for execution problems, etc.
Judgmental & Empirical
ReplyDeleteBeing at a Special Education School, forms and practices of being bias happens at some point or the other, especially if we do not cater for each child specific needs. Having a class with diverse ability students suffering from deafness, visual impairment, down syndrome, slow learning and cerebral palsy. You as the teacher need to teach by grouping the students to the different level they are on and also do assessment for each child learning needs. An effective teacher would provide assessments that will measure their students learning that links with their weekly objectives. Assessment can be bias to some extent. This statement is true because at the infant division at the school i teacher, there is a nine year old child, who does not know how the alphabets nor the sound the letter makes. We as teachers want to move on to the scheduled curriculum we have for the school year, so either we move, ignoring the child or make time for him during our breaks or free time. We need to be careful and meet students at the level where they are. The two approaches of bias detection are judgmental and empirical. Judgmental approach can be bias with the way of expressing personal feelings and opinion. i do not believe that the empirical approach yield towards assessment bias because when we use match, true and false, fill in the blank, multiple choice and answer in complete sentences we use the empirical method. Using alternative ways and methods to retrieve information is an empirical approach. Debates may be considered as judgmental. In conclusion both approaches whether it be empirical or judgmental can be considered bias, but both approach can be used in a positive way to promote problem solving, observation, etc.