In social science research, the terms reliability and validity refer to the accuracy of measurement. Inaccurate measurements may lead to erroneous or artificial conclusions or inferences.
Reliability refers to consistency and uniformity of measurements across multiple administrations of the same instrument. More simply, there should exist some measure of equivalence and consistency in repeated observations of the same phenomenon. Thus, the more consistent the results across repeated measures, the higher the level of reliability. Conversely, the less consistent the results across repeated measures, the lower the level of reliability. Reliability cannot be computed precisely because of the impossibility of the calculation of variances and needs to be estimated.
In political science, there are four primary methods for assessing the reliability of empirical measurements: the test-retest, alternative-form, split-halves, and internal consistency methods.
The test-retest method, or retest method, involves administering the same test of the same phenomenon to the same sample at two different times. If the measurements of both tests are consistent, the measure has high reliability. This method is not without its limitations. Some phenomena, for instance, are measurable only at one particular time, rendering the test-retest method impossible to conduct. Additionally, the test-retest method may suggest unreliable measurements not because the measure itself is unreliable, but because the phenomenon being studied may have changed in the interval between the two tests, often times as a result of reactivity, or the actual act of testing, which may itself inspire a change in the phenomenon.
The alternative-form method is similar in principle to the test retest method; however, a different mode of test is administered to measure the phenomenon at the second instance. Thus, two parallel but different measures are used instead of measuring the phenomenon using the same measure at each instance. Using two different measures reduces the risk of reactivity. As with the test-retest method, there is risk that the phenomenon itself may change between tests, artificially deflating the perceived reliability of measurements. Furthermore, the alternative-form method poses a unique challenge in that the tests need to be different yet parallel to yield comparable results.
The split-halves method affords the researcher the ability to conduct two measures of the same phenomenon at the same time, eliminating the risk of reactivity or the phenomenon itself changing over time. The researcher groups the items being measured into equivalent halves and administers the same test to each and then compares the results. If the results are consistent, the measurements demonstrate a high level of reliability. The split-halves method has its limitations, particularly as there are multiple possibilities as to how the items in each test can be grouped into halves. If the two halves are not identical, there is a high likelihood that the two tests may result in a different measurement on the basis of the halves being nonequivalent.
Finally, reliability can be assessed using the internal consistency method. This method, like the split-halves method, allows the researcher to measure a phenomenon at a single point in time. Moreover, like the alternative-form method, internal consistency is based on different measures of the same phenomenon; however, this method measures consistency and correlations among multiple similar items on the same test, thereby eliminating the need to administer two different tests or to split the items being measured into two groups. Therefore, it is implicit that similar items measuring the same phenomenon be included in a single test. If similar measures produce similar scores or illustrate a strong correlation, the measures demonstrate high internal consistency and are highly reliable.
Validity refers to how well an instrument actually measures what it is designed to measure, or the degree to which a measurement procedure captures a theoretical concept being measured. A measur ing instrument that measures what it purports to measure is said to be valid. Conversely, an instrument that does not measure what it is designed to measure is said to be invalid. However, it is important to note that an instrument may be valid for measuring one particular type of phenomenon but be completely invalid for measuring another. Therefore, when assessing the validity of a given measure, a researcher does not evaluate the measuring instrument itself, but the measuring instrument in relation to its ultimate purpose
Types Of Validity And Validity Assessment
Disciplines define validity in different terms. In political science, there are four fundamental types of validity. The first is known as face validity, which refers to how well an instrument appears on its face to measure the phenomenon it purports to measure. When there is reason to question the correlation between a measuring instrument and the phenomenon it is designed to measure, an instrument lacks face validity. As such, when an instrument seems not to be relevant to the phenomenon it is measuring, or if there is a weak link between the measure and the concept that is being researched, the measure lacks face validity. Therefore, the assessment of face validity is a product of a researcher’s deductive reasoning and judgment based on the appearance that a given measure is valid.
Content validity is similar to face validity in that it predicates individual judgment, but it is often more difficult to assess, as it evaluates the extent to which an instrument adequately measures all aspects of a given concept or domain. As a result, the assessment of content validity necessitates the identification of all facets of a phenomenon and then determines the extent to which a measurement reflects and measures each facet. This is particularly difficult when the phenomenon in question is complex or multidimensional, as an instrument is only content valid when it measures all aspects of a given domain. However, if all aspects of a phenomenon are not sufficiently or adequately defined and enumerated, a measure cannot be content valid.
The third type of validity is construct validity, the degree to which a measure of a variable corresponds to the theoretical framework of a concept. An instrument is considered to be construct valid when it corresponds to the measure of a related concept. To determine construct validity, a researcher must first identify a potential relationship between two concepts, establish a measure for each, then examine the strength of the relationship between the pair. If the relationship is significant, a measure is construct valid. If no relationship is determined, the measure lacks construct validity, as at least one or the measures must not accurately reflect or measure the phenomenon.
The fourth way to assess validity is known as interitem association. It relies on using multiple measures of the same concept to determine validity. Similar to the internal consistency method, it evaluates the reliability of a measure by using more than one measure of a concept within a single measurement instrument. A consistency within outcomes demonstrates the validity of the overall measurement instrument.
- Alwin, Duane, F., and Jon A. Krosnick. “The Reliability of Survey Attitude Measurement: The Influence of Question and Respondent Attributes.” Sociological Methods and Research 20, no. 1 (1991): 139–181
- Bohrnstedt, George W. “Reliability and Validity Assessment in Attitude Research.” In Attitude Measurement, edited by Gene F. Summers, 80–99. Chicago: Rand McNally, 1970.
- Carmines, Edward G., and Richard A. Zeller. Reliability and Validity Assessment. Beverly Hills, Calif.: Sage, 1979.
- Johnson, Janet Buttolph, and H.T. Reynolds. Political Science Research Methods. With Jason D. Mycoff. 6th ed.Washington, D.C.: CQ Press, 2008.
- King, Gary, Robert O. Keohane, and Sidney Verba. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, N.J.: Princeton University Press, 1994.
- Zeller, Richard A., and Edward G. Carmines. Measurement in the Social Sciences: The Link between Theory and Data. New York: Cambridge University Press, 1980.
This example Reliability And Validity Assessment Essay is published for educational and informational purposes only. If you need a custom essay or research paper on this topic please use our writing services. EssayEmpire.com offers reliable custom essay writing services that can help you to receive high grades and impress your professors with the quality of each essay or research paper you hand in.
In tackling essay-writing, especially in the essay question section of exams, students often face three problems:
- First, some students may feel that they just dont know where to begin. How can I answer a question thats so broad? I just dont have enough information.
- Second, even if they feel they know something about the subject, they may wonder how to organize the information in order to present a coherent and convincing argument. How do I begin to put together all the various pieces to the puzzle so that what I say makes sense?
- Finally, students may be unsure about the relationship between the presentation of factual information and the expression of their own views on the issue at hand. The professor never told me whether he wanted me to repeat what he had said in class, or if he was just looking for my opinion.
Start at the BeginningWhen you first read an essay question on an exam (or begin to think about an assigned topic for a term paper or take-home final), you should ask yourself two sets of questions:
1. What does the essay question really say? What kinds of issues is it asking me to address? What assumptions underlie the question itself?
Professors ask essay questions for a reason. They use essays as a way of getting you to go beyond the material presented in class and in the required readings for the course. They intend for you to reflect critically on the information you have read, assess its validity, think about its implications, and use it creatively in order to answer the question that has been posed. So, when you encounter an essay question, spend a few minutes thinking about what the question really asks, and make sure that you have a clear idea of the kinds of issues and concepts that the question is trying to get you to address.
2. What are the most useful sources of information on which I can draw in order to answer the question? What kinds of data will best support my argument?
During any semester-long course, you will encounter a huge amount of information, both factual and conceptual. Many students treat essay questions as dumping grounds for the information that they acquired in the days and weeks preceding the exam. They pile on fact after fact, concept after concept, date after date, name after name, with little thought about whether all this information helps them answer the question. If I throw in enough stuff, a student may say, at least the professor will know that Ive been paying attention.
Wrong. The professor will know that you have managed to cram a great deal of irrelevant information into your short-term memory. But whether you have really thought about the issues at hand and used the knowledge you have gained in order to reflect critically on an important question will remain a mystery. So, after you feel that you understand the kind of response that the essay question is trying to elicit, ask yourself about which bits of information will be the most relevant to your response. Dont try to throw everything into the pot. Be selective. Use those facts and ideas that are most helpful in supporting your overall argument. After doing the reading and attending the lectures, you do have enough information to answer the question effectively. What is crucial, though, is to organize the information and to present it in a way that buttresses the main theme of your essay.
Organization Is EverythingBecause they have not stopped to ask themselves the questions above, many students plunge right into an essay without thinking about how to organize their thoughts. If I just get enough stuff down on paper, a student might argue, then the professor will at least know that Ive tried to answer the question. Wrong again. The professor will know that you are a wind-bag not that you have thought seriously about the question.
Once you are sure that you know what the question is asking and have spent a few minutes reflecting on the kinds of information that you want to use in attempting to answer it, spend a further few minutes sketching out the form that your answer will take. Here are a few ideas on how to begin:
Make an OutlineSketch out how you plan to structure the essay. You can even use the exam booklet or the back of the exam in order to write a brief outline, flow chart, diagram, or whatever form you find the most helpful in organizing your thoughts. The important thing is to have a clear idea of what you want to say and how you are going to say it before you begin writing the essay itself.
There is an additional advantage to writing an outline or essay plan: It may turn out that you simply budgeted your time poorly and did not have time to complete the entire essay as you had planned. But if the professor sees that you had a clear idea of what you wanted to argue, you are likely to receive at least some credit for what you have written. On the other hand, if you have managed to fill up a dozen pages without making a coherent argument, chances are that the professor will remain relatively unimpressed.
Keep It SimpleThink back to eighth grade composition class. Remember the three-point enumeration essays you probably had to write? They consisted of an opening paragraph, three further substantive paragraphs and a conclusion. The opening paragraph set out the general ideas you were going to explore, the three following paragraphs expanded on each of those ideas, and the final paragraph wrapped up what you had said.
The same format with perhaps some modifications can be used to write responses to essay questions.
- Opening sentence and first paragraph: State clearly the main point that you wish to make in the essay. In other words, someone should be able to read the first sentence and know exactly how you plan to answer the question. Dont try to be too cute, but a catchy opening sentence which states simply and clearly the line of argument you intend to take is always desirable. Other sentences in the first paragraph should then support the first sentence and sketch out the ways in which subsequent paragraphs will expand on the theme of the essay itself.
- Body of the Essay: For normal essay questions on exams (say, those in which you have an hour to complete two essays), you should have no more than three or four paragraphs in the body of the essay. Each paragraph should make a clear and discrete point, and that point should support your overall argument. If it doesnt, dont write it. Your thoughts in the body of the essay should follow on logically from the points you set out in the opening paragraph. And each paragraph should begin just like the opening paragraph, with a clear statement of the topic that the paragraph will address.
- Concluding Paragraph: Sum up what you have said in the essay in a final paragraph. Remind the reader of your main point, but avoid repeating it in exactly the same words. End the essay with a sentence that wraps up your thoughts and leaves the reader with a sense of closure.
Your Opinion Is More Than Just Your OpinionEssay questions are not extended short-answer questions, and they are not exercises in penmanship. A professor puts essay questions on exams not in order to see if you can repeat verbatim what he/she said in class, but in order to solicit your informed views on a particular subject that you should have mastered in the course. In this sense, essay questions do ask for your opinion, but it is an opinion that should be intelligent, informed and well-structured. No conceptual questions in political science have once-and-for-all answers. Essay questions ask you to address important issues by using your brain constructing a coherent, logical and informed view on a given topic. After sitting in a course of lectures and doing the required reading, you are more than capable of completing such a task. Your opinions should have evolved and become more sophisticated, and you should have developed a reasonable level of expertise in the main issues addressed during the course itself. Your opinions matter, for they were what your professor was trying to get you to develop all along.
Again, essays are not simply receptacles for regurgitated factual information. Your knowledge of facts can be assessed using multiple-choice questions, true/false, identify, define, short-answer and a range of other examination formats, most of which you probably experienced in grade school. At the college level, however, you are expected to think. And thinking requires creatively using the knowledge you have acquired to take a clear position on a contentious issue.
How do you do all that? Here a few guidelines:
- Make An Argument. Take a stance. Stake out a position. Argue for a particular point of view. Simply reeling off dates and names or even using political science jargon will not do the trick.
- Support Your Argument. Use relevant facts, concepts and other information to buttress the points you wish to make. Throwing in irrelevant information will impress no one. It will simply cloud your argument and convince the professor that you really dont know what youre talking about.
- Be Creative. How creatively you make your argument is always important. Style matters. Some professors may even prefer essays that are well-structured and well-written but not particularly brilliant, to those that contain a truly original insight cloaked in language that would make Webster and Fowler turn in their graves. But be careful: Dont get cute. Writing a sonnet or a short one-act play is probably not a good idea. You should, however, bring all your skills as a writer to bear on the essay topic. After all, thats why the question is an essay question, rather than a true/false or short-answer.
- Answer the Question. Let me repeat: Answer the question. If you write page after page of text, but never really address the issue at hand, few professors are likely to give you much credit. Always keep your overall point in mind, and make sure that everything you write relates back to your central argument. And that argument, in turn, should squarely address the question posed on the exam.
© Copyright 1996, Georgetown University