.

Teaching and Learning Resource (TLR)

1. Title

Evaluating the Credibility of Knowledge Claims

2. Keywords

Authoritativeness; Credibility; Knowledge claims / claims-making

3. Introduction

This TLR is part of a series in which students are encouraged to think critically about the problematic nature of knowledge, including scientific knowledge; and to think about the production of knowledge as a process of 'making knowledge claims' (or 'claims-making'). On this view of knowledge, all knowledge claims - be they positive or normative - are seen to be subject to (actual or potential) contestation. (For a brief account of the thinking behind this view of knowledge, see the Classifying Knowledge Claims TLR and its accompanying Briefing Sheet.)

If knowledge is seen in this light, then it is important that students are able to evaluate the credibility of knowledge claims (ie the extent to which they are believable) - which may, in part, derive from the authoritativeness of the claims-maker (ie the extent to which a claims maker has the necessary expertise or status to make such claims). (See accompanying Briefing Sheet.) To do this, they need to adopt a reflective approach to their reading in general. More specifically, they need to adopt a critical approach to the assessment of arguments and evidence, and the context within which those arguments and evidence were produced.

4. Aim

This TLR aims to help students develop their ability to evaluate the credibility of knowledge claims; and, more generally, to promote in students a more critical approach to the reading of academic - and other - literature. It is based on an exercise in which students are required to evaluate the knowledge claims made in a piece of published academic writing.

5. Learning outcomes

After using this TLR, students should:

• be able to analyse arguments in terms of the knowledge claims from which they are constructed;
• be able to evaluate the credibility of individual knowledge claims in relation to the nature of the supporting evidence and/or arguments upon which they are based - and the context within which the knowledge claims were produced;
• be able to evaluate the credibility of a work as a whole, based on their evaluation of its constituent knowledge claims.

6. Pre-requisites

Students should be familiar with the idea that the production of knowledge can be seen as a process of 'making knowledge claims'; and should be able to distinguish between different kinds of knowledge claim and between the different kinds of source upon which knowledge claims can be based. Students should also have some idea of the ways in which bibliographic citation contribute to the credibility - or otherwise - of knowledge claims. (See Section 13.)

7. How to use TLR

The TLR has been designed to be used in two stages. In the first stage, the students should be introduced to the TLR, given the two pieces of stimulus material (see Section 9) and instructed to undertake some preparatory work. The should be carried out as an individual activity, but could take place during class time or private study. In the second stage, students are required to work in groups and then take part in a whole-class discussion. This should take place during class time.

Stage 1

1. Introduce the TLR, reviewing the nature of the content, the aims and learning outcomes, and the way in which the TLR will be used. (10 mins)
2. Give the students the two items of stimulus material (see Section 9). Direct them to read both, and then analyse the piece of academic writing (with reference to the Briefing Sheet) according to the following instructions:

• Identify the key knowledge claim(s) in each paragraph of the stimulus material, and make a note of whether each one is (largely) positive or normative.
• Make a note of the extent to which each of the claims is widely 'known', uncertain, provisional or contested.
• Make a note of the credibility of each claim, bearing in mind the nature and source of any supporting evidence and arguments. Where appropriate, suggest possible ways in which claims could be made more credible.
• Make a note - insofar as you are able to - of the extent to which the claims (in general) might be influenced by factors such as the institutional or socio-political context of claims-maker/s.
• Make a note on the credibility of the work as a whole, bearing in mind the credibility of its constituent knowledge claims and the authoritativeness of the claims-maker/s.

Stage 2

1. Divide the class into groups of 4-6 students, and ask group members to compare the notes they produced during Stage 1 of the exercise. In particular, they should discuss and, if possible, resolve any areas of disagreement. (40 mins)
2. Ask each group to report back on its evaluation of the knowledge claims contained in the stimulus material. They should not comment mechanically on every single knowledge claim they have discussed but, rather, focus on:

• any difficulties they encountered;
• any areas of unresolved disagreement within their group. (40 mins)

3. Conclude the session with a general discussion of issues that have arisen. This might involve asking questions such as:

• What is it that makes some claims makers more authoritative than others?
• What is it that makes some knowledge claims more credible than others?
• What is the relationship between the authoritativeness of claims makers and the credibility of their knowledge claims?
• When comparing competing knowledge claims (eg different theories regarding BSE), would you always accept claims made by the (apparently) more authoritative claims maker in preference to a less authoritative one?
• How would you relate your ideas on credibility and authoritativeness to questions concerning the problematic nature of scientific 'proof'? (This question would probably only be appropriate for students with some knowledge of the philosophy of science.)
• How would you relate your ideas on credibility and authoritativeness to Thomas Kuhn's (1962) ideas on paradigm shifts and, more generally, to the view that all knowledge is socially constructed? (This question would probably only be appropriate for students with some knowledge of the sociology of knowledge/science.)

As described above, the TLR could be used with classes of up to about 40 students. Stage 1 would require 10-15 minutes of class time and, perhaps, an hour of private study. Stage two would require about two hours of class time. However, the time required could be reduced by selecting a suitably short item of academic writing for the analysis.

Follow-up activities

This exercise could be repeated with other examples of published work until the learning outcomes have been achieved. For the sake of variety, these could be drawn from different academic disciplines (spanning the natural, physical and social sciences) and/or from non-academic sources, such as governmental bodies, NGOs and private sector organisations. The same kind of analysis could also be applied to popular writings on the environment, and even the content of television documentaries.

Achievement of learning outcomes could be reinforced by encouraging students to do at least some of the recommended reading. (See Section 15.)

8. Instructions to students

As directed by tutor.

9. Stimulus Material

This TLR involves two items of stimulus material. The first is a Briefing Sheet entitled Evaluating Knowledge Claims (see Appendix). The second is a piece of academic writing. This could, in principle, be any piece of published academic writing (article or book chapter), provided that its subject matter and style of writing are accessible to the students. In practice, the exercise is likely to work best if:

• the subject matter of the work is directly relevant to the students' programme of study, is highly topical or is of great general interest;
• the work is relatively short, but contains a diverse mixture of positive and normative knowledge claims;
• some of the knowledge claims are controversial, and are likely to be contested by the students.

If the students have previously completed the Classifying Knowledge Claims TLR, it might be beneficial to use the same stimulus material.

10. Degree stage

In principle, this TLR could be used with students at any degree stage - provided that they satisfied the pre-requisite requirements. (See Section 6.) In practice, however, it is likely that students at levels two or three would get more out of the exercise than those at level one.

11. Resource requirements

This TLR has no special resource requirements.

12. Preparation

See Section 7.

13. Links with other TLRs

Where students are unfamiliar with the idea of 'knowledge claims' and unable to identify different kinds of knowledge claim and different kinds of source upon which knowledge claims can be based, it is recommended that the Classifying Knowledge Claims TLR is used prior to this one.

Where students unaware of the ways in which bibliographic citation can contribute to the authoritativeness - or otherwise - of knowledge claims, it is recommended that the Bibliographic Citation for Authoritative Academic Writing TLR is used prior to this one. Taken together, these three TLRs can be seen as preparation for the Producing Credible Knowledge Claims TLR.

More generally, the aims and/or learning outcomes of this TLR are related to those of other TLRs listed in the following 'thematic cluster':

• Critical thinking skills

14. Follow-up activities

See Section 7.

15. Recommended reading

Chalmers, Alan (1988) What is this Thing called Science? (2nd edn.) Open University Press, Milton Keynes.

Kuhn, Thomas (1962) The Structure of Scientific Revolutions. University of Chicago Press.

16. Users' comments

• Some student comments on a hybrid version of this TLR and Producing Credible Knowledge Claims:

“Quite challenging - right level for second year students.”

“The seminar was very interesting because I actually got to think carefully.”

“ [The] lecture was interesting but probably the most difficult one of the course.”

“[I learned] how to be critical of sources.”

Appendix

Briefing Sheet: Evaluating Knowledge Claims

1. Introduction

There is no simple recipe for evaluating knowledge claims, but in general it may involve some combination of:

• evaluation of the claims themselves, with reference to any supporting evidence and/or arguments (see Sections 2 and 3);
• consideration of the claims maker(s) and the context within which the claims are being made (see Section 4).

2. Evaluating Positive Knowledge claims

Evaluation of positive knowledge claims generally involves assessing the credibility of evidence and the soundness of argument upon which claims are based. Consider, for example, the following claim:

Most scientists agree that mean sea levels will rise by 2m during the next century.

The first step in evaluating this claim would involve checking the credibility of any sources cited in support of the claim. If this proved to be inconclusive, one might then go directly to the relevant scientific literature (in this case, searching under keywords such as 'climate change', 'global warming' or 'greenhouse effect') to see whether they provided support for - or against - the claim.

Assessing the soundness of an argument can be illustrated by considering the following series of claims:

The scientists who are now proclaiming that the Earth is getting warmer as a result of carbon dioxide emissions are the same people who also claimed, just a decade or so ago, that the Earth was getting cooler. It follows that they don't know what they are talking about and, therefore, that global warming simply isn't happening.

It is possible that all of the individual statements in this argument are credible (ie that the scientists did previously believe that the Earth was getting cooler; that they are/were wrong; that global warming isn't occurring), but the argument, as presented, is not a logical one. The fact that the scientists may have changed their minds does not necessarily mean either that they don't know what they are talking about or that they are now wrong. Therefore, we cannot know, on the basis of what is presented here, whether global warming is occurring or not. In other words, the conclusion (about global warming) does not follow from the premises (about the changing views of the scientists).

3. Evaluating Normative Knowledge Claims

Evaluation of normative knowledge claims may also involve assessing the soundness of an argument. For example, consider the following argument:

Since all individual animals are equally valuable, the government has decided to do whatever it takes, at whatever cost, to protect the last few pairs of breeding golden eagles.

In this argument, the decision "to do whatever it takes" to protect the eagles suggests that the government places an exceptionally high value on the remaining eagles and, by implication, that it sees them as more valuable than, say, an equivalent number of rats. Thus, the policy does not appear to be consistent with the stated value-position. A more consistent approach might be:

Since all individual animals are equally valuable, the government has decided not allocate resources to the protection of the last few pairs of breeding golden eagles.

The argument cannot now be challenged on grounds of consistency, but the decision could be challenged if one disagreed with the value-position upon which it is based. For example, if one believed - for whatever reason - that it was important to preserve species, then one could argue for the allocation of resources to protect the eagles. However, it is difficult - and some would say impossible - to make absolute judgements about the validity of the specific values underlying particular normative knowledge claims.

4. Assessing Claims Makers and Context

Evaluation of knowledge claims should, in general, involve some consideration of the authoritativeness of the claims maker. There is no simple way of deciding whether or not a claims maker is authoritative, but the claims maker's authority, experience, expertise, power, qualifications or right to make such claims may be relevant. It is important to notice that a particular claims maker may be regarded as being authoritative when making some kinds of knowledge claim, but not others. For example, one might regard a professor of epidemiology as an authoritative claims maker when talking about epidemiology but not (necessarily) when talking about economics.

Evaluation of knowledge claims should also involve some consideration of the context within which clams are made. Studies in the history, philosophy and sociology of knowledge / science suggest that all knowledge claims reflect - to some extent - the historical and cultural circumstances under which they are produced. For example, claims makers may be influenced - consciously or otherwise - by:

• their own personal circumstances;
• their own expectations concerning research outcomes;
• peer pressure;
• prevailing 'knowledge commitments' (ie commitments to particular ideas, theories, paradigms or worldview) within their discipline, intellectual community, or society more generally.
• vested interests of the institution or organisation for which they work;
• vested interests of institutions or organisations which commission, sponsor or otherwise support their work - or may do so in the future;

Consideration of factors such as these seems especially important in the environmental field. This is because environmental knowledge claims are generally concerned with human and natural systems, whose interactions are:

• usually complex;
• rarely localised in terms of space or time;
• constantly changing - sometimes quite rapidly - as a result of quantitative and qualitative changes in human activity (eg the development of novel processes and substances).

Moreover, these knowledge claims are very often produced within a context of hotly contested social and environmental values and competing economic and political interests.