Factors Impacting Credibility

People nowadays have access to information and research results everywhere they go, but it has become more frequent that a lot of information and research results gained are incorrect or misleading. This puts a lot of pressure on designers and researchers to improve the credibility of information and results gained from research. 

There are many different issues that will affect the credibility of research results, and some are even considered before reading the results or information, the first credibility indication are perceived very quickly. They way the content is presented and designed affects the credibility straight away, content needs to have clear layout and easy to access, so everyone can see everything related to the information or how certain results are obtained. Nearly 75% of respondents reported making credibility judgments based on the content presentation rather than by evaluating the content’s or creator’s authority, trustworthiness, reputation, or expertise.

http://www.usability.gov/articles/102009news.html#intro

Improving Credibility through Triangulation

In qualitative research it is common to see researchers employ a triangulation strategy to improve the credibility of their inquiry. Denzin (1988) outlined four distinct types of triangulation:

• Data
• Observer
• Methodological
• Theory

In some cases, ‘multiple triangulation’ is used; this uses a combination of two or more techniques.

Data:

This technique can be broken down into three different types; these are: time, space and person. Time triangulation requires the researcher to alternate the times at which data is collected. Space triangulation requires the researcher to alternate where the data is collected. Person triangulation requires the researcher to collect data from a wide variety of individuals/groups.

Observer:

This procedure requires multiple researchers with divergent backgrounds to work together in prominent roles within the same study. Each researcher collects their own data; they are not given the data. Once each observer has collected their data, they meet and reach a conclusion.

Methodological:

This strategy involves multiple methods for collecting the data. The aim of this technique is to achieve a more holistic and greater understanding of the phenomenon under investigation. This is an excellent method of evaluating the authenticity of the data collected, thus increasing the collected data’s credibility. This is the most time consuming and costly triangulation technique.

Theory:

This technique requires that multiple theories are explored during the analysis of the findings. As there is no perspective that fits any situation, it is ideal to explorer multiple theories to gain a better understanding on the data collected. It is thought that ‘the best perspective is one which contains many perspectives’

References:

http://www.delmar.edu/socsci/rlong/intro/perspect.htm

http://www.qualres.org/HomeTria-3692.html

The Credibility of the Researcher

The researcher themselves is the most important part of a research project so it is important that a research report includes information about the researcher.

Examples of information to include:

•   What experience, training, and perspective does the researcher bring to the field?
•   What personal connections does the researcher have to the people, program, or topic studied?
•   Who funded the study and under what arrangements with the researcher?
•    How did the researcher gain access to the study site?
•  What prior knowledge did the researcher bring to the research topic and the study site?

There can be no definitive list of questions that must be addressed to establish the researcher’s credibility. The important point is to report any personal and professional information that may have affected data collection, analysis, and interpretation either negatively or positively in the minds of users of the findings.

Background characteristics of the researcher (e.g., gender, age, race, ethnicity) may also be relevant to report in that such characteristics can affect how the researcher was received in the setting under study and related issues.

For research to be credible it is vital that the results show who the researcher was and information about why he carried out this research. If there was any relationship between the study area and the researcher before the research was conducted it must be explained.

references:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089059/pdf/hsresearch00022-0112.pdf

Bias in research results

Research bias, also called experimenter bias, is a process where the investigator performing the research influences the results, in order to portray a certain outcome. This can have a huge effect on the credibility of the results because without fully explaining how the results limited or eliminated bias the results cannot possibly be trusted. 

Some bias in research arises from experimental error and failures to take into account all of the possible variables. Other research bias arises when researchers select subjects that are more likely to generate the desired results, a reversal of the normal processes governing science.

Research bias is the one factor that makes qualitative research much more dependent upon experience and judgment than quantitative research. (Quantitative Research denial of any Research Bias, Qualitative Research Bias Acceptance and Acknowledgment of Research Bias)

The main point to remember with research bias is that, in many disciplines, it is unavoidable. Any experimental design process involves understanding the inherent biases and minimizing the effects. In quantitative research, the researcher tries to eliminate bias completely whereas, in qualitative research, it is all about understanding that it will happen.

One thing that can influence bias is if the project is funded. The source of funding for a research project may bias the reporting of results. The funding source is usually included in the results. When reading an article from a funded research project, you must consider whether the funders of this research had anything to gain by the results. When research is paid for by a source that does not have an interest in the results, funder's expectations or preferences were less likely to have influenced the results.

References:

http://www.experiment-resources.com/research-bias.html

http://www.cdc.gov/des/consumers/research/understanding_deciding.html

Ensuring Credibility of Qualitative Research

It is hard to ensure that qualitative research is accurate or correct; therefore the following 4 aspects need to be taken into consideration when conducting qualitative research:

• Credibility/trustworthiness
• Transferability
• Dependability
• Confirmability

These terms are used by Lincoln and Guba, to replace ‘reliability’ and ‘validity’, which are usually linked to quantitative research.

Credibility/trustworthiness:

The credibility is involved in establishing that the results of the research are believable. This is a classic example of ‘quality not quantity’. It depends more on the richness of the information gathered, rather than the amount of data gathered. There are many techniques to gauge the accuracy of the findings, such as data triangulation, triangulation through multiple analysts and ‘member checks’. In reality the participants/readers are the only ones who can reasonably judge the credibility of the results.

Transferability:

Transferability refers to the degree in which the research can be transferred to other contexts; this section is defined by readers of the research. The reader notes the specific details of the research situation and methods, and compares them to a similar situation that they are more familiar with. If the specifics are comparable, the original research would be deemed more credible. It is essential that the original researcher supplies a highly detailed description of their situation and methods.

Dependability:

Dependability ensures that the research findings are consistent and could be repeated. This is measured by the standard of which the research is conducted, analysed and presented. Each process in the study should be reported in detail to enable an external researcher to repeat the inquiry and achieve similar results. This also enables researchers to understand the methods and their effectiveness.

Confirmability

Confirmability questions how the research findings are supported by the data collected. This is a process to establish whether the researcher has been bias during the study; this is due to the assumption that qualitative research allows the research to bring a unique perspective to the study. An external researcher can judge whether this is the case by studying the data collected during the original inquiry. To enhance the confirmability of the initial conclusion, and audit trail can be completed throughout the study to demonstrate how each decision was made.

References

http://www.socialresearchmethods.net/kb/qualval.php

http://www.qualres.org/HomeLinc-3684.html

http://www.angelfire.com/theforce/shu_cohort_viii/images/Trustworthypaper.pdf

Reliability and Validity!

To be credible the research needs to be reliable and to have validity. So to understand the credibility of research it is important to know what each are:

What is reliability?

Reliability is the extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable.

What is validity in quantitative research?

Validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are. In other words, does the research instrument allow you to hit "the bull’s eye" of your research object? Researchers generally determine validity by asking a series of questions, and will often look for the answers in the research of others. 

Credibility can be strengthened

Credibility of research results can become more credible and trust worthy by;

• Providing an accurate and complete description of the results.
• Modifying your explanation of the results depending on your involvement in the research.
• Member Checking.
• Peer debriefing and support.
• Accessing data which does which does not fit with your explanation.
• Using reflexivity to identify possible bias.

Credibility Issues in Qualitative Research

While researching are chosen topic area (credibility of research) I stumbled across this video on youtube about the credibility issues in qualitative research. It gives explanations of many credibility issues such as who carried out the research, and  are the findings believable.

I think watching this short video is a good starting point on are subject.
What do you think about it?
Any other ideas?
are there any different credibility issues in quantative research compared to qualitative?