Importance of Cross-Cultural Instrument Validation
For years the only concern for most researchers was to obtain data from sample or representative populations.  More often than not cultural differences were not considered and were not accounted for in early studies.  The belief was that everyone would respond the same to the questionnaires or react in the same way to a specific event.  This incorrect belief caused a lot of theories to be deemed credible and others to be deemed not credible for fallible reasons.  At the time, the world was very separated and isolated, especially the United States.  This isolation led to researchers in the United States to make assumptions that again were only viable for the American citizen.   When America opened their doors to more immigrants, these same research practices were still used on the new residents.  This often led to incorrect diagnosis or incorrect theories on development.  Eventually it all became obvious that cultural, religious and ethnic differences could skew the information in the research and create incorrect and unusable data.  This is when the need to validate instruments of research cross-culturally.  This is a new concept and is still in the beginning stages of understanding, but basic foundations have led to basic guidelines that are helping researcher gain better data and hence better information and theories from the cross-cultural validated research.

The Beginning

All research needs to validate the construction as a tool that can measure concepts or behaviors.  The early researchers, in every country, based their standards on the people within their countries and believed that these standards would follow across nationalities and borders.  Even those in the United States believed that their standardizations and construct validations of their research tools would fair well in all other countries.

Part of this belief came from Jeremy Bentham (1748-1832) an English philosopher who created the “Theory of Fictions” (Maher, & Gottesman, 2005).  This theory had two basic tenets.  The first was “real entities” which were the behaviors and tangible aspects of the world around the person that could be understood through the use of the five senses.  The second was the “fictitious entities” where were the ideas and concepts, and were not tangible to the person.  Bentham believed people viewed these fictitious entities with the same level of understanding as the real entities thus causing misunderstanding (Maher, & Gottesman, 2005).

Hans Vaihinger took the theory one step further with his “As If” theory (Maher, & Gottesman, 2005).  This theory took the fictions and hypothesized that people knew they were false but were necessary for individuals to conceptualize ideas and circumstances.  Francis Galton, expanded the previous theories and added a bit of Darwin’s theory for his “Nature’s Intentions” (Maher, & Gottesman, 2005).  Nature’s intention was the concept that nature was trying to create the perfect human or the mean value of humanity.  The deviations from the mean value of humanity were the errors in the individual and therefore, there was not such thing as perfection in humanity.  So like those before him, there were factors that influenced the humans that were fallible (Maher, & Gottesman, 2005).

Gordon Allport started working in 1937 with personality traits of people and how they related to understanding psychological advancement.  His theory was based on the fact that each person had specific traits that would be the same in any circumstance and any country.  It was not long before his theory was discredited with ideas that different influences would create different traits (Maher, & Gottesman, 2005).  The concept did not take hold right away, but it laid the groundwork, so that when it became apparent that immigration of new cultures changed the data on tests and research that had previously been credible new ways of looking at the every changing population could be integrated into the research field.  It was not an over night change and it is still being developed, but the early work and theories help with understanding the differences that need to be accounted for in research.

Validation of Research Instrument across Cultures

The world is much smaller today than it was even 30 years ago.  With air travel, the internet, and other forms of transportation and communication, people are able to go or talk to people anywhere in the world and even into outer space to the Mir Space Station.  The influx of immigrants and traveler around the globe has taken it toll on psychological research.  Tools that were created with the average citizen of a specific population no longer work in helping to understand the population (Hitchcock, Sarkar, Nastasi, Burkholder, Varjas, & Jayasena, 2006).   In fact, the non-English speaking child population was up to 18% of the overall child population in 2000 and continues to increase (Peña, 2007).   This has caused a major problem for educators because the development of the non-English speaking children is not necessarily on the same line with children born to English speaking parents (Peña, 2007).

Another area that shows the need to create cross-cultural valid instruments is the Korean Women’s Abuse Intolerance Scale (KWAIS) which measures the beliefs of Korean women in relation to their tolerance to abuse or their desire to leave the abusive situation (Choi, Phillips, Figueredo, Insel, & Min, 2008).  This measurement showed that the underlying traditions and the teaching of Confucianism have taught Korean women to accept their husbands’ abuse and heavy drinking.  It has only been in recent years that Korean women have become intolerant, but the majority still tolerate and stay in the relationship.  The main reason is that if a Korean woman divorces a Korean man than she is shamed and it is her fault even if she left due to abuse.  Tradition holds that shame is not acceptable, so the woman stays and takes what ever is put in her path (Choi et al, 2008).

If researchers in the United States were to use the KWAIS to understand the tolerance of abuse for American women, the tool would not work for the fact that American women do not hold the same stigma of divorce or shame or have traditions that advocate abuse in the marital relationship as Korean women.  This does not mean abuse does not happen or that it is not tolerated by some, but the majority would get out of the situation.  Therefore, the KWAIS would have to be revamped and reworked to be a useful tool in the United States, or elsewhere in the world (Choi et al, 2008; Hitchcock et al, 2005).

One fact that has been discovered is that if the non-English speaking person is not a first generation or came to the United States at a young age, then the traditions of the home country are not as ingrained.  While the generalized American tests still do not work perfectly, the longer the person stays in the United States or the farther the generation from first arrival, the more they acclimate and become more Americanized.  It is at this time that the standardized tests do not show much of a difference in the measurements and can be used.  The fact is that even at this stage the test used should be across many cultures to ensure that the information is up-to-date and accurate especially in understanding childhood development (Kuo, Roysircar, & Newby-Clark, 2006).

Equivalence in Research Tools

There are five areas that need to be equivalent when making a research tool valid across cultures.  The use of these equivalences help to standardize the tool, gain construct validity, and help to ensure the data is informative and useful in determining theories and concepts.  The fact is that using only one equivalence will not give you a cross-culturally valid construct.  All four have to be use in order to have the construction of the measurement instrument validated (Peña, 2007).

The first equivalency is linguistic.  Technically this means a translation from the originating language to another language.  There are two types of translations that are used in research to ensure construct validation.  The first is that the publisher of the tool translates it and then has one or more native speakers of the language verify the translation for accuracy and understanding.  The second type is the translation and back translation.  This is a little more detailed and gives a more accurate translation.  The tool is translated by one or more experts, then one or more different experts retranslate the document back to the original language.  Then the two original language documents are compared and any discrepancies are fixed on the translation (Peña, 2007).

Function equivalency is the second necessary phase.  The functional phase works to ensure that the translation is the same content and context of the original language.  Some languages have several words that have similar meanings, which if the wrong word is used will change the context of the question or tool.  One way researchers are learning to gain functionality of their construct is by constructing two different language tools at the same time.  This ensures the functionality and the linguistic challenges will be met and overcome (Peña, 2007).

The culture itself can be an obstacle that needs to be considered.  This difference also includes the vocabulary differences in differing geographical locations as well as traditions and beliefs associated with the country or location within the country.  For example, Spanish has many dialects in Mexico alone, but moving into Central and South America gives even more dialects.  These differences need to be considered when creating the construct for validation purposes and to make the finding of cross-cultural research credible (Peña, 2007).

The last is the metric and psychometric phases of equivalency.  The metric equivalency is the level of content must be the same for all translations of the construct.  The psychometric equivalency allows for cross-cultural comparisons and development of theories across borders.  This also takes into account cultural differences and allocates them to ensure that comparisons are in the same context and the construct validity remains intact (Peña, 2007).

Steps to Ensure Instrument Validation Cross-Culturally

Basic guidelines for translation and adaptability have been created and published in 1992 by the International Test Commission (ITC) (Hambleton, 2001).  The commission broke the steps into four steps or phases that need to be used with the equivalencies to ensure that the instruments are valid.  The first is the context of the test.  This means that cultural differences need to be accounted for and the test needs to be adjusted to the differing cultures without losing the meaning of the instrument or the construct validity (Hambleton, 2001).

The development of the test and the adaptation is the next phase that must be completed.  This ensures that the culture, language, responses, statistical measurements and construct validity remain the same for each adaptation.  From there, the administration techniques are considered.  This would include the possible problems associated with different cultures, the type of material used in the research, the way in which the data is collected, and objectivity of the proctor or research interviewer.  The last stage is the interpretation of the scores on the documentation.  This is the detailed part of the process.  The researcher must account for all possible variances between the cultures and the subjects, remembering that a true comparison is not possible due to cultural and language barriers, but that basic assumption can be used to help understand the basic concepts of the research (Hambleton, 2001).

Cross-Cultural Studies

There are many instruments available that have used the equivalences and the ITC guidelines to create viable and useful studies with valid constructs that cross cultural lines.  One such is the Posttraumatic Stress Symptoms in Children (PTSS-C).  This instrument was created to cross cultural lines to diagnose and identify the circumstances that cause Post-Traumatic Stress Disorder (PTSD) in children.  The test was given to three different populations to validate the construct and measurements of the test.  The first group was Iraqi Kurdistan “Anfal” survivors, Kurdistan refugees in Sweden, and a sample group of Swedish children. The internal constituencies, measurements, and construct validity were significant and information across the cultural lines and yielded great results in the ability to create and administer a valid construct instrument across different cultures (Ahmad, Sundelin-Wahlsten, Sofi, Oahar, & von Knorring, 2000).

A second research group was tasked with creating and developing a cross-cultural dementia research instrument for use in several developing countries.  Their development and research led to the identification of several problems in working across cultures.  The first was the low education level of many of the test subjects.  The second major problem was that it was almost impossible to create a “culture-free” instrument (Martin, Acosta, Chiu, Scazufca, & Varghese, 2003).  At some point even with translations, and trying to incorporate the guidelines, the task proved to be daunting and the data collected show the cross-cultural problem.  The researchers also mentioned that other factors such as depression could go undiagnosed and skew the data (Martin, Acosta, Chiu, Scazufca, & Varghese, 2003).

Conclusion

Cross-cultural instrument construct validation is more important now then ever before, and the need to have research tools to use anywhere with anyone will become more prevalent.  The main idea is to ensure that the sample population understands the instrument and therefore giving accurate results about the targeted population.  With so many people moving to and from other countries, the populations are more mixed and their traditions and cultures have to be taken into consideration to understand the population.  By ignoring other culture, the researcher is doing a disservice to him/herself and their study.  Cross-cultural validation is a necessity and as more is learned in the process the guidelines will become more detailed and the instruments will become more reliable.

References
Ahmad, A., Sundelin-Wahlsten, V., Sofi, M.A., Oahar, J.A., & von Knorring, A.L. (2000). Reliability and validity of a child-specific cross-cultural instrument for assessing posttraumatic stress disorder.  European Child & Adolescent Psychiatry. 9(4). pp. 285-295

Clark, L. A., & Watson, D. (1995). Construction validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319.

Choi, M., Phillips, L. R., Figueredo, A. J., Insel, K., & Min, S. (2008). Construct validity           of the Korean women’s abuse intolerance scale. Nursing Research, 57(1), 40-50.

Dean, E., Caspar, R., McAvinchey, G., Reed, L., & Quiroz, R. (2007). Developing a

            low-cost technique for parallel cross-cultural instrument development: The           question appraisal system (QAS-04). International Journal of Social Research                       Methodology, 10(3), 227-241.

Hambleton, R. K. (2001). The next generation of the ITC test translation and adaptation            guidelines. European Journal of Psychological Assessment, 17(3), 164-172.

Hitchcock, J. H., Sarkar, S., Nastasi, B. K., Burkholder, G., Varjas, K., & Jayasena. A. (2006). Validating culture and gender-specific constructs: A mixed-method approach to advance assessment procedures in cross-cultural settings. Journal of Applied School Psychology, 22(2), 13-33.

Kuo, B. C. H., Roysircar, G., & Newby-Clark, I. R. (2006). Development of the

            cross-cultural coping scale: Collective, avoidance, and engagement coping.          Measurement and Evaluation in Counseling and Development, 39(3), 161-181.

Maher, B. A., & Gottesman, I. I. (2005). Deconstruction, reconstruction, preserving

            Paul E. Meehl’s legacy of construct validity. Psychological Assessment, 17(4), 415-422.

Martin, P., Acosta, D., Chiu, H., Scazufca, M., & Varghese, M. (2003, Mar 15). Dementia diagnosis in developing countries: a cross-cultural validation study. Lancet. 361(9361) pp. 909-918

Peña, E. D. (2007). Lost in translation: Methodological considerations in cross-cultural   research. Child Development, 78(4), 1255-1264.

Shoeb, M., Weinstein, H., & Mollica, R. (2007). The Harvard trauma questionnaire: Adapting a cross-cultural instrument for measuring torture, trauma and posttraumatic stress disorder in Iraqi refugees. International Journal of Social Psychiatry, 53(5), 447-463.