Perceived Credibility , Evaluation of Outcome and Attitude towards Behavioural Change in 360-Degree Assessment

This study analyzes how Perceived Credibility of a 360-degree feedback and Evaluation of Outcome impact on ratees’ attitudes towards behavioral change. The study adopts a qualitative-quantitative mixed method approach. Reasoned Action Theory was selected as the theoretical framework, and ten 360-degree assessment ratees were interviewed on their perceived credibility, evaluation of outcome and attitude toward behavioral change in the 360-degree assessment practices. The qualitative inputs were analyzed under the methodology of grounded theory. Additional 159 respondents participated in the quantitative research phase of the research to validate the findings in the qualitative research phase. It was validated that Perceived Credibility, Evaluation of Outcome has impact on Attitude Toward Behavioural Change in 360-degree assessment.

INTRODUCTION 360-degree assessment is a process or approach to gather perceptions from various groups of raters that include the supervisors, colleagues, direct reports and other sources such as customers and suppliers in reference to one employee in an organization (Bracken, 2009;Chakraborty & Rajkumar, 2005;Potočnik & Anderson, 2012;Tyson & Ward, 2004).
The notion of people receiving feedback on their behaviour and productivity is not an entirely new concept.When the practice of giving feedback initially emerged in the business world, the source of the feedback largely came from the focal individual's supervisor or the business owner (Church & Bracken, 1997).With the advent of the 20th century, descriptions of working conditions presented us with a picture of the feedback being given largely focused on productivity.In addition, the feedback being given often came at the whim of the boss and most probably in negative forms considering that it was often at times when businesses were not going well.Multi-source feedback appraisal gradually became popular in organizations in the 1980s when it was mainly used as an executive development tool (Coates, 1996;Huggett, 1998).Although upward appraisal and peer appraisal had received some attention in the 1970s and early 1980s, it was not until the last decade of 20th century that 360-degree feedback received wider attention (Coates, 1996;Hegarty, 1974).360-degree assessment has become a popular management and leadership development tool today, and is found in many organizations (Denison, Kotrba, & Castano, 2012;Fiscus, 2011;Greguras, Ford, & Brutus, 2003).Millions of dollars have been invested by organizations worldwide on 360-degree assessment administration (Atwater & Yammarino, 1996), and every Fortune 500 firm is either conducting 360-degree assessment or thinking about it (Lepsinger & Lucia, 2009;London & Smither, 1995).Successful 360-degree assessment practice creates enhanced self-awareness that leads to effective career development activities.Lepsinger and Lucia (2009) suggested 360-degree assessment is like a mirror to the ratee, collecting perceptions about the ratee's behaviors from multiple sources in order to change and improve ratee's behavior at workplace.The changed and improved behavior relies on what the ratee will do after the assessment (Maurer & Palmer, 1999).Attitude is one of the key factors that decide behavioral change following 360-degree assessment (McCarthy & Garavan, 2006).

LITERATURE REVIEW
The Greek philosopher Aristotle defined three categories of persuasion-Ethos (character), Pathos (emotion appeal), Logos (logical appeal), and the Greek word "Ethos" which stands for character, is the origin of credibility (Woodward & Denton, 2000).In addition to the ancient philosopher's conjecture, modern research has already clearly demonstrated the link between credibility, persuasion, and the impact on affective and behavioural response.For example, the source credibility theory declares that it is easier to persuade people when the source presents itself as credible (Hovland, Janis, & Kelly, 1953;Ohanian, 1991;Petty & Cacioppo, 1986;Sternthal, Phillips, & Dholakia, 1978), and sources with high credibility ensure a more positive attitude and induce more commitment than sources with low credibility (Hovland & Weiss, 1951;Ohanian, 1991;Petty, Cacioppo, & Schumann, 1983).In the context of 360-degree assessment, the ratees' behaviours after the assessment depends on the credibility of the raters (Bracken & Rose, 2011;Taylor & Bright, 2011).Ajzen and Fishbein's (1980) Theory of Reasoned Action suggests an individual's intention to perform a behaviour as the direct determinant of the actual behaviour.Such intention is a result of two other determinants, namely "attitude" and "subjective norm".Attitude is referred to as the personal factor of "positive or negative evaluation of performing the behavior (p.6), and "subjective norm" is referred to "the person's perception of the social pressures put on him to perform or not perform the behavior (p.6)".Evaluation of outcome (EOO) is one factor contribute to attitude and is defined as 'an individual's belief that a behavior (change) leads to certain outcomes, and his judging and computing on how beneficial these outcomes are; it is the determinant of an individual's 'attitudes toward such behavior (change)' (Ajzen & Fishbein, 1980).Ajzen and Fishbein (1980) pointed out that there are external factors which are not included in the Theory of Reasoned Action but may influence the beliefs a person holds or the relative importance they attach to attitudinal and normative considerations.They mentioned three types of external variables namely Demographic Variables, Attitudes toward targets, and Personality Traits and called for further research on those external variables.By introducing another variable, behavior control, into the model, Ajzen (1991) formulated the Theory of Planned Behavior, which suggests attitude, subjective norm and behavioral control are the three determinants of intention toward a certain behavior.McCarthy and Garavan (2006) applied Ajzen and Fishbein's work to explain behavior changes following a 360-degree feedback and found that organizational support and attitude towards behavioral change were significant predictors for behavior change following a 360-degree feedback while locus of control was not.McCarthy and Garavan considered that attitude in 360-degree assessment practice consist of two factors: feedback accuracy and cynicism.These two attitudinal factors significantly affect the raters' behavior changes after the 360degree assessment practices: cynicism (β = -0.17,p < .001)and feedback accuracy (β = 0.09, p< .05).

METHODOLOGY
The research adopted a qualitative-quantitative mixed method.The research collected, analyzed, and integrated data gathered from quantitative and qualitative study during the two phases of the research (Creswell & Plano Clark, 2010).The qualitative phase aimed to collect data based on participants' own categories of meaning, analyze the dynamic process and use the qualitative method of "grounded theory" to generate inductively a tentative but explanatory theory (Johnson & Onwuegbuzie, 2004).The quanitative phase aimed to test and validate the constructed theory and generalize the research finding to a larger population (Johnson, Onwuegbuzie, & Turner, 2007).Using the mixed research method, this research aimed to identify how Perceived Credibility (PC) and Evaluation of Outcome (EOO) affect the atttitude formation in 360-degree feedback.

A. Design
Semi-structured interview was adopted to gather qualitative data in this study because it enables participants to report the richness of their experiences (Fontana & Frey, 2000) and expression of their perceptions pertinent to their PC and EOO of a 360-degree feedback exercise.
The respondents in this study were recruited by means of purposive sampling (Neuman, 1997) so that respondents with the appropriate characteristics could be targeted.Purposive sampling enables "cases that are especially informative" (Neuman, 1997) to be selected from the population.Purposeful sampling is based on the assumption that the investigator wants to discover, understand and gain insight and select a sample from which the most can be learned" (Merriam, 1998, p. 61).One of the aims of this study was to understand from the ratee's perspective, how PC and their EOO in a 360-degree feedback context would influence their attitudes toward behavioural change.Hence only samples who had experienced 360-degree feedback as ratees (subjects receiving feedback) were selected.
Ten participants from two organizations participated in the interviews.All of them had participated in 360-degree assessment as the ratee 24 months prior to the interview.Two rounds of interviews were conducted for each participant.The participants were asked to reflect their personal experience in their past 360-degree feedback assessments, and share their perceptions towards PC, EOO and Attitude Toward Behavioural Change in 360-degree assessment.

B. Findings
11 components were identified to contribute the PC in 360degree assessment, namely: 1. Rater's understanding on the constructs of the 360-degree assessment tool.2. Reliability of the system and process.3. Working relationship between raters and the ratee.4. Assurance of confidentiality and anonymity to the raters.5. Objectivity of the system administrator and process.6. Holistic approach of the feedback report.7. Personal relationship between raters and the ratee.8. Credibility of the coach providing the report interpretation.9. Effectiveness of the system administrator and process.10.Efficiency of the system administrator and process.11.Demographic similarity between raters and the ratee.
The participants also demonstrated 5 different reactions towards how their PC and EOO affected their Attitude towards Behaviorual Change in 360-degree assessment: Group A: 2 participants would embrace positive atttitudes twoard behavioural change as long as they perceive the 360-degree assessment is credible; Group B: 3 participants would embrace positive attitudes toward behavioural change as long as they evaluate the behavioural change is benefitial to them; Group C: 2 participants would embrace positive attitudes toward behavioural change when they either perceive the assessment to be credible or they evaluate the behavioural change is benefitial to them; Group D: 2 participants would embrace positive attitudes toward behavioural change only when both PC and EOO are positive; Group E: 1 participant would always have a positive attitude towards the behavioural change regardless of PC or EOO.
During the interview, it was observed that participants have different preferences on PC and EOO.Group A participants expressed strong preference on PC: "credibility is a necessity", and "without credibility then there is no point in going through the (360-degree assessment) process".In contrast, Group B participants expressed strong preference on EOO: "only when I see there can a positive outcome then I see the meaning", "regardless if it (360-degree assessment) is credible or not, if the outcome just doesn't benefit anyone in the end, I would choose not to act on it." During the interview, it was also observed that the participants demonstrate different levels of optimism / pessimism.Such individual differences in their optimism and pessimism are related to their attitudes toward behavioral change in 360-degree assessments.Group C participants demonstrated strong optimism: they were "generally positive" and "tend to think positively and look at things positively."Group D participants demonstrated strong pessimism: they were more "skeptical about things" and do "not allow things to go wrong." Based on the findings, the researcher propose the following hypothesis to explain the observed phenomenon, that in 360degree assessment, a ratee's attitude towards behavioral change is affected by effect of PC ( ) and effect of EOO ( ), which is affected by the ratee's personal preference on PC ( ), preference on EOO ( ), optimism ( ⊕ ) and pessimism ( ⊝ ).

C. Design
The quantitative phase was designed to test Hypothesis 1.A survey was designed to collect information on demographic information (i.e.organization setup, business sector, hierarchical position, gender, years of working experience, organization tenure, and job tenure), last 360-degree assessment experience (i.e.time from last 360-degree assessment, purpose of last 360-degree assessment, PC, EOO and attitude towards behavioral change), personal preference (i.e. , , ⊕ and ⊝ ).7-point Likert Scales, the most common construct of questionnaire in social (Monette, Sullivan, & DeJong, 2010) and human subject research (Boslaugh, 2013), were used to capture the scaled data.Life Orientation Test Revised (LOT-R) (Glaesmer et al., 2012;Scheier, Carver, & Bridges, 1994) was designed to capture ⊕ and ⊝ .
The targeted population of the research was set to include the ratees of 360-degree feedback assessment.Such targeted population is large enough to be considered as infinite, and simple random sampling method was selected to ensure the representativeness in terms of the aspects to be measured.To achieve simple random sampling, the participants selected came from the targeted population, and they were selected independently (Anderson, Sweeney, William, Camm, & Cochran, 2013;Gravetter & Forzano, 2010).
In order to determine a sample size required for analysis, the researcher referred to Cohen's work on power analysis (Cohen, 1977(Cohen, , 1992)): power of 0.8 (power = 0.8) is an appropriate value for a wide range of behavioural research, a medium effect size (for regression analysis, f 2 = 0.15) is typical for social science, and a Type I Error is suitable to be set at traditional level (α = 0.05) (Cohen, 1977;Green, 1991).Thus in order to perform a linear regression analysis with two independent variables at this level (power = 0.8, f 2 = 0.15, α = 0.05, u = 2), minimum of 66 or 67 participants (Coper, 2013;Green, 1991) is required; in order to perform a hierarchical regression analysis with six set A independent variables and two set B entered independent variables (power = 0.8, f 2 = 0.15, α = 0.05, w = 6, u = 2), minimum of 73 participants (Coper, 2013) is required.For descriptive statistical analysis, a sample size of 100 is required to reach confidence level of 95% with standard error of 0.05 for simple random selection from an infinite population ("Sample size calculator," 2013).600 working professionals from varied industries were randomly selected and invited to participate in the questionnaire survey.159 out of the 215 people who responded had participated in 360-degree feedback as ratees, thus they were from the targeted population and qualified for analysis.They were from 18 different industries out of 20 industry categories by NAICS (North American Industry Classification System, 2012), with averaged 16.9 years of working experience, 7.4 years of tenure with current organization, and 5.91 years of tenure with current job.

D. Findings
The table below shows participants' reported PC, EOO and Attitudes toward suggested Behavioural Change in their last 360-degree assessment as a ratee.The results are captured on a 7-point scale -3 (negative) to 3 (positive).It was also observed that ratees who participated in 360degree assessments with appraisal purposes perceived the assessments significantly less credible than ratees who participated in 360-degree assessments with only developmental purposes (F = 13.963,p < 0.001).Hierarchical regression analysis was also performed to verify whether , and other variables, i.e. demographic variables such as gender, hierarchical position, organizational setup, working experience, organization tenure and job tenure, had an influence on Attitude Towards Suggested Behavioural Change.Model 1, loaded with demographic variables, did not significantly predict the Attitude toward Suggested Behavioural Change (R 2 = 0.043, p > 0.05).When , and were introduced, Model 2 explained the dependent variable, Attitudes towards Behavioral Change, significantly well (R 2 = 0.614, p < 0.001).Thus the hypothesis has been validated, and the generalizability of the hypothesis is affirmed.

II. DISCUSSION
The validated hypothesis contributed to new knowledge to administrators of 360-degree assessment to enhance the success of achieving the key objective of 360-degree assessment initiatives, which is to see behavioural change of the participating ratees.
First of all, this research confirms that PC and EOO impact on ratee's attitude towards behaviour change.This finding calls for 360-degree assessment administrators' attention to ensure the assessment is perceived as credible by the ratees, and to illustrate the positive outcomes of that can be derived from behaviour change.One way to achieve that is to avoid using 360-degree assessment for appraisal purposes.
Secondly, the framework suggests optimism could drive ratees to embrace a positive attitude towards behaviour change even when either PC or EOO is negative.This finding is aligned with existing literature, which has already associated positive thinking (Koyen, 2003), positive living (Lurton, 1950) and optimism (Snyder, Sympson, Michael, & Cheavens, 2001) with adopting positive attitude, and asserted the importance of positive psychology in the workplace (Froman, 2010).This finding calls for 360-degree assessment administrators and facilitators to create positive thinking and optimism in pre-feedback and post-feedback sessions surrounding the administration of any 360-degree process.An individual's ways of positive emphasis or negative emphasis is affected by the individual's traits, experiences, characteristics, etc., thus his/her positive or negative emphasis may be predetermined before the 360-degree feedback exercise.However, by drawing the ratee's attention away from negative thoughts and attracting the ratee with positive ideas, a competent 360-degree coach is able to increase the ratee's potential positive emphasis before and during the feedback process, and thus increase the chance of the ratee embracing a positive attitude and implementing a behaviour change suggested by the 360-degree assessment.
Thirdly, the identified formula and framework suggest different ratees place different weightage on PC and EOO.This finding suggests that it is possible to design an intervention to profile the ratees and identify the weightage they place on PC and EOO.Knowing the profile of ratees will equip administrators of 360 degree assessment to be better prepared with relevant emphasis on PC/ EOO or PC and EOO to connect with the ratees to facilitate a positive attitude towards behavioural change.Such an assessment tool could be a huge contribution to the field of 360-degree assessment practices and potential impact on behaviour change commitments following a 360-degree assessment.
Literature indicates that millions of dollars have been invested in 360-degree assessment by organizations worldwide (Atwater & Yammarino, 1996), even though many organizations have failed to benefit from the assessment (McCarthy & Garavan, 2006).More specifically, not only money but also other organization resources such as time, manpower, reputation and precious transformation opportunities have been wasted in failed 360-degree feedback practices as the ratees fail to perform a behavioural change.With the formula and revised theoretical framework, a ratee profiling method could be developed, to derive the ratees' 4 coefficients in a 360-degree context to facilitate a higher success rate in getting them to embrace a positive attitude toward behavioural change.Hence, the limited organization resources can be used more effectively to ensure a positive attitude and hence a desired behavioural change of the ratees, by concentrating on perceived credibility, or evaluation of outcome, or even specific components of perceived credibility.For example, when a ratee is profiled to emphasize on evaluation of outcome, the 360-degree feedback administrators can concentrate available organizational resources on creating the ratee's positive evaluation of outcome, by illustrating the benefits of suggested behavioural change better (spend more time to illustrate, or hire an approaching coach to illustrate), or offering the ratee greater benefits (reward package of vacation or bonus).When a ratee is profiled to emphasize on perceived credibility, the administrator can concentrate available organizational resources on creating his/her positive perceived credibility, by customizing an 'in-house' feedback instrument (which requires more time, money and manpower), or hiring a credible coach from a credible agency.
In addition, identifying a ratee's positive and negative emphasis is also important to ensure a positive attitude towards the behavioural change after a 360-degree feedback, especially when the ratee shows a strong emphasis on the negatives.In such case, the 360-degree feedback administrator could provide the ratee with certain positive psychology trainings to reduce the negativity and increase the positivity.
Even if a profiling method were developed to identify the ratee's weightage placed on PC and EOO so that the 360degree feedback administrators can concentrate resources on the highly weighted PC or EOO, it should be noted that the EOO or PC at least need to be maintained at positive level but not negative, because when both positive and negative components (i.e.PC and EOO) of attitude are accessible, the unpleasant experience of ambivalence may occur (van Harreveld, van der Pligt, & de Liver, 2009).To illustrate this, when a ratee is identified with high weightage on PC and goes through a feedback with positive PC and negative EOO, indeed the relatively strong driving force of positive PC can overpower the relatively weak restraining force of negative EOO and the ratee in appearance may embrace a positive attitude and implement the behavioural change, however the clash between driving and restraining forces could lead to unpleasant experiences of ambivalence.Hence administrators of 360-degree feedback should spare no efforts to facilitate the ratees to have non-negative PC and non-negative EOO to minimize the potential ambivalence.

TABLE 1 :
PARTICIPANTS' PC, EOO AND ATTITUDE TOWARD SUGGESTED BEHAVIOURAL CHANGE IN THE LAST 360-DEGREE ASSESSMENT

TABLE 2 :
PARTICIPANTS' PC BY PURPOSES OF 360-DEGREE ASSESSMENT

TABLE 3 :
HIERARCHICAL REGRESSION ANALYSIS MODEL SUMMARY

TABLE 4 :
HIERARCHICAL REGRESSION ANALYSIS ANOVA

TABLE 5 :
HIERARCHICAL REGRESSION ANALYSIS COEFFICIENTS