InterGroup Agreement?
Locked
 
|
I am trying to determine which agreement measures to use with a data set that I've been given.
Forty managers completed this leadership assessment tool (self rating). Next, six scores are derived - five sub-scores and an overall leadership score (average of the five subscores). The rating scale for each item is ordinal. Each manager asks two types of raters to assess their level of leadership using the same leadership assessment tool. The same scores are computed for both observer types. The first group is the manager's supervisor, in which n=1 (always). The second group are the manager's subordinates, and the number can vary, anywhere from n=5 to an n=20. Therefore, there are three types of ratings for each manager: self, supervisor (n=1), subordinate (number varies). This is a 360 degree assessment approach to determining the degree to which the manager exhibits aspects of transformational leadership. Basically, of the 40 managers--each with the three types of ratings, the person that I am working with wants to select three managers for which there is the highest level of agreement across the self (so one rating), supervisor (n=1), and the subordinates (number of raters varies per manager). I've looked through the archives....what is the best way / index for identifying those with the highest levels of transformational leadership? I've looked at a variety of indices, but it seems my issue is that I have only the one self rating and the one supervisor rating and the number of subordinate raters varies. Any advice would be greatly appreciated. Christina |
|
Christina
You need to let us see the instrument used. Without seeing your data or your *.sav file: Looks like you need to derive a mean from a varying number of subordinates, but you will encounter a problem of missing values which could returen a null value for whichever statistic you are trying to compute. You don't say how many points are on the rating scales: I assume 5. Assuming self-assessmen is self1 and supervisor super1 and that subordinate assessments are named sub1 to sub20 you need something like this (untested): Count subsum = sub1 to sub20 (1 thru 5). Formats subsum (f2.0). Compute submean = sum.5 (sub1 to sub20)/subsum. List self1 super1 submean /cases 40. Then eyeball the list. John F Hall (Mr) [Retired academic survey researcher] Email: [hidden email] Website: www.surveyresearch.weebly.com SPSS start page: www.surveyresearch.weebly.com/1-survey-analysis-workshop -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Christina Gregory Sent: 05 May 2015 08:06 To: [hidden email] Subject: InterGroup Agreement? I am trying to determine which agreement measures to use with a data set that I've been given. Forty managers completed this leadership assessment tool (self rating). Next, six scores are derived - five sub-scores and an overall leadership score (average of the five subscores). The rating scale for each item is ordinal. Each manager asks two types of raters to assess their level of leadership using the same leadership assessment tool. The same scores are computed for both observer types. The first group is the manager's supervisor, in which n=1 (always). The second group are the manager's subordinates, and the number can vary, anywhere from n=5 to an n=20. Therefore, there are three types of ratings for each manager: self, supervisor (n=1), subordinate (number varies). This is a 360 degree assessment approach to determining the degree to which the manager exhibits aspects of transformational leadership. Basically, of the 40 managers--each with the three types of ratings, the person that I am working with wants to select three managers for which there is the highest level of agreement across the self (so one rating), supervisor (n=1), and the subordinates (number of raters varies per manager). I've looked through the archives....what is the best way / index for identifying those with the highest levels of transformational leadership? I've looked at a variety of indices, but it seems my issue is that I have only the one self rating and the one supervisor rating and the number of subordinate raters varies. Any advice would be greatly appreciated. Christina -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/InterGroup-Agreement-tp5729488 .html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
Given that as John pointed out we don't have more specific information:
I would start by trying to construct a 3D scatterplot. X is the managers own score. Y is the supervisor's rating of the manager Z is the subordinates' summary or individual rating of the supervisor. Z could be a max rating by subordinates for one look, a min for another, s mean for another, and a SD for another. Another view might be to use a scatterplot with each raw case as a data point, i.e., with no form of overall score across subordinates. Look at the scatterplot in the output window, rotate and look at it from different perspectives. You should consider providing one or more rotatable scatterplots to the persons using this data to make decisions. Some grains of salt for interpreting such data. (a) there is an anomaly in self rating whereby individual with higher levels of intelligence, grammar skills, humor, and other qualities sometimes tend to rate themselves lower than the more objective measures would indicate. Where as individuals with middle to low levels of the above sometimes tend to rate themselves higher than the more objective measures would show. (b) raters often confuse liking with skill or competence, e.g., play on the same company sport team. (c) raters often downgrade people who are seen as a threat in some way, e.g., more experience, education, etc. (d) etc.
Art Kendall
Social Research Consultants |
|
|
Thank you for your responses!
A four point scale is used: 0 = Not at all; 1=Once in a While; 2=Sometimes; 3=Fairly Often; 4=Frequently, if not always The data look like this (I have not yet imported into SPSS until I have a plan) - of course, the data varies more than what you see below: Subscore I Subscore II Subscore III Subscore IV Subscore V Average Subscores 1-5 Manager I 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate I 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 2 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 3 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 4 Subordinate 5 Supervisor Manager II Subordinate I Subordinate I Subordinate 2 Subordinate 3 Subordinate 4 Subordinate 5 Subordinate 6 Subordinate 7 Supervisor Manager III etc. This data is being used in a mixed methods research project (explanatory sequential) - the person is using the data above to select the managers for participation in the qualitative phase (i.e., to participate in a series of interviews). If more information is needed, I will supply. Christine |
|
|
I'd like to suggest that for each manager-supervisor-subordinates group a measure of agreement is either (or both) the mean of the variances for the three rater pairs or the variance of the three rater pair variances. Across your set of 40 groups the highest levels of agreement are indicated by smallest rater pair variances.
Gene Maguin -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Christina Gregory Sent: Tuesday, May 05, 2015 10:15 AM To: [hidden email] Subject: Re: InterGroup Agreement? Thank you for your responses! A four point scale is used: 0 = Not at all; 1=Once in a While; 2=Sometimes; 3=Fairly Often; 4=Frequently, if not always The data look like this (I have not yet imported into SPSS until I have a plan) - of course, the data varies more than what you see below: Subscore I Subscore II Subscore III Subscore IV Subscore V Average Subscores 1-5 Manager I 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate I 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 2 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 3 3.25 2.5 3.1 2.25 1.5 2.52 Subordinate 4 Subordinate 5 Supervisor Manager II Subordinate I Subordinate I Subordinate 2 Subordinate 3 Subordinate 4 Subordinate 5 Subordinate 6 Subordinate 7 Supervisor Manager III etc. This data is being used in a mixed methods research project (explanatory sequential) - the person is using the data above to select the managers for participation in the qualitative phase (i.e., to participate in a series of interviews). If more information is needed, I will supply. Christine -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/InterGroup-Agreement-tp5729488p5729491.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by Christina Gregory
It seems to me that potentially there is a conflict -- and therefore a choice to be
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
made -- between "agreement" and "good character". Or, would you want to conclude that someone is "best" when he is well-agreed-up (including, by his own ratings) to be a lying, crazy incompetent? I gave consultation once to a psychologist who advised some companies about new hires. For the basic level jobs, one of the most damning test profiles was someone who considered himself always better than his peers. Of course, that is what you really want to see from the ratings of others, not "self". "The rating scale for each item is ordinal". Is that a perfunctory description of verbal anchors that are intended as equal-interval, and should be analyzed as such? Or is it a warning about really lousy scale development done before now? - Looks to me like you should be looking for "good" characteristics, as rated by others. But I know nothing about "transformational leadership." -- Rich Ulrich > Date: Mon, 4 May 2015 23:05:43 -0700 > From: [hidden email] > Subject: InterGroup Agreement? > To: [hidden email] > > I am trying to determine which agreement measures to use with a data set that > I've been given. > > Forty managers completed this leadership assessment tool (self rating). > Next, six scores are derived - five sub-scores and an overall leadership > score (average of the five subscores). The rating scale for each item is > ordinal. > > Each manager asks two types of raters to assess their level of leadership > using the same leadership assessment tool. The same scores are computed for > both observer types. > > The first group is the manager's supervisor, in which n=1 (always). The > second group are the manager's subordinates, and the number can vary, > anywhere from n=5 to an n=20. > > Therefore, there are three types of ratings for each manager: self, > supervisor (n=1), subordinate (number varies). This is a 360 degree > assessment approach to determining the degree to which the manager exhibits > aspects of transformational leadership. > > Basically, of the 40 managers--each with the three types of ratings, the > person that I am working with wants to select three managers for which there > is the highest level of agreement across the self (so one rating), > supervisor (n=1), and the subordinates (number of raters varies per > manager). > > I've looked through the archives....what is the best way / index for > identifying those with the highest levels of transformational leadership? > > I've looked at a variety of indices, but it seems my issue is that I have > only the one self rating and the one supervisor rating and the number of > subordinate raters varies. > > Any advice would be greatly appreciated. > > Christina > > > > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/InterGroup-Agreement-tp5729488.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD |
|
|
To those who responded to my post, thank you.
I found something in this facilitator's manual that would applies to the research conducted by my colleague: https://www.envisialearning.com/resources_files/TLV/TLV360-Facilitators-Manual-2.pdfhttp://, especially in Appendix C. It is an instrument that is similar to the one used by my colleague (the MLQ).
Best Regards,
Christine
|
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |