Scale development question

[Salbod]

Happy 2013!

This is a scale development issue. I want to create a measure
of text message understanding.

There will be 40 messages with this format:

Sample Message(1):
Y rnt u responding to my text messages?!?
ARE U IGNORING ME!?

What is the intended meaning of this message?
A. ___affectionate
B. ___aggressive
C. ___sarcastic
D. ___witty/humorous
E. ___I don't know

(10 messages for each type of content.)

The aim is to reduce the number of messages to 20 (i.e., 5 per content type).

I've never seen how a scale like this gets developed.

The 40 messages will be administered to 100 students. For each message, persentages will
be generated for the four content types. The message will be sorted in four groups based
on the modal content type. The messages within each content type will be ranked and the
top five select for inclusion in the final measure.

This is the first time I've seen this format.

If anyone has an scale development article with
this type of format, I really would appreciate being sent the reference.

Many thanks,

Stephen Salbod, Pace University, NYC

[David Marso]

Even that single message suggests an expansion of your content types to include
Stalkerish?
Pathetic?
Sociopathic?
Eventual 911 call?
---
mt B nc 2c exs uv udder cnd8 txt msgs.
---
I'd pbly slm da GIGO mess n2 crspndc analz 2c f NEtn cums out wrt ntprtl dims ;-)))
MB 42tis rzltz?
FNA!
;-))))
Personally I absolutely deplore the degeneration of human communication into IM speak/twitter brain farts.
Good luck in your inquiry...
---

Salbod wrote

Happy 2013!

This is a scale development issue. I want to create a measure
of text message understanding.

There will be 40 messages with this format:

Sample Message(1):
Y rnt u responding to my text messages?!?
ARE U IGNORING ME!?

What is the intended meaning of this message?
A. ___affectionate
B. ___aggressive
C. ___sarcastic
D. ___witty/humorous
E. ___I don't know

(10 messages for each type of content.)

The aim is to reduce the number of messages to 20 (i.e., 5 per content type).

I've never seen how a scale like this gets developed.

The 40 messages will be administered to 100 students. For each message, persentages will
be generated for the four content types. The message will be sorted in four groups based
on the modal content type. The messages within each content type will be ranked and the
top five select for inclusion in the final measure.

This is the first time I've seen this format.

If anyone has an scale development article with
this type of format, I really would appreciate being sent the reference.

Many thanks,

Stephen Salbod, Pace University, NYC

[Salbod]

David, thanks for you interesting comments. You mention a few ideas (e.g., Stalkerish) that I will discuss with the PI. It is really not my project, I'm the technical support team.

I'm just stuck with how to approach this analytically. I'm sure the format has been successfully used before; how was it analyzed?    

[David Marso]

I suspect the analytics would fall out of the intentions of the construction of the measure.
I have a problem with the nature of the actual categories and the specifics of the message.
For example, no lazy IM-twatter is going to actually spell out "messages" or "responding" or "ARE"!.
I suspect the best you would actually get might be:

"Y rnt u rspnd 2 txt msgs?!?
R U NORING ME!?

Also the categories might actually overlap or be very differently interpreted depending upon the perceived relationship between parties.
In this case the category is obviously Aggressive but how often is this unambiguous?
For example I frequently attempt to toss a little "humorous" into my NG interactions.
People who don't know me or that I have serious filter-dysfunctionality and utter contempt for authority... might miscategorize it as sarcastic.

Furthermore, this same communication might be viewed by some as affectionate (tough love) and others might see it as aggressive.  

If I somehow ended up being the "technical support team" on this project I would try to stop the train wreck before it gets too far and try to nail down the specific purpose of the 'scale'.  Is it success/failure in establishing communicative intention?  Emotive intent of the messages? Parsability of the garbly-gook.
In fact in my attempt at humorous comment I actually injected the suggestion of subjecting the data to  correspondence analysis to see if you find anything useful wrt dimensionality.

Note this is completely seat of the pants musing  (I have no bkg in linguistics or communication theory etc and hence I am likely completely full of shit as far as this goes)!
--
------------------------------

Salbod wrote

David, thanks for you interesting comments. You mention a few ideas (e.g., Stalkerish) that I will discuss with the PI. It is really not my project, I'm the technical support team.

I'm just stuck with how to approach this analytically. I'm sure the format has been successfully used before; how was it analyzed?

[Salbod]

Hi David,  

I agree with most of what you're saying. But, the train has already
left the station ;) Thanks for the 'seat of the pants musing'
they take me out of the center of the problem.  
Being too focused on a solution is sometimes not helpful.  

Thanks for the push, Steve