need some basis on research vs assessment; surveys vs existing data analysis

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

need some basis on research vs assessment; surveys vs existing data analysis

William Peck
I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: need some basis on research vs assessment; surveys vs existing data analysis

Jeff A
To be somewhat blunt, I'm unsure what it is you're asking or getting at other than some data is measured without error (or is very objective) and some with error (or is very subjective) and how to address and think about such matters. I'm unsure what the Belmont Report has to do with these matters other than being able to tie this into the above by saying that due to ethical concerns, someone doing research might have to settle for research methods that contain much more error or is much less rigorous than a hypothetical ideal since a measurement technique that provides more definitive conclusions or less error might be unethical (e.g., you obviously can't randomly assign children to different parents to examine the effect of parenting style on juvenile delinquency). These matters, however, sound like they are more to do with the design of research and data collection than about statistics and the software that's used to perform statistical analysis. If I'm correct, then you want to look at sources about "research methods" rather than sources about "statistics" or "statistical software".

Best,

Jeff








-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck
Sent: Friday, 31 May 2019 9:24 PM
To: [hidden email]
Subject: need some basis on research vs assessment; surveys vs existing data analysis

I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: need some basis on research vs assessment; surveys vs existing data analysis

William Peck
Jeff,

thanks for the feedback. 

>>you want to look at sources about "research methods" rather than sources about "statistics" or "statistical software". 
- yes, this best answers my question, which is more about research methods than statistical analysis/software

Having been in the statistician role for 9 months (with a data background), I'm trying to put together a powerpoint presentation for the non-statistician bosses, that captures the components of what's in my world.

So far, it's

Surveys 
      - Research (e.g., leadership, and requires adherence to the principles laid out in the Belmont report)
      - Assessment (survey of the food service offerings)

Data Analysis of the student grades database, to determine
      - what are the success factors for students in XYZ major
      - comparison of college students' performance for high school grads vs. those who went to 1-year prep school)
   
and looking to see if I have it about right, at a very high level  

On Sun, Jun 2, 2019 at 6:21 AM Jeff <[hidden email]> wrote:

To be somewhat blunt, I'm unsure what it is you're asking or getting at other than some data is measured without error (or is very objective) and some with error (or is very subjective) and how to address and think about such matters. I'm unsure what the Belmont Report has to do with these matters other than being able to tie this into the above by saying that due to ethical concerns, someone doing research might have to settle for research methods that contain much more error or is much less rigorous than a hypothetical ideal since a measurement technique that provides more definitive conclusions or less error might be unethical (e.g., you obviously can't randomly assign children to different parents to examine the effect of parenting style on juvenile delinquency). These matters, however, sound like they are more to do with the design of research and data collection than about statistics and the software that's used to perform statistical analysis. If I'm correct, then you want to look at sources about "research methods" rather than sources about "statistics" or "statistical software".

Best,

Jeff








-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck
Sent: Friday, 31 May 2019 9:24 PM
To: [hidden email]
Subject: need some basis on research vs assessment; surveys vs existing data analysis

I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--

Bill Peck

Information Technology Services Division

Institutional Research / Business Intelligence

United States Naval Academy

410-293-1475 (w)

410-207-3350 (c)


**** Data is your friend ===> at home, at work, and on the battlefield ****


**** Data is the fuel of your organization ****

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: need some basis on research vs assessment; surveys vs existing data analysis

William Peck
To close the loop on this - - - 

- I ordered "Statistics As Principled Argument by Robert Abelson, looks really good

- I did some quick research on "research methods", Since I'm not a researcher, and 90% of my academic institution clients are not either, it's not directly relevant. Most of what we do are your basic surveys without any statistical analysis beyond the very basics.

But still, this thread has been helpful, I look forward to the Abelson book.

Thanks Gary Oliver and Jeff!



On Sun, Jun 2, 2019 8:36 PM, Bill Peck [hidden email] wrote:
Jeff,

thanks for the feedback. 

>>you want to look at sources about "research methods" rather than sources about "statistics" or "statistical software". 
- yes, this best answers my question, which is more about research methods than statistical analysis/software

Having been in the statistician role for 9 months (with a data background), I'm trying to put together a powerpoint presentation for the non-statistician bosses, that captures the components of what's in my world.

So far, it's

Surveys 
      - Research (e.g., leadership, and requires adherence to the principles laid out in the Belmont report)
      - Assessment (survey of the food service offerings)

Data Analysis of the student grades database, to determine
      - what are the success factors for students in XYZ major
      - comparison of college students' performance for high school grads vs. those who went to 1-year prep school)
   
and looking to see if I have it about right, at a very high level  

On Sun, Jun 2, 2019 at 6:21 AM Jeff <[hidden email]> wrote:

To be somewhat blunt, I'm unsure what it is you're asking or getting at other than some data is measured without error (or is very objective) and some with error (or is very subjective) and how to address and think about such matters. I'm unsure what the Belmont Report has to do with these matters other than being able to tie this into the above by saying that due to ethical concerns, someone doing research might have to settle for research methods that contain much more error or is much less rigorous than a hypothetical ideal since a measurement technique that provides more definitive conclusions or less error might be unethical (e.g., you obviously can't randomly assign children to different parents to examine the effect of parenting style on juvenile delinquency). These matters, however, sound like they are more to do with the design of research and data collection than about statistics and the software that's used to perform statistical analysis. If I'm correct, then you want to look at sources about "research methods" rather than sources about "statistics" or "statistical software".

Best,

Jeff








-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck
Sent: Friday, 31 May 2019 9:24 PM
To: [hidden email]
Subject: need some basis on research vs assessment; surveys vs existing data analysis

I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--

Bill Peck

Information Technology Services Division

Institutional Research / Business Intelligence

United States Naval Academy

410-293-1475 (w)

410-207-3350 (c)


**** Data is your friend ===> at home, at work, and on the battlefield ****


**** Data is the fuel of your organization ****



Bill Peck

Information Technology Services Division

Institutional Research / Business Intelligence

United States Naval Academy

410-293-1475 (w)

410-207-3350 (c)


**** Data is your friend ===> at home, at work, and on the battlefield ****


**** Data is the fuel of your organization ****

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: need some basis on research vs assessment; surveys vs existing data analysis

Rich Ulrich
In reply to this post by William Peck
Abelson's is a fine book.

Apart from new surveys, you  described having a large database
from the past, and that eventually reminded me of the subject of
"data-mining" which is a term that IBM adapted for some software.
 
Twenty years ago, we (experienced data analysts) spent some time
dumping scorn on "data-dredging" -- the same thing, when done
with no caution.  Most /books/  on data-mining probably give proper
attention to the problems of too-many-variables (and too many tests);
the hazards of step-wise procedures; and the methods of splitting
samples to provide for cross-validation.

--
Rich Ulrich


From: SPSSX(r) Discussion <[hidden email]> on behalf of William Peck <[hidden email]>
Sent: Friday, May 31, 2019 7:24 AM
To: [hidden email]
Subject: need some basis on research vs assessment; surveys vs existing data analysis
 
I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: need some basis on research vs assessment; surveys vs existing data analysis

William Peck
Rich,

The (not so) large database is simply our student grades database at our college, that has 20+ years of data.

So data mining is the category I'm looking for - it accurately describes what is happening, separate from research / surveys. That's going to be on my PPT slide. It's not a new term to me (at all) but helps to dissect what is going on here, in understandable terms.

Thanks for following up!



On Mon, Jun 3, 2019 4:43 PM, Rich Ulrich [hidden email] wrote:
Abelson's is a fine book.

Apart from new surveys, you  described having a large database
from the past, and that eventually reminded me of the subject of
"data-mining" which is a term that IBM adapted for some software.
 
Twenty years ago, we (experienced data analysts) spent some time
dumping scorn on "data-dredging" -- the same thing, when done
with no caution.  Most /books/  on data-mining probably give proper
attention to the problems of too-many-variables (and too many tests);
the hazards of step-wise procedures; and the methods of splitting
samples to provide for cross-validation.

--
Rich Ulrich


From: SPSSX(r) Discussion <[hidden email]> on behalf of William Peck <[hidden email]>
Sent: Friday, May 31, 2019 7:24 AM
To: [hidden email]
Subject: need some basis on research vs assessment; surveys vs existing data analysis
 
I need some high level understanding of the entire statistics industry. I've been doing SPSS for 9 months but now taking a step back.

I deal with a lot of surveys at an institution of higher learning. I understand within surveys there is research and assessment. Generally, Research is to gain new knowledge while Assessment is more program management, accountability, decision-making, and budgeting. And the Research is guided (in the US anyway) by "Ethical Principles and Guidelines for the Protection of Human Subjects of Research" which was promulgated in 1979. But in all of this the data is provided by someone taking an online survey (or perhaps a researcher carries a clipboard around and "surveys" patients). We are careful to ensure participation is anonymous.

Another thing I do is assessment of academic performance, where the data is coming from the database. So we don't need to pull in student id's, we just pull grades, academic year/semester, college placement scores, etc. And we can look at things like comparison groups (prep school kids vs high school kids [who came to our college]), success factors for those in XYZ major, etc. etc. So we just pull the data as opposed to taking a survey. Is this activity some other "categorization" of research/assessment?

Anyway, just looking for some broad overview, as opposed to "how the heck do I do this in SPSS?" (which assumes you kind of know your end goal, you know what you are doing [but don't know how to get it in SPSS]). Stuff that's already been written, so if you can refer me to a book, some good site, or even, ahem, a Wikipedia page, that would be great.

Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


Bill Peck

Information Technology Services Division

Institutional Research / Business Intelligence

United States Naval Academy

410-293-1475 (w)

410-207-3350 (c)


**** Data is your friend ===> at home, at work, and on the battlefield ****


**** Data is the fuel of your organization ****

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD