It sounds like you want to do model comparison.
The difficulty is that you can run different
families of models or models that are not
strictly nested so that the usual statistical
test for model comparison is not known.
One idea I've seen is to obtain estimates of the
true error from each method and then perform a
statistical test to assess whether the errors
differ statistically. The devil is in the details
and is likely to be computationally intensive or
do-able only with some non-trivial effort in the package of your
choice. You need to employ something like k-fold crossvalidation
or bootstrapping to get estimates of the true error.
Witten and Frank "Data Mining, 2nd edition," published
by Morgan Kaufmann, has a nice general discussion
in its Chapter 5, plus some specific suggestions
on a statistical test you could perform in Section 5.5.
-----Original Message-----
From: SPSSX(r) Discussion [mailto:
[hidden email]] On Behalf Of
Euclid Associates
Sent: Saturday, January 27, 2007 3:59 AM
To:
[hidden email]
Subject: clementine assistance -beginner's question
Dear All,
I am a beginner and please help me to understand pertaining to clementine.
While using clementine (8.1 version) for neural application, i am getting
the importance of input variables. But Is there any way I can evaluate
models by changing number of hidden layers by comparing RMSE or goodness of
fit (R SQUARE)indices OR MAPE value. List dedicated to clementine is quite
silent - so I took the liberty to requesting for your valuable assistance.
Please help.
With regards
Euclid
Locked
