Detect Irregular Behaviour or Change in Time Series

[Muhammad Sadiq]

I have a time series of memory usage.  I need to detect any irregular (
random) pattern in the series.

Your prompt help will be highly appreciated.


Kind Regards
Muhammad Sadiq
SPSS Business Analyst
0092 321 462 1660

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

I think you need to be more specific.  Outlier detection?
Pattern detection?

When I start up my computer, I sometimes kill time
by watching the CPU/memory statistics in Task Manager.

I see Firefox grab a few megabytes at a time until it gets
to 360 MB, while starting and opening 4 or 5 tabs.  That
memory usage, I suppose, is an "irregular pattern" in
two senses.  It is not monotonic, so it is irregular as a linear
increase; and it only happens once in a session, so the
increase is a one-time shot in the overall computer memory
usage for the session. 

Neither of these seems mysterious enough to be interesting.
What would seem more interesting is if someone found a way
to detect a "memory leak" which would be a *regular* pattern.

--
Rich Ulrich

> Date: Sun, 7 Oct 2012 10:13:44 -0400
> From: [hidden email]
> Subject: Detect Irregular Behaviour or Change in Time Series
> To: [hidden email]
>
> I have a time series of memory usage. I need to detect any irregular (
> random) pattern in the series.
>
> Your prompt help will be highly appreciated.
...

[Rich Ulrich]

Maybe you should be asking for help from a time-series group,
instead of SPSS.  I've never done time series analyses, but I
am familiar with some discussions, and I know that those analysts
often use specialized software.

The simple, overall answer is that you fit a time-series model
to account for patterns for day of week, time of day, and slow
change over the six months.  Subtract that off, and what is left
might be defined as "irregular patterns".  Anything that is regular
should be further labelled and subtracted out.  Then you look for
explanations for the largest remaining deviations, like "holiday",
"new client", and so on.

--
Rich Ulrich

---

CC: [hidden email]
From: [hidden email]
Subject: Re: Detect Irregular Behaviour or Change in Time Series
Date: Mon, 8 Oct 2012 02:31:11 +0500
To: [hidden email]

Thanks for your reply.

Assume I have a time series for hourly memory usage of a server machine for the last six months.

I need to discover is there any irregular usage of memory?

Reason may be other resources addition or removal or maybe some unknown reason or more processes are       being executed.

I need help.

Sent from my iPhone

On Oct 8, 2012, at 12:33 AM, Rich Ulrich <[hidden email]> wrote:

I think you need to be more specific.  Outlier detection?
Pattern detection?

When I start up my computer, I sometimes kill time
by watching the CPU/memory statistics in Task Manager.

I see Firefox grab a few megabytes at a time until it gets
to 360 MB, while starting and opening 4 or 5 tabs.  That
memory usage, I suppose, is an "irregular pattern" in
two senses.  It is not monotonic, so it is irregular as a linear
increase; and it only happens once in a session, so the
increase is a one-time shot in the overall computer memory
usage for the session. 

Neither of these seems mysterious enough to be interesting.
What would seem more interesting is if someone found a way
to detect a "memory leak" which would be a *regular* pattern.

--
Rich Ulrich

> Date: Sun, 7 Oct 2012 10:13:44 -0400
> From: [hidden email]
> Subject: Detect Irregular Behaviour or Change in Time Series
> To: [hidden email]
>
> I have a time series of memory usage. I need to detect any irregular (
> random) pattern in the series.
>
> Your prompt help will be highly appreciated.
...

[Jon K Peck]

If you use the time series Expert Modeler, you can specify that it should carry out outlier detection, and you can choose up to seven different patterns to look for.  You will get a table of outliers and types, so you can use that as a starting point.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        Rich Ulrich <[hidden email]>
To:        [hidden email]
Date:        10/07/2012 05:37 PM
Subject:        Re: [SPSSX-L] Detect Irregular Behaviour or Change in Time Series
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Maybe you should be asking for help from a time-series group,
instead of SPSS.  I've never done time series analyses, but I
am familiar with some discussions, and I know that those analysts
often use specialized software.

The simple, overall answer is that you fit a time-series model
to account for patterns for day of week, time of day, and slow
change over the six months.  Subtract that off, and what is left
might be defined as "irregular patterns".  Anything that is regular
should be further labelled and subtracted out.  Then you look for
explanations for the largest remaining deviations, like "holiday",
"new client", and so on.

--
Rich Ulrich

---

CC: [hidden email]
From: [hidden email]
Subject: Re: Detect Irregular Behaviour or Change in Time Series
Date: Mon, 8 Oct 2012 02:31:11 +0500
To: [hidden email]

Thanks for your reply.

Assume I have a time series for hourly memory usage of a server machine for the last six months.

I need to discover is there any irregular usage of memory?
Reason may be other resources addition or removal or maybe some unknown reason or more processes are       being executed.

I need help.




Sent from my iPhone

On Oct 8, 2012, at 12:33 AM, Rich Ulrich <[hidden email]> wrote:

 
I think you need to be more specific.  Outlier detection?
Pattern detection?

When I start up my computer, I sometimes kill time
by watching the CPU/memory statistics in Task Manager.

I see Firefox grab a few megabytes at a time until it gets
to 360 MB, while starting and opening 4 or 5 tabs.  That
memory usage, I suppose, is an "irregular pattern" in
two senses.  It is not monotonic, so it is irregular as a linear
increase; and it only happens once in a session, so the
increase is a one-time shot in the overall computer memory
usage for the session.  

Neither of these seems mysterious enough to be interesting.
What would seem more interesting is if someone found a way
to detect a "memory leak" which would be a *regular* pattern.

--
Rich Ulrich

> Date: Sun, 7 Oct 2012 10:13:44 -0400
> From: sadiq.pmp@...
> Subject: Detect Irregular Behaviour or Change in Time Series
> To: [hidden email]
>
> I have a time series of memory usage. I need to detect any irregular (
> random) pattern in the series.
>
> Your prompt help will be highly appreciated.
...