Unsupervised Anomaly Detection Algorithm Selection [closed]2019 Community Moderator ElectionHow to compare two unsupervised anomaly detection algorithms on the same data-set?Open source Anomaly Detection in PythonHow does Elastic's Prelert (formerly Splunk Anomaly Detective App) work?What are the most suitable machine learning algorithms according to type of data?Unsupervised feature reduction for anomaly detection with autoencodersAutoencoder behavior with All White/Black MNISTUnsupervised Anomaly Detection in ImagesML Algorithm for anomaly detection in paired time-seriesAnomaly detection on time seriesUnsupervised Anomaly Detection on system metrics like memory, cpu, io, net, etcUnsupervised learning for anomaly detection
Mathematical cryptic clues
Smoothness of finite-dimensional functional calculus
Languages that we cannot (dis)prove to be Context-Free
Why did the Germans forbid the possession of pet pigeons in Rostov-on-Don in 1941?
How can I make my BBEG immortal short of making them a Lich or Vampire?
How can bays and straits be determined in a procedurally generated map?
What is the offset in a seaplane's hull?
TGV timetables / schedules?
Finding angle with pure Geometry.
Dragon forelimb placement
Can I make popcorn with any corn?
Why don't electron-positron collisions release infinite energy?
Arthur Somervell: 1000 Exercises - Meaning of this notation
Watching something be written to a file live with tail
Is it legal for company to use my work email to pretend I still work there?
How does strength of boric acid solution increase in presence of salicylic acid?
"to be prejudice towards/against someone" vs "to be prejudiced against/towards someone"
Can divisibility rules for digits be generalized to sum of digits
Why was the small council so happy for Tyrion to become the Master of Coin?
Why not use SQL instead of GraphQL?
How to write a macro that is braces sensitive?
Is it important to consider tone, melody, and musical form while writing a song?
Have astronauts in space suits ever taken selfies? If so, how?
What defenses are there against being summoned by the Gate spell?
Unsupervised Anomaly Detection Algorithm Selection [closed]
2019 Community Moderator ElectionHow to compare two unsupervised anomaly detection algorithms on the same data-set?Open source Anomaly Detection in PythonHow does Elastic's Prelert (formerly Splunk Anomaly Detective App) work?What are the most suitable machine learning algorithms according to type of data?Unsupervised feature reduction for anomaly detection with autoencodersAutoencoder behavior with All White/Black MNISTUnsupervised Anomaly Detection in ImagesML Algorithm for anomaly detection in paired time-seriesAnomaly detection on time seriesUnsupervised Anomaly Detection on system metrics like memory, cpu, io, net, etcUnsupervised learning for anomaly detection
$begingroup$
Good day,
I am in the process of building an Anomaly Detection tool. My current methodology is that I run a number of algorithms from PYOD in order to identify anomalies. However the only way I can verify which algorithm is actually doing the best is to test my implementation against a data set that already has anomaly labels to which I can compare afterwards.
My question: is there away to intuitively know which algorithm will perform better before actually training a model with the algorithm?
For example, I would like to have the following logic before running my engine:
If all features are normally distributed, then HBOS algorithm would do the best.
If more than 1000 samples, then Auto-encoder would do the best.
The reason for this question is that my 'use all algorithms and see which one performs best' approach is computationally heavy.
PYOD: https://github.com/yzhao062/pyod
Thanks in advance.
algorithms unsupervised-learning anomaly-detection
asked Mar 27 at 10:03
wichwich
11
$endgroup$
closed as primarily opinion-based by Esmailian, Mark.F, Siong Thye Goh, Dawny33♦ Mar 28 at 16:05
Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
Good day,
I am in the process of building an Anomaly Detection tool. My current methodology is that I run a number of algorithms from PYOD in order to identify anomalies. However the only way I can verify which algorithm is actually doing the best is to test my implementation against a data set that already has anomaly labels to which I can compare afterwards.
My question: is there away to intuitively know which algorithm will perform better before actually training a model with the algorithm?
For example, I would like to have the following logic before running my engine:
If all features are normally distributed, then HBOS algorithm would do the best.
If more than 1000 samples, then Auto-encoder would do the best.
The reason for this question is that my 'use all algorithms and see which one performs best' approach is computationally heavy.
PYOD: https://github.com/yzhao062/pyod
Thanks in advance.
algorithms unsupervised-learning anomaly-detection
asked Mar 27 at 10:03
wichwich
11
$endgroup$
closed as primarily opinion-based by Esmailian, Mark.F, Siong Thye Goh, Dawny33♦ Mar 28 at 16:05
Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
1
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38
add a comment |
$begingroup$
Good day,
I am in the process of building an Anomaly Detection tool. My current methodology is that I run a number of algorithms from PYOD in order to identify anomalies. However the only way I can verify which algorithm is actually doing the best is to test my implementation against a data set that already has anomaly labels to which I can compare afterwards.
My question: is there away to intuitively know which algorithm will perform better before actually training a model with the algorithm?
For example, I would like to have the following logic before running my engine:
If all features are normally distributed, then HBOS algorithm would do the best.
If more than 1000 samples, then Auto-encoder would do the best.
The reason for this question is that my 'use all algorithms and see which one performs best' approach is computationally heavy.
PYOD: https://github.com/yzhao062/pyod
Thanks in advance.
algorithms unsupervised-learning anomaly-detection
asked Mar 27 at 10:03
wichwich
11
$endgroup$
Good day,
I am in the process of building an Anomaly Detection tool. My current methodology is that I run a number of algorithms from PYOD in order to identify anomalies. However the only way I can verify which algorithm is actually doing the best is to test my implementation against a data set that already has anomaly labels to which I can compare afterwards.
My question: is there away to intuitively know which algorithm will perform better before actually training a model with the algorithm?
For example, I would like to have the following logic before running my engine:
If all features are normally distributed, then HBOS algorithm would do the best.
If more than 1000 samples, then Auto-encoder would do the best.
The reason for this question is that my 'use all algorithms and see which one performs best' approach is computationally heavy.
PYOD: https://github.com/yzhao062/pyod
Thanks in advance.
algorithms unsupervised-learning anomaly-detection
algorithms unsupervised-learning anomaly-detection
asked Mar 27 at 10:03
wichwich
11
asked Mar 27 at 10:03
wichwich
11
edited Mar 27 at 10:50
wich
asked Mar 27 at 10:03
wichwich
11
asked Mar 27 at 10:03
wichwich
11
asked Mar 27 at 10:03
wichwich
11
11
closed as primarily opinion-based by Esmailian, Mark.F, Siong Thye Goh, Dawny33♦ Mar 28 at 16:05
Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as primarily opinion-based by Esmailian, Mark.F, Siong Thye Goh, Dawny33♦ Mar 28 at 16:05
Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
1
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38
add a comment |
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
1
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
1
1
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38
add a comment |
0
active
oldest
votes
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Check out this post: datascience.stackexchange.com/q/47658/67328
$endgroup$
– Esmailian
Mar 27 at 10:43
$begingroup$
Thank you, this post refers to comparing algorithms based on their output, however I am looking for a way to shortlist algorithms before actually training it.
$endgroup$
– wich
Mar 27 at 10:48
1
$begingroup$
I have found this excellent paper: vs.inf.ethz.ch/edu/HS2011/CPS/papers/…
$endgroup$
– wich
Mar 27 at 11:38