Trjtdtk

Has a Nobel Peace laureate ever been accused of war crimes?

What helicopter has the most rotor blades?

What's the difference between using dependency injection with a container and using a service locator?

What kind of equipment or other technology is necessary to photograph sprites (atmospheric phenomenon)

How to keep bees out of canned beverages?

Etymology of 見舞い

Does using the Inspiration rules for character defects encourage My Guy Syndrome?

Why these surprising proportionalities of integrals involving odd zeta values?

C variable type assert

Will I be more secure with my own router behind my ISP's router?

How to calculate density of unknown planet?

Will the Antimagic Field spell cause elementals not summoned by magic to dissipate?

Is it OK if I do not take the receipt in Germany?

Is there a verb for listening stealthily?

false 'Security alert' from Google - every login generates mails from 'no-reply@accounts.google.com'

What is the ongoing value of the Kanban board to the developers as opposed to management

Is Vivien of the Wilds + Wilderness Reclamation a competitive combo?

A journey... into the MIND

Is "ein Herz wie das meine" an antiquated or colloquial use of the possesive pronoun?

Why not use the yoke to control yaw, as well as pitch and roll?

What is the evidence that custom checks in Northern Ireland are going to result in violence?

Knights and Knaves question

How can I wire a 9-position switch so that each position turns on one more LED than the one before?

Who's this lady in the war room?

Very low probability in naive Bayes classifier 1

0

$begingroup$

I have some training data (TRAIN) and some test data (TEST). Each row of each table contains an observed class (X) and some columns of binary (Y). I'm using a Python script that is intended to predict the probability (Pr) of X given Y in the test data based on the training data. It uses a Bernoulli naive Bayes classifier. Here is my script:

https://stackoverflow.com/questions/55187516/look-up-bernoullinb-probability-in-dataframe

It works on the dummy data that is included with the script.

On the real data, I know from experience which class some of the Y columns are indicative of. My script however is giving probability predictions like "1" where I don't think that the class is correct and "6e-77" on correct classes.

Any advice on what I can try please?

Edit

There are two problems. The very low probability is caused by the naive assumption that nothing is related to anything else. This is described here:
https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration_curve.html#sphx-glr-auto-examples-calibration-plot-calibration-curve-py

The incorrect answers are caused by my code getting confused about which class is which, as described on my Stack Overflow post.

machine-learning prediction probability naive-bayes-classifier

share|improve this question

edited Apr 11 at 15:05

R. Cox

asked Apr 5 at 16:21

R. CoxR. Cox

12

$endgroup$

add a comment | 

0

$begingroup$

I have some training data (TRAIN) and some test data (TEST). Each row of each table contains an observed class (X) and some columns of binary (Y). I'm using a Python script that is intended to predict the probability (Pr) of X given Y in the test data based on the training data. It uses a Bernoulli naive Bayes classifier. Here is my script:

https://stackoverflow.com/questions/55187516/look-up-bernoullinb-probability-in-dataframe

It works on the dummy data that is included with the script.

On the real data, I know from experience which class some of the Y columns are indicative of. My script however is giving probability predictions like "1" where I don't think that the class is correct and "6e-77" on correct classes.

Any advice on what I can try please?

Edit

There are two problems. The very low probability is caused by the naive assumption that nothing is related to anything else. This is described here:
https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration_curve.html#sphx-glr-auto-examples-calibration-plot-calibration-curve-py

The incorrect answers are caused by my code getting confused about which class is which, as described on my Stack Overflow post.

machine-learning prediction probability naive-bayes-classifier

share|improve this question

edited Apr 11 at 15:05

R. Cox

asked Apr 5 at 16:21

R. CoxR. Cox

12

$endgroup$

add a comment | 

$begingroup$

I have some training data (TRAIN) and some test data (TEST). Each row of each table contains an observed class (X) and some columns of binary (Y). I'm using a Python script that is intended to predict the probability (Pr) of X given Y in the test data based on the training data. It uses a Bernoulli naive Bayes classifier. Here is my script:

https://stackoverflow.com/questions/55187516/look-up-bernoullinb-probability-in-dataframe

It works on the dummy data that is included with the script.

On the real data, I know from experience which class some of the Y columns are indicative of. My script however is giving probability predictions like "1" where I don't think that the class is correct and "6e-77" on correct classes.

Any advice on what I can try please?

Edit

There are two problems. The very low probability is caused by the naive assumption that nothing is related to anything else. This is described here:
https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration_curve.html#sphx-glr-auto-examples-calibration-plot-calibration-curve-py

The incorrect answers are caused by my code getting confused about which class is which, as described on my Stack Overflow post.

machine-learning prediction probability naive-bayes-classifier

share|improve this question

edited Apr 11 at 15:05

R. Cox

asked Apr 5 at 16:21

R. CoxR. Cox

12

$endgroup$

share|improve this question

edited Apr 11 at 15:05

R. Cox

asked Apr 5 at 16:21

R. CoxR. Cox

12

share|improve this question

edited Apr 11 at 15:05

R. Cox

asked Apr 5 at 16:21

R. CoxR. Cox

12

asked Apr 5 at 16:21

R. CoxR. Cox

12

asked Apr 5 at 16:21

R. CoxR. Cox

12