Trjtdtk

I am dealing with an imbalanced class with the following distribution :
(Total dataset size : 10763 X 20)

0 : 91%

1 : 9%

To build model on this dataset having class imbalance, I have compared results using

1) SMOTE and

2) Assigning more weight to the minority class when applying fit

and the latter seems to be working better.

After experimenting with Decision Tree, LR, RF SVM(poly and rbf) I am now using XGBoost classifier which gives me the below classification results(these are the best numbers I've got so far)

The business problem I'm trying to solve requires the model to have high precision, as the cost associated with that is high.

Here's my XGBClassifer's code :

xgb3 = XGBClassifier(
 learning_rate =0.01,
 n_estimators=2000,
 max_depth=15,
 min_child_weight=6,
 gamma=0.4,
 subsample=0.8,
 colsample_bytree=0.8,
 reg_alpha=0.005,
 objective= 'binary:logistic',
 nthread=4,
 scale_pos_weight=10,
 eval_metrics = 'logloss',
 seed=27)
model_fit(xgb3,X_train,y_train,X_test,y_test)

And here's the code for model_fit :

def model_fit(algorithm,X_train,y_train,X_test,y_test,cv_folds = 5 ,useTrainCV = "True", early_stopping_rounds = 50):
if useTrainCV:
 xgb_param = algorithm.get_xgb_params()
 xgbtrain = xgb.DMatrix(X_train,label = y_train)
 xgbtest = xgb.DMatrix(X_test)
 cvresult = xgb.cv(xgb_param,xgbtrain,num_boost_round = algorithm.get_params()['n_estimators'],
 nfold = cv_folds,metrics='logloss', early_stopping_rounds=early_stopping_rounds)
 algorithm.set_params(n_estimators=cvresult.shape[0])

algorithm.fit(X_train, y_train, eval_metric='logloss') 
y_pred = algorithm.predict(X_test)
cm = confusion_matrix(y_test,y_pred)
print(cm)
print(classification_report(y_test,y_pred))

Can anyone tell me how can I increase the precision of the model. I've tried everything I know of. I'll really appreciate any help here.