Trjtdtk

I'm running a binary classification logistic regression. When I run gradient descent for 100 iterations I get ~ 90% prediction accuracy (cost function is decreasing constantly but hasn't converged yet). When I run the same algorithm for 1000, 10000 and 100000 iterations I get identical ~ 85% accuracy with cost function convergence.

When I run the classification using a built in logistic regression function in R, I get exactly the same ~ 85% accuracy as I got for my 1000, 10000, 100000 iteration runs.

I'm not sure whether this should be an impossibility and my code must therefore contain an error or there is something I'm not understanding regarding gradient descent for classification.

Any help would be appreciated, thanks.

Note: The quoted model accuracy is based on fitting the coefficients to the training data.

Edit: I used the glm() function combined with predict() to check my results.
Here is the code I am using for my own implementation:

# Logistic classification -----------------------------------------

# Features and targets
predictors <- matrix(mtcars[, 'hp'])
outcome <- mtcars[, 'vs']

# Feature scaling (only one this time)
for (i in ncol(predictors)) 
 predictors[, i] <- (predictors[, i] - min(predictors[, i])) /
 (max(predictors[, i]) - min(predictors[, i]))


# Ready features and targets 
X <- cbind(rep(1, length(predictors)),
 predictors)
y <- outcome

# Parameters
Thetas <- rep(0, ncol(X))
iterations <- 10000 # varying
Alpha <- 0.1
m <- length(y)

# Track updates
cost_history <- numeric()
Theta_history <- list()

# Sigmoid activation
g <- function(x) 
 1 / (1 + exp(-x))


# Gradient descent
for (i in 1:iterations) 
 cost <- - 1 / m * (t(y) %*% log(g(X %*% Thetas)) +
 (1 - y) %*% log(1 - g(X %*% Thetas)))
 cost_history[i] <- cost
 Theta_history[[i]] <- Thetas
 Thetas <- Thetas - Alpha * (t(X) %*% (g(X %*% Thetas) - y))


# Inspect convergence
plot(cost_history)

# Return predictions
probs <- g(X %*% Thetas)
preds <- ifelse(probs >= 0.5, 1, 0)

# Do predictions match
pred_match <- numeric()

for (i in 1:length(preds)) 
 pred_match[i] <- ifelse(preds[i, ] == y[i], 1, 0)
**strong text**

# Model accuracy
mean(pred_match)