Trjtdtk

Fairly newbie to Pytorch & neural nets world.Below is a code snippet from a binary classification being done using a simple 3 layer network :

n_input_dim = X_train.shape[1]
n_hidden = 100 # Number of hidden nodes
n_output = 1 # Number of output nodes = for binary classifier
# Build the network
model = nn.Sequential(
 nn.Linear(n_input_dim, n_hidden),
 nn.ELU(),
 nn.Linear(n_hidden, n_output),
 nn.Sigmoid())

x_tensor = torch.from_numpy(X_train.values).float()
tensor([[ -1.0000, -1.0000, -1.0000, ..., -99.0000, -99.0000, -99.0000],
 [ -1.0000, -1.0000, -1.0000, ..., 0.1538, 5.0000, 0.1538],
 [ -1.0000, -1.0000, -1.0000, ..., -99.0000, 6.0000, 0.2381],
 ...,
 [ -1.0000, -1.0000, -1.0000, ..., -99.0000, -99.0000, -99.0000],
 [ -1.0000, -1.0000, -1.0000, ..., -99.0000, -99.0000, -99.0000],
 [ -1.0000, -1.0000, -1.0000, ..., -99.0000, -99.0000, -99.0000]])
y_tensor = torch.from_numpy(Y_train).float()
tensor([0., 0., 1., ..., 0., 0., 0.])
#Loss Computation
loss_func = nn.BCELoss()
#Optimizer
learning_rate = 0.0001
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

train_loss = []
iters = 500
for i in range(iters):
 y_pred = model(x_tensor)
 loss = loss_func(y_pred, y_tensor)
 print " Loss in iteration :"
 print (i, loss.item())

 optimizer.zero_grad()
 loss.backward()
 optimizer.step()
 train_loss.append(loss.item())

In the above case , what i'm not sure about is loss is being computed on y_pred which is a set of probabilities ,computed from the model on the training data with y_tensor (which is binary 0/1).
Is this way of loss computation fine in Classification problem in pytorch? Shouldn't loss be computed between two probabilities set ideally ? If this is fine , then does loss function , BCELoss over here , scales the input in some manner ?

Any insights towards this will be highly appreciated

You are right about the fact that cross entropy is computed between 2 distributions, however, in the case of the y_tensor values, we know for sure which class the example should actually belong to which is the ground truth. So, you can think of the binary values as probability distributions over possible classes in which case the loss function is absolutely correct and the way to go for the problem. Hope that helps.