Trjtdtk

I would like to use a YOLOv3 network,load in the preexisting trained weights, then retrain the ending layer to recognize say, 20 labels.

Would adding labels to a single network in training reduce its accurary?

Would it be better to train multiple models on 5,10 and 5 labels respectively.

e.g.

Model A : recognizes 5 labels (ferrari,honda,ford,toyota,tesla)
Model B : regonzies 10 labels (store front / stopsign, speed sign, super charger station , etccc. )
Model C : recongizes 5 labels (pedestrian, dog, marathon runner, bicycler, scooter rider)

OR

Model D: Recognizes 20 labels (all labels above put together)

Would Model D be as accurate as Model A,B,C put together?

It depends specially on data size and model capacity.

Therefore, we cannot give a definite "yes" or "no" prior to any experiment. However, we can enumerate possible cases, causes, and ways to improve, regardless of any specific model or dataset.

1. If the number of data points per category is insufficient, which is frequently the case in reality, a single model for combined categories performs better, since the effect of data shortage will be attenuated by sharing a model among different but related tasks, i.e. shared knowledge. However, we should not forget about the capacity of model, if model capacity is not enough for the combined task (which depends on the architecture and size of the model), this sharing would backfire as the model spreads its limited capacity across multiple tasks and becomes "jack of all trades, and master of none". Therefore, this approach is favorable if model capacity permits, which is less of a concern for neural networks with flexible size constraints.




2. If the number of data points per category is sufficient, which is rarely the case, or model capacity is limited (for example, when model performance stops improving when trained on more than 10% of data), single model per category is better since no capacity will be wasted on different (although related) tasks. From a different point of view, although this choice might be less favorable in terms of model accuracy, but it allows parallel development on smaller tasks, i.e. faster production, which could be a justified benefit for the cost we pay in terms of model accuracy.

Both approaches

We can also combine the two approaches. First we train a model on combined categories, second we feed a part (or parts) of the trained model (such as a layer, the final prediction, etc.) to a specific model that will be trained on one category. This is exactly what a pre-trained model does for us, it is trained on a more general task with more data (possibly even a different but related task), and then we use it for a specific task.

In your case, you can go for three levels:

1. The pre-trained YOLO which is already built,


2. Training MyYOLO on all categories (for example, by using one input from the last layer of YOLO, and one input directly from image), and


3. Training MyYOLO2 on specific categories (for example, by using one input from MyYOLO and one input directly from image). Noting that at each level, weights of previous model remain constant to serve a constant input.