categorical classification

Machine Learning on a 'Real Problem' - Part 3: Final Model

Machine Learning on a 'Real Problem' - Part 3: Final Model

In our last post we did a total overhaul of our model, using a more appropriate neural network type and a more powerful framework. We simplified the problem by doing a binary classification and only using two classes: our normal and our ceiling effects plots. We were able to get fantastic validation accuracy, but never checked accuracy on a test set, and never considered alternate metrics of evaluating model performance ("accuracy" is not always the most informative metric).

In this post, well create our final model that predicts all four classes, we'll evaluate its accuracy on a set of data held out from any training or validation, and look at a metric other than accuracy to give us more information about our model performance.