/w=3840,quality=90,fit=scale-down)
Overview:
I decided I wanted to work on a project involving AI (since it’s all the rage these days), and learn more about neural networks. I had a simple idea of tracking a user’s engagement using their webcam, and deploying the model on a webpage. I felt this was a good combination of what I know (React and web development) as well as something new (neural networks and Tensorflow).
The Dataset:
In addition to the images I collected of myself sitting at my computer (about 400 images), I download 3000 images of human faces from https://storage.googleapis.com/openimages/web/visualizer/index.html of human faces. Which, looking back, wasn’t the best decision since most of these photos aren’t representative of the problem situation (people sitting in front of their webcam or in front of their phone). Next iteration, I will look to collect better training data more representative of the problem.
Training:
I trained the SSD model on my personal computer with a Nvidia RTX 3070. I was able to enable training with my GPU which significantly improved the speed of training. Though I had issues running out of RAM since I only have 16 GB on my machine. Next time I train a model, I plan on using Google Collab or another cloud based service.
I implemented early stopping and checkpoints to make sure I get the best weights and to prevent loosing progress if I inevitably run out of RAM.
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
early_stopping = EarlyStopping(monitor='val_total_loss', patience=10, verbose=1, restore_best_weights=True)
model_checkpoint = ModelCheckpoint(filepath='ssd_100_eyes_additional_checkpoint', save_best_only=True, save_weights_only=True, monitor='val_total_loss', verbose=1, save_format='tf')Sadly, the validation loss did not decrease continuously. I wish this were the case, since then I would feel more confident that the model works well on unseen data. I think in the future, having a better dataset would help this.
Tensorflow.js:
I learned that I needed to convert my Tensorflow Keras HDF5 model to a Tensorflow.js model. These were the short steps I took to do this:
!pip install tensorflowjs
!tensorflowjs_converter --input_format=keras --output_format=tfjs_layers_model /content/drive/MyDrive/EngagementTracker/ssd_100_eyes_additional.h5 /content/drive/MyDrive/tensorflow_js_converter/converted/ssdI needed to run this in Google Collab since my machine runs Windows. Apparently tensorflowjs_converter was designed to run on Linux so this was needed.
Take-aways:
I learned after labeling the data and training the model, that I should use more specific training data (people sitting at their computer, taking a selfie). This would allow for higher quality data the neural network could learn from. Instead I got mostly all my training data from https://storage.googleapis.com/openimages/web/visualizer/index.html, a free set of human faces that were labeled.
