Projects

These are some of my public projects that I worked on in my learning journey.

2023

Kaggle: Patent Phrase to Phrase Matching

Objective: To determine the semantic similarity between phrases in patents.

Notes: Practicing and learning NLP techniques used in Kaggle competitions. This one was a little bit tougher than usual, but fun nonetheless. I had to research and learn a lot on my own since most solutions posted on the Kaggle discussion contain mininal explanation and you have to do the digging in yourself to understand the solution and why it works. Notable things I learned was being creative with the Dataset (combining information) from outside sources and Adversarial Weight Perturbation which a lot of winning solutions on different NLP competitions seem to be using. AWP does seem to improve the results but at the expense of overhead computation. Maybe in future I can try come up with creative ways to implement it without incurring the overhead cost, so that I can use it more widely. Research has already started on how to do so.

I also picked up a new PyTorch training library that I am starting to like. Mosaic ML’s Composer. Looks like I will be sticking with it, at least for now.

Snake Species Identification (Ongoing)

Objective: To determine the species a snake belongs to.

Levers: Use Image classification to identify the genus and species (binomial) and then use other factors like country/continent where a snake is most likely to be found.

Data: AI Crowd Species 2021 Identification Project. Can be found here

Demo: Snake Species Identification

Notes: This was my first large project where I was working with a lot of data and had to predict a significant amount of categories (386,006 photographs of 772 snake species). I learned how to scale down the data to a manageable size in the protoyping phase so as to increase the speed of trying out new ideas and then gradually implement the ideas that worked on bigger and bigger scales of the data. I also had first hand experience with techniques like MixUp, Progressize Resizing, and Self-Supervised learning when working on this project. Read the full blog post on my experience for more details.

2021

Book Recommendation System

Objective: To recommend books to a user based on the books they have read.

Levers: Use Collaborative Filtering to recommend books to a user based on the books they have read. Link the recommended books to Goodreads to get more information about the books.

Data: Goodreads Books 10k Dataset. Can be found here

Demo: Similar Books

Notes: This was my first project where I used a dataset that was not in the form of images. I learned how to use collaborative filtering as a recommender engine. I also learned about what embeddings are and how they are used in the deep learning universe (spoiler alert: they are numbers! Meaningful numbers). Read the full blog post on my experience for more details.

American Sign Language Project

Objective: To build an app that can translate American Sign Language alphabet to English alphabet in realtime.

Levers: Use opencv to process the video feed from the webcam and then use a CNN to classify the sign language alphabet.

Data: American Sign Language Dataset. Can be found here

Demo: Demo on YouTube

Notes: In this particular project, my main aim was the inference part of the model, i.e. how to make the model useful in the real world. That’s why I went the webcam video way, because this way, the sign language alphabet can be translated in real time. Training the model was routine image classifcation training. I also learned how to use opencv to process video feeds from the webcam. And how to process the video frame by frame. I had to come up with a unique way to make the predictions consistent, since they could change from frame to frame, leading to flickering predictions. Using the queue data structure was the solution to this problem. Read the full blog post on my experience for more details.