SelfLearningBots: AlphaZero-like AI
'SelfLearningBots' is a library to create your own AlphaZero-like AI. The library allows one to train reinforcement learning AI for any deterministic two player game. Like many other game playing AIs, the library uses a tree search algorithm to plan ahead; however, unlike most other AIs, the tree search is pruned using a machine learning model which predicts the desiriablity of moves available to the player.
Features
- Uses multiprocessing to run multiple instances of self-play
- Uses multithreading to parallelize the tree search algorithm (Monte Carlo Tree Search w/ Virtual Loss)
- The whole training process is automated, you can leave it running
- Can plug into any machine learning framework (given you implement the specified interfaces).
- Provides a default ResNet architecture (the one used by the folks at DeepMind for the original implementation).
Challenge(s)
- Exceedingly long edit-compile-debug cycle (!!)
- No official implementation to reference
- Limited resources for training (free GCP credits)
- Generation of data is slow with python (plan on using a statically-typed language)
- Current approach does not take advantage of distributed systems (but it can, see note)
Tutorial.js
The App Attention Index 2017 showed that 60% of users deleted an app after one attempt of trying to use it and 80% will lose interest if they continue to run into problems. 'Tutorial.js' is a library for onboarding new users to minimize frustration.
Features
- Library is lightweight as it has no dependencies (vanilla JS and CSS)
- Supports three different tutorial formats: Basic, Timeline, TimedSequence
- Fully documented
Challenge(s)
- Logic for tooltip positioning needed tweaking to feel natural
- Library classes needed to be designed with callbacks to allow for customization (ie. composed classes)
- There are still some bugs with tooltip DOM manupilation that need to be ironed out
- TODO: use observer pattern to make tooltip trailing more robust.
- Currently when an element moves while the tooltip is trailing it, tooltip does not update
Tumour Classifier
A webapp that predicts whether a skin tumour is benign or malignant from an image.
Features
- Trained a ResNet neural net architecture to classify benign and malignant tumours.
- Deployed the model using Python Flask and preprocessed images with PIL.
- Created a frontend with React to communicate with Flask API.
Challenge(s)
Although the model achieved a test accuracy of 83%, it performed poorly with real world data. This can be attributed to a variety of reasons:
- Poor preprocessing: most images needed to be resized to a significantly smaller size which resulted in a loss of information
- Poor model choice: in hindsight, a ResNet model doesn’t fit here; a better choice would have been to add some max pooling layers (similar to VGG)
- Since max pooling layers down sample the previous layer, the model becomes more robust to changing image orientations and loss of information due to preprocessing