Repository navigation
API Description
There will be multiple methods for doing transfer learning using the learned entity embedding. All these methods currently involve the target model to be a fastai learner object.
-
From a trained source learner object.
-
From saved embeddings of a trained model.
In addition to methods involving learner objects, we can directly take the required information as parameters or something, and make it a bit general where people using pytorch models with required architecture can also use it.
The restriction on the model front may be that they give the index/reference/name of the layer group which contains the embedding layers. The library can then automatically take all the layers/some specified indices of layers corresponding to the categorical variables to transfer can do its thing.
Also we can make our own format for saving the embeddings from models which may have various methods. And then loading the embeddings from that format itself. This will decouple the two steps, saving us from writing multiple methods for loading to a learner/model.
For the weights of the new entity embedding, we can:
- Ignore the increase in columns (dimension) due to potential increase in cardinality and just transfer the weights and fill the new rows (due to increased cardinality) with weight mean.
- Fill the columns with 0.
- Fill it with mean.
We might have to experiment and see the effect of having additional columns and how it affects the previous embeddings as there is another dimension.