Great article. I do have a couple of questions as this is a really interesting topic and you might know some of the answers:

1) Running a behemoth prediction model such as BERT to every touch would be prohibitive, but something really simple as a vanilla decision tree might not yield much practical precision. Where do current technology stands in that matter? Are neural networks being run or more practical models such as XGBoost?

2) So far, the user-based element is the keystroke heatmap. How are the prediction models adapting to the word choices of people and their unique writing style? For instance, the bigrams mentioned are a prior of the English language, but it might vary slightly from person to person. Likewise, is the prediction model being fine tuned over time for the user?

3) If we take commercial solutions such as SwiftKey, how far ahead is it from the presented model? I assume much is the same, but many techniques change in flavor or a developed further.

4) While all presented approaches make a lot of sense, how can one evaluate their relative improvement over the baseline pixel-perfect keyboard? E.g., using the heatmap improves the UX by 20%, adding bigram priors adds another 30%, etc.

5) Last question, how all of this evolves to multilanguage targets? I, for one, live in Brazil and am constantly typing in Portuguese and English on the same device. I suppose you can merely merge datasets for a PtBr-EnUs experience, but that does not seem commercially interesting to maintain. Many people write on 3+ languages daily or different alphabets.

Thanks for your time, I really appreciate reading this post and I will definitively be checking out some of your other links later.

Best regards,