A statistical language model is a probability distribution over sequences of words. Due to Google and OpenAI work big pre-trained language models gained recognition as a multitask and few-shot learners bringing as a step closer to general artificial intelligence.

Big pre-trained language models contain a lot of implicit knowledge about the world, however retrieval of that knowledge is not always reliable. These models are also expensive to update with new knowledge, because to do so they would require additional training.

One way to address above issue could be augmenting language models with the capability of traditional search engines like Google. An example attempt at this task is the paper REALM: Retrieval-Augmented Language Model Pre-Training utilizing relatively small 330M parameters model.

This question asks will a language model with at least 100B parameters trained to do external information retrieval exist before 2023?

This question will resolve positive when a language model with at least 100B parameters trained to do external information retrieval will be announced and negative if no such model will be publicly known to exists before 2023.

Importantly, the model must have at least 100B parameters and it must be trained by some means to do external information retrieval as in the REALM paper mentioned above. Just augmenting 100B model with e.g. TF-IDF after per-training will not suffice. The model must be aware of the external information retrieval during the training procedure. The specifics of achieving that goal are not relevant, so any method applied during training will suffice.