chris looks under the hood of huggingface models such as TinyLlama and Mistral 7B. In the video Chris presents a high level reference model of large language models and uses this to show how tokenization and the AutoTokenizer module works from the HuggingFace transfomer library linking it back to the HuggingFace repository. In addition we look at the tokenizer config and Chris shows how Mistral and Llama2 both use the same tokenizer and embeddings architecture (albeit different vocabularies). Finally Chris shows you how to look at the model configuration and model architecture of hugging face models.

As we start to build towards our own large language model, understanding these fundamentals are critical no matter whether you are a builder or consumer of AI.

Google Colab:
https://colab.research.google.com/dri...