LLM Training vs Serving in Terms of Memory Access Patterns
This article investigates differences of LLM training vs serving in terms of memory access patterns.
Recent posts
New Turkish Pre-training Datasets
Language model pre-training requires massive amount of high quality text. For Turkish language pre-training there are not many options available.
Haber GPT-2
HaberGPT2 is a decoder only language model with 100M parameters. It is trained from scratch using turkish news. This post shares details about training proce...
Transformer Pre-training Notes
In this post, I have compiled architectural and training details of prominent language models. All information is based on the published papers.
Complex Numbers
Standard Form \(z = x + iy\)