Linear Layout in Triton (2): LL Examples
Prerequisites
- Part 1 of this series (concepts, GF(2) intuition, and basis vectors)
- A working knowledge of CUDA-thread hierarchy (CTA → warp → lane) or an equivalent AMD mental model
What you will learn here
- How to map hardware dimensions to layout inputs without memorising folklore
- When to pick each constructor (
identity1D,Click to read more ...Linear Layout in Triton (1): Basic Idea
Click to read more ...Introduction
As I study Triton, I realized that one of the most difficult concepts in Triton is
Linear Layout(LL for short), but there are only a few resources to learn about it (see references). Therefore, I’m writing down my understanding and struggles with this interesting and important concept to help others who want to study Triton and Linear Layout.Warning: This post is not...
PTX Mental Model
Simplest matmul using PTX
During the Chuseok (Mid-Autumn Festival) break I decided to build a clear mental model for how
mmaworks at the PTX level.The example below is close to the simplest matrix multiply that mixes CUDA and PTX.
- The problem size is m16n8k16
- A, B, and accumulator operands all use
f16Click to read more ...Adding Scaled Dot Product to Triton
Click to read more ...Recently, my first contribution to Triton (PR #7918) was merged. This features a new matrix multiply operation introduced in sm_120 NVIDIA architectures (RTX 50XX series and RTX Pro 6000). Thanks to @masahi, and @mobiacham, I was able to make my first contribution to triton.
With this addition, fast quantization and accelerated inference us...
Mental disorder and work-life balance
Click to read more ...Work culture
I’ve seen many people on LinkedIn saying that work-life balance is a lie. They claim that Europe, with its pro–work-life-balance culture, has been declining, while Chinese tech giants—famous for the 996 culture—are on the rise. Every time I see such posts, I block the authors, so I can’t name them, but as I recall, most of them are not from the tech industry. That’s odd to me, and I believe they’re missing some key points.
Sometimes, overwor...
7월 19일의 일기
Click to read more ...Coming back from hiatus
우울한 시기가 도졌다가, 3-4달을 거쳐 돌아오고 있는 중이다. 잠을 하루에 12시간씩 잤고, 사실 지금도 하루에 10시간은 자고 있는 것 같지만, 아무쪼록 돌아오고 있는 중이다. 우울에 빠지기 위한 계획을 잔뜩 세우고 우울증 시기에 돌입했을 때 어떻게 할 지 프로토콜을 잔뜩 세워놨지만 우울함에게 한 대 쳐맞으면 다 무용지물이 되어버리는 것이다. 운동이 중요하다고 여러 번 계획했지만 운동을 할 의욕도 나지 않는다. 약을 잘 먹어야 한다고 했지만 왠지 모르게 약에 절대 손이 가지 않았다. 뭐 그런 시기도 있는 법이다.
그냥 이런 우울함이 내 삶의 상수라고 생각하자. 그런 시기가 일년에 이따금 오게 되는 법이라고 생각하자. 내가 어찌 할 수가 없는 일이고, 이러한 시기가 찾아오는 것을 내 삶의 계획에 포함시켜야 한다. 너무 무리하는 일은 하지 말고, ...