Recently, my first contribution to Triton (PR #7918) was merged. This features a new matrix multiply operation introduced in sm_120 NVIDIA architectures (RTX 50XX series and RTX Pro 6000). Thanks to @masahi, and @mobiacham, I was able to make my first contribution to triton.

With this addition, fast quantization and accelerated inference us...

Work culture

I’ve seen many people on LinkedIn saying that work-life balance is a lie. They claim that Europe, with its pro–work-life-balance culture, has been declining, while Chinese tech giants—famous for the 996 culture—are on the rise. Every time I see such posts, I block the authors, so I can’t name them, but as I recall, most of them are not from the tech industry. That’s odd to me, and I believe they’re missing some key points.

Sometimes, overwor...

Coming back from hiatus

우울한 시기가 도졌다가, 3-4달을 거쳐 돌아오고 있는 중이다. 잠을 하루에 12시간씩 잤고, 사실 지금도 하루에 10시간은 자고 있는 것 같지만, 아무쪼록 돌아오고 있는 중이다. 우울에 빠지기 위한 계획을 잔뜩 세우고 우울증 시기에 돌입했을 때 어떻게 할 지 프로토콜을 잔뜩 세워놨지만 우울함에게 한 대 쳐맞으면 다 무용지물이 되어버리는 것이다. 운동이 중요하다고 여러 번 계획했지만 운동을 할 의욕도 나지 않는다. 약을 잘 먹어야 한다고 했지만 왠지 모르게 약에 절대 손이 가지 않았다. 뭐 그런 시기도 있는 법이다.

그냥 이런 우울함이 내 삶의 상수라고 생각하자. 그런 시기가 일년에 이따금 오게 되는 법이라고 생각하자. 내가 어찌 할 수가 없는 일이고, 이러한 시기가 찾아오는 것을 내 삶의 계획에 포함시켜야 한다. 너무 무리하는 일은 하지 말고, ...

Introduction

This is a super brief summary of SVDQUant. Please refer to the original paper if interested.

\[Q_X = \operatorname{round}(X / s_X)\]

$s_X = \max(\vert X \vert ) / q_{max}$ and $q_{max} = \text{possible max value in repr}$

\[Q(X) = \text{dequantization of }X = s_X\,Q_X\]

$XW$ can be approximated by

\[XW = Q(X)Q(W) = S_X\,S_W\,Q_X\,Q_W\]

SVDQuant introduces two-path quant...

다짐 I’m proud Nvidian. I work hard, ethically.

I have little experience with open source, but currently I’m neither associated with nor active in any open source project. I want to be, and I have a good plan for it. From my experience, I believe it’s a good plan, so other people who want to get involved but struggle might find it helpful. Keep in mind this is my plan and may not suit everyone.

Precaution

While many guides paint a rosy picture, keep in mind two key points:

Click to read more ...
• Thoughts
• Feb 09, 2025