What does HackerNews think of micrograd?

Watch Karpathy's recent lectures. They're gold. Start here[1] with micrograd[2]. It doesn't use linear algebra/matrices to start, but the principles are the same. The matrix multiplication is how the weights of the connections between neurons and the input values are combined (to form an activation value that then may lead to that neuron "firing" or not, depending on whether it passes some threshold function). We use matrices to model the connections between neurons - each row is a connection, and each column is a weight corresponding to an input.

[1] https://www.youtube.com/watch?v=VMj-3S1tku0 [2] https://github.com/karpathy/micrograd

Understanding Automatic Differentiation in 30 lines of Python | Aug 2023

Interesting video by Andrej Karpathy building an autograd engine, quite insightful:

https://youtu.be/VMj-3S1tku0?si=wuKhELwOwoYbzpt7

Repo:

https://github.com/karpathy/micrograd

Understanding Automatic Differentiation in 30 lines of Python | Aug 2023

Also micrograd from Andrej Karpathy: https://github.com/karpathy/micrograd

Ask HN: Does ChatGPT scare you? It scares me | Dec 2022

Its just another evidence of what I already came to believe once I understood the concept of ML about 6 years ago.

1. There are a good subset of jobs across multiple industries that are simply "decision tree lookup" operations. These types of job will most certainly be replaced. For example, I worked for an aerospace company, we hired a consultant for advising on a manufacturing process. He basically looked at what we are trying to make, and advised on the tooling, process, e.t.c. This is the type of job that can be easily done by a future version ChatGPT that is sufficiently trained on both text and mathematical contexts. Software jobs often fall into above category, replicating common patterns that developers have learned. ChatGPT right now is even smart enough to take an input json and output json and write code to transform one into the other.

2. The actual "compute" operations jobs (like making software that requires figuring out a new pattern of transforming data or interfacing with a new piece of hardware like a 3D display) won't be replaced, but the skill will shift to a lot more computer science centric in being able to either a) additive train generic models on specific tasks, or b) use state of the art AI assisted tools effectively.

3. Overall, quality of life is going to improve, as it will get a lot cheaper to do things.

TLDR; if are a software dev and you haven't already, get super familiar with ML concepts, Pytorch, etc.

https://github.com/karpathy/micrograd is a very good primer to start with once you understand the basic concepts.

TinyGL 0.4.1 | Nov 2022

Expand Context ↕

You might like micrograd: https://github.com/karpathy/micrograd. There's also this video where the author explains how he built it: https://www.youtube.com/watch?v=VMj-3S1tku0

Tinygrad | Apr 2022

Also: https://github.com/karpathy/micrograd

Neural Network from Scratch | Jan 2022

For autograd from scratch, see https://github.com/karpathy/micrograd and/or

https://windowsontheory.org/2020/11/03/yet-another-backpropa...

The matrix calculus you need for deep learning (2018) | Apr 2021

You do not need any matrix calculus for deep learning and micrograd (https://github.com/karpathy/micrograd), which implements backpropagation for neural nets in 100 lines of code, is a proof. Everything else is just vectorization.

From scratch: reverse-mode automatic differentiation (in Python) | Jun 2020

Nice introductory resource.

On the same topic, see also:

- Andrej Karpathy's elegant micrograd library: https://github.com/karpathy/micrograd

- This tiny neural networks API inspired by it: https://github.com/bpesquet/pyfit