What does HackerNews think of ml-stable-diffusion?
Stable Diffusion with Core ML on Apple Silicon
It would be instructive to compare their speedups on the iPhone to the Apple CoreML implementation: https://github.com/apple/ml-stable-diffusion
I have a M1 Max and have mostly avoided running SD on my machine since many SD implementations are built for windows or linux, supporting apple silicon had been less of a priority. Many of these frameworks will work on an M1 but i find using colab is simpler with similar performance.
However, since apple has released[1] Core-ML support SD models I have been trying some native apps like Mochi-diffusion[2] they seem to work well but are not yet as feature rich as the older webuis
1: https://github.com/apple/ml-stable-diffusion 2: https://github.com/godly-devotion/mochi-diffusion
I'll concede that Stable Diffusion may or may not meet the threshold for SOTA but still think this is indicative of inference eventually becoming supported for any compelling LLM on consumer-grade hardware. The possibilities for creative tools are just too vast.
[1] https://machinelearning.apple.com/research/stable-diffusion-...
The repo is aimed at developers and has two parts. The first adapts the ML model to run on Apple Silicon (CPU, GPU, Neural Engine), and the second allows you to easily add Stable Diffusion functionality to your own app.
If you just want an end user app, those already exist, but now it will be easier to make ones that take advantage of Apple's dedicated ML hardware as well as the CPU and GPU.
>This repository comprises:
python_coreml_stable_diffusion, a Python package for converting PyTorch models to Core ML format and performing image generation with Hugging Face diffusers in Python
StableDiffusion, a Swift package that developers can add to their Xcode projects as a dependency to deploy image generation capabilities in their apps. The Swift package relies on the Core ML model files generated by python_coreml_stable_diffusion
https://github.com/apple/ml-stable-diffusionThey have some benchmarks on the github repo: https://github.com/apple/ml-stable-diffusion
For reference, previously I was getting about <3 minutes for 50 iterations on my Macbook Air M1. I haven't yet tried Apple's implementation but it looks like a huge improvement. It might take it from "possible" to "usable".