What does HackerNews think of ml-stable-diffusion?

I've used Apple's port of Stable Diffusion on my Mac Studio with M1 Ultra and it worked flawlessly. I could even download models from Hugging Face and convert them to a CoreML model with little effort using Apple's conversion tool documented in their Stable Diffusion repo [1]. Some models on Hugging Face are already converted – I think anything tagged with CoreML.

[1] https://github.com/apple/ml-stable-diffusion

I've played around with many SD frameworks but I typically use: https://github.com/AUTOMATIC1111/stable-diffusion-webui since it has a lot of features and usually works well but it does have quirks/bugs that can be irritating. I use this webui on google colab (check "online services" in readme)

I have a M1 Max and have mostly avoided running SD on my machine since many SD implementations are built for windows or linux, supporting apple silicon had been less of a priority. Many of these frameworks will work on an M1 but i find using colab is simpler with similar performance.

However, since apple has released[1] Core-ML support SD models I have been trying some native apps like Mochi-diffusion[2] they seem to work well but are not yet as feature rich as the older webuis

1: https://github.com/apple/ml-stable-diffusion 2: https://github.com/godly-devotion/mochi-diffusion

Yes, it's using https://github.com/apple/ml-stable-diffusion

Apple has made a sample project available for Stable Diffusion that can run on both macOS and iOS.[1] The sample code is here.[2]

I'll concede that Stable Diffusion may or may not meet the threshold for SOTA but still think this is indicative of inference eventually becoming supported for any compelling LLM on consumer-grade hardware. The possibilities for creative tools are just too vast.

[1] https://machinelearning.apple.com/research/stable-diffusion-...

[2] https://github.com/apple/ml-stable-diffusion

>How come you always have to install some version of pytorch or tensor flow to run these ml models?

The repo is aimed at developers and has two parts. The first adapts the ML model to run on Apple Silicon (CPU, GPU, Neural Engine), and the second allows you to easily add Stable Diffusion functionality to your own app.

If you just want an end user app, those already exist, but now it will be easier to make ones that take advantage of Apple's dedicated ML hardware as well as the CPU and GPU.

>This repository comprises:

    python_coreml_stable_diffusion, a Python package for converting PyTorch models to Core ML format and performing image generation with Hugging Face diffusers in Python

    StableDiffusion, a Swift package that developers can add to their Xcode projects as a dependency to deploy image generation capabilities in their apps. The Swift package relies on the Core ML model files generated by python_coreml_stable_diffusion

https://github.com/apple/ml-stable-diffusion

With the full 50 iterations it appears to be about 30s on M1.

They have some benchmarks on the github repo: https://github.com/apple/ml-stable-diffusion

For reference, previously I was getting about <3 minutes for 50 iterations on my Macbook Air M1. I haven't yet tried Apple's implementation but it looks like a huge improvement. It might take it from "possible" to "usable".