On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available in the App Store that lets iPhone owners run the popular Stable Diffusion AI image generator. Enter a description, and the app generates an image in minutes. This is a notable step toward bringing image synthesis to a wider audience—with the added privacy of running it on your own hardware.
Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates novel images from text descriptions (called “prompts”). Typically, people run SD through the commercial DreamStudio service, on a remote cloud machine with rented computing time, or locally on a PC using a custom open source implementation. When running locally, SD requires a fairly beefy GPU to generate images quickly, but some developers optimized the model to run on older GPUs with less VRAM (if you don’t have to wait longer to see results).
In the same way, Liu Liu managed to optimize Stable Diffusion to run on the iPhone, a somewhat difficult process that the developer described in a blog post. “The main challenge is running the app on the 6GiB RAM iPhone devices,” Liu Liu writes. “6GiB sounds like a lot, but iOS will start killing your app if you use more than 2.8GiB on a 6GiB device, and more than 2GiB on a 4GiB device.”
When you first run Draw Things, the app downloads several necessary files—including the Stable Diffusion 1.4 model—to your iPhone. To use it, enter a prompt at the top of the screen, then tap “Generate”. Between generating images, tap the number in the top center of the screen to randomize the seed, which is a number that partially guides the generation of the image.
On our iPhone 11 Pro, generating a 384×384 image took a little more than two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, generating an image in about a minute. In any case, SD is computationally intensive. After successive generations, our iPhone is notably warm to the touch.
It is worth noting that with stable diffusion, 384 × 384 images often generate relatively poor, low-detail results because the creators of SD trained the model with 512 × 512 images. When we tried to generate a 512×512 image on our iPhone 11 Pro, we got a warning and anyway, but the app crashed to a black screen.
In addition to regular image generation duties, Draw Things also supports inpainting, which allows you to replace part of an image with AI-generated images, and loading extra image synthesis models such as the unauthorized “Modern Disney Diffusion” model (which generates Disney – looking characters) and the anime-powered “Waifu Diffusion” model (we tested, and it is possible to generate NSFW material using the app, so be warned). The inclusion of these capabilities means that Draw Things may not stay in the App Store for long if it becomes popular, as the content it generates may violate Apple’s Terms of Service.
Whatever the fate, Draw Things feels like an important proof-of-concept, showing that stable diffusion can run locally on the iPhone, even if slowly. If the app sticks around—perhaps with some filters or modifications—Liu Liu describes room for potential future optimizations that could speed things up: “I’ve probably left 50% performance on the table yet.”
This isn’t the first time Stable Diffusion has run on an iPhone. In September, developer Matt Waller ran Stable diffusion locally on his iPhone XS, but he did not provide an App Store app that would allow others to replicate the feat. We will likely see more local AI image synthesis on smartphones as these devices continue to increase in computational power.
Go to the discussion…