Goals
I would like to be able to run an instance of A1111 or ComfyUI remotely. My main desire for this is the fact that I only have a GTX1080 in my home machine and sometimes generating images can be quite slow.
Issues
I have done this before and the results are pretty good. The issue comes down to cost and setup.
Cost
Obviously renting a cloud GPU is not free. I have made use of the Vast.ai service before and the prices seem rather good. For example, if I were to buy a new RTX 3080 (this is just a random example) it would cost me roughly $1,000 AUD. However, if I rent an RTX 3080 on vast.ai then it cost roughly 0.20 USD an hour. Given the current exchange rate (\1USD ~= $1.50AUD) then I can get well over three thousand hours of usage of the cloud GPU.
There are a few other cost to consider. For instance the storage is a cost consideration. Speaking of storage I saw a post on Reddit about cheap storage fee and someone suggested checking out Hetzner for a block of storage. The idea is that you store all your models and photos on the Hetzner block storage then mount it to your GPU instance and use it as a drive. The cost is roughly $6AUD a month which isn’t too bad. One thing that might be an issue is the fact that the storage is only in Helsinki (Finland) or Falkenstein (Germany).
Why not just buy a new GPU?
Well, part of the issue is that I don’t think I will actually use the image generation that much. If I needed to make lots of images for a commercial purpose then I’d start to look at a GPU then. I know that I could also use the GPU for games, but I don’t play games that often, and when I do the GTX1080 works fine. I have never felt the need to have everything look ultra-HD to enjoy a game.
Rough plan
Rent the 1TB Hetzner storage block. Then I will create a bash script that I can run on any instance of ComfyUI or StableDiffusion that will connect the storage block to the machine and repoint all the folders to use the block storage.
Progress
- Rent Storage block from Hetzner
- Create setup scripts
- ???
- Profit
Scripts
A1111
Things I have done since logging in:
- Create mount folder
- install cifs-utils
Snags
So I have run into a bit of a snag. I was planning on using the Hetzner storage block on a vast.ai instance. However, vast.ai instances are actually docker containers, and you can’t mount a drive inside of docker containers.
Potential fix
I can get rclone to work instead. So what I’ll do is have my startup script setup rclone then clone the model folders to the local image. I’ll have to create one more script that will automagically upload all image generation to my hetzner box via rclone again. Or maybe I’ll just learn the command to do it manually.
Side Note: Cost
I was curious about what was the best bang for buck on vast.ai. After a not-to-thorough web search I found this Tom’s Hardware article where they look at the performance of a few different GPU’s and record the average images per minute. I wasn’t worried about a super exact recording, just a generic overview of the performance of each card.
I then went to vast.ai and had a look at the network calls. To my very great surprise simply calling the base API returns all the results of all their machines available.
Now I could bore you with how I used DuckDB to consume the stupid large JSON file I got and the CSV I made from the Tom’s Hardware data in the article, but for now it is enough to simply say I did it. (Spoiler: I did anyway)
So, what’s the results? I am glad you asked!
| Average cost per hour ($) | Images Per Minute | Images per average cost | GPU |
|---|---|---|---|
| 0.10778260869565218 | 26.04 | 14495.845098830172 | RTX 3070 |
| 0.1786 | 41.62 | 13982.082866741319 | RTX 3080 Ti |
| 0.1288 | 29.31 | 13653.726708074533 | RTX 2080 Ti |
| 0.13 | 28.7 | 13246.153846153846 | RTX 3070 Ti |
| 0.20093750000000005 | 37.0 | 11048.211508553652 | RTX 3080 |
| 0.27307692307692305 | 46.85 | 10293.802816901409 | RTX 3090 Ti |
| 0.2656593087016583 | 42.9 | 9689.101475795313 | RTX 3090 |
| 0.15 | 23.71 | 9484.000000000002 | RTX 4060 Ti |
| 0.4755596828767127 | 75.13 | 9478.93642440802 | RTX 4090 |
| 0.35000000000000003 | 51.55 | 8837.142857142857 | RTX 4080 |
| 0.13384000000002 | 17.81 | 7984.160191271968 | RTX 3060 |
| 0.4809400000000001 | 23.3 | 2906.807501975298 | RTX 3060 Ti |
| 0.8 | 31.95 | 2396.25 | Titan RTX |
(If you are curious, I multiplied the Images Per Minute to get Images per hour then divided that by the average cost per hour which gives me the images per average cost!)
Now, these numbers represent (if my math is correct) the number of images you can generate per $1 spent. However, this makes the HUGE assumption that you are generating images non stop.
It also doesn’t take into account things like your willingness to wait for the images and whatnot. Also this completely ignore the rest of the machine’s capabilities. So it’s entirely possible that you’ll be held back by the CPU or RAM. I think personally I’ll use either the RTX 3070 or the RTX 3080Ti if I am feeling impatient.