ChatGPT has been out for a while, which raises lots of discussion and interesting on using those artificial intelligence technologies to complete some work orginal done by human. Using AI to draw a realistic photo is another interesting topic which has been deeply developped for certain purpose. In this post, I will show a Github project which you can use it to create your own text-2-image Web UI page then use free Google Colab platofrm to install. You also are able to install it into your local pc to draw the pictures based on your inputs, either a image or some texts, but you had better to have a good performance machine, especially high end graphic card.
Table of Contents
Introduction
Model Details
- Developed by: Robin Rombach, Patrick Esser
- Model type: Diffusion-based text-to-image generation model
- Language(s): English
- License: The CreativeML OpenRAIL M license is an Open RAIL M license, adapted from the work that BigScience and the RAIL Initiative are jointly carrying in the area of responsible AI licensing. See also the article about the BLOOM Open RAIL license on which our license is based.
- Model Description: This is a model that can be used to generate and modify images based on text prompts. It is a Latent Diffusion Model that uses a fixed, pretrained text encoder (CLIP ViT-L/14) as suggested in the Imagen paper.
- Resources for more information: GitHub Repository, Paper.
Github Project: stable-diffusion-webui-colab
1 Go to Github Page : https://github.com/camenduru/stable-diffusion-webui-colab
2 Got to Colab section, select one option to run
Click one button to continue to load it in Colab website.
3 Colab site with loaded code.
Click run button to run those commands in Colab environment.
4 Once completed the running, you can scroll down to find out those two WebUI links:
5 Stable Diffusion WebUI page will be opened:
6 You can enter keywords, sentences or other related words to describe the photo you wants, then click generate to see the output.
You can generate it multiple times to see different results.
Prompt Examples
Example 1:
In a way you gonna have experience as an artist to know what make art look good.
I want a VICTORIAN GIRL in a style of OIL PAINTING
Eye and Face are important in art so she must have PERFECT FACE, SEXY FACE and her eye have DETAILED PUPILS
I want she to have LARGE BREAST, TONED ABS and THICK THIGH.
She must look FEMININE doing EVOCATIVE POSE, SMIRK and FULL BODY wearing NIGHT GOWN
The output must be INTRICATE, HIGH DETAIL, SHARP
And in the style of {I’m not give out the artist names to avoid trouble. Apologize.}
Prompt :
VICTORIAN GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),ARTIST,ARTIST,(ARTIST). OIL PAINTING. (((LARGE BREAST)),((TONED ABS)),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP
This is where you put trait that you dont want to see in the finale output. i.e nipple
Negative Prompt :
((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))
TIPS:
- prompt in () will strengthening the prompt, adding more will strengthening it even further
- prompt in [] will weakening the prompt , adding more will weakening it even further
- Adding more artist will create amalgamation that end up look better or worst, experiment it yourself. I use 4 artist to create most of the output.
- Restore Face is GOD SEND, USE IT.
- It always depend on what model you use.
The level here is whatever, feel free to experiment. This is what I use.
Steps: 42, Sampler: DDIM, CFG scale: 10
Example 2 : Manga style
Prompt : GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP,(((loli)))
Negative Prompt : ((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))
Local Installation of Stable Diffusion Web UI
GitHub Page: https://github.com/automatic1111/stable-diffusion-webui/
Steps:
- Install Python 3.10.6, checking “Add Python to PATH”.
- Install git.
- Download the stable-diffusion-webui repository, for example by running
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git. - Close GUI interface.
Download models (model.ckpt) file, and put them under models directory.- Run
webui-user.batfrom Windows Explorer as normal, non-administrator, user.
http://127.0.0.1:7860/
Total folder size: 9.40GB
Error message:
you might be getting some python related error. You can quickly update pip and reboot your system to try it again.
Fix: reboot system and try it again.
Another error message might relate to your GPU.
AssertionError: Torch is not able to use GPU; add –skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check.
If your system does not have much RAM , you might want to add som other parameters to avoid installing issues:
“set COMMANDLINE_ARGS = –lowvram –precision full –no-half –skip-torch-cuda-test”
Models
Model: Cocoa.ckpt , yohan-diffusion.ckpt
 |
 |
 |
yohan_diffusion_webui_colab andite/yohan-diffusion |
Model Stable Diffusion v2.1
|
|
|
stable_diffusion_v2_1_webui_colab stabilityai/stable-diffusion-2-1 |
It is using Stable Diffusion checkpoint : 768-v-ema.ckpt
Negative Prompt: lowres, low quality, doll, bad anatomy, poorly drawn, ugly, maiden, virgin, malformed, deformed, masculine, manly, black skin, asian, blurry, out of focus, freckles, skin defects, skin marks, long neck, anime, highlights, shadows,
Some Sites to Use AI
LoRA and Checkpoint
What is LoRA?
LoRA models are small Stable Diffusion models that apply tiny changes to standard checkpoint models. They are usually 10 to 100 times smaller than checkpoint models. That makes them very attractive to people having an extensive collection of models.
LoRA models are small modifiers of checkpoint models. You can easily use them in AUTOMATIC1111 by including a phrase in the prompt.
Civitai hosts a large collection of LoRA models. Apply the LORA filter to see only LoRA models. You may find that they all tend to be similar: female portraits, anime, realistic illustration styles, etc.
Hugging Face is another source of LoRA libraries. You will find more varieties of LoRA models. But there are not as many LoRA models there. Their collection is a lot smaller.
Search LoRA models in Hugging Face
Usage:
As an example, I’ll use Arcane LoRA and download the .safetensors file. When the file is finished downloading, put it in stable-diffusion-webui/models/Lora.
For the SD checkpoint: You can use any of your existing models, but for guaranteed pretty good results download the AnyLoRA model. Like the rest of your models, this goes in the folder stable-diffusion-webui/models/Stable-diffusion.
|
|
|
openjourney_v2_diffusion_webui_colab prompthero/openjourney-v2 |
Videos
References
- https://civitai.com/
- https://github.com/camenduru/stable-diffusion-webui-colab
- https://www.youtube.com/watch?v=A05B3cgYHp8
- https://cyberes.github.io/stable-diffusion-models/
- https://github.com/automatic1111/stable-diffusion-webui/
- https://getimg.ai/
- Prompt Examples
- https://arthub.ai/
- SDModels
- Example code of installing minimal Stable Diffusion WebUI on Google Colab
