GIMP Forum

This procedure explains how to generate highres Flux images starting from line art, ie. black lines on a white background.

doe.1971 wrote: ↑Thu Apr 24, 2025 6:02 pm If you could make a step by step guide of what to install, from where, what buttons to press to generate AI based on a drawing or other image, I'm sure there will be a chorus of appreciative voices around here.

I do have a 8GB RTX 4060 Ti on a Ryzen 9, 16 cores, 128 GB RAM. I lack the guiding hand to make it churn stuff.

The hardest part is installing ComfyUI. Once it is installed, copy the workflow file, below, onto your local machine, then drag and drop it on top of a pre-opened ComfyUI window. You may be prompted to install missing custom nodes (hit yes). If some nodes are still unrecognized, go to the ComfyUI Manager Menu and click Install Missing Custom Nodes.

I use an RTX 4060 Ti with 16GB of VRAM (= memory on the graphics card), with 32GB of system memory. Not bad, not too fancy either. Flux requires a lot of VRAM. This workflow will work on lower spec machines, to a certain point; in cases of insufficient VRAM, ComfyUI will try to load part of the model in system memory. This works but image generation will be slower. If a PC doesn't have enough VRAM and it doesn't have enough system memory either, ComfyUI will throw an insufficient memory error.

Workflow file: https://filebin.net/svtmfphl88wgyzxz

Result (input, output):

You need to download the following model files (you may have to create an account before you can download the files) and put them in the correct folder, before you can generate images:

msFluxSFWNSFWV3_v3.safetensors : https://civitai.com/models/1333695/ms-flux-sfwnsfw-v3
Location: ComfyUI/models/unet

t5xxl_fp8_e4m3fn.safetensors : https://huggingface.co/comfyanonymous/f ... afetensors
Location: ComfyUI/models/clip

clip_l.safetensors : https://huggingface.co/comfyanonymous/f ... afetensors
Location: ComfyUI/models/clip

fluxVaeSft_aeSft.sft : https://huggingface.co/Albert-zp/flux-v ... _aeSft.sft
Location: ComfyUI/models/VAE

ControlnetUnion_v10_Quant.safetensors : https://civitai.com/models/1488208/flux ... -pro-20fp8
Location: ComfyUI/models/ControlNet
Note: In my workflow I've renamed this file and placed it in a subfolder called Flux. As long ComfyUI finds the right file, it will work.

4x_NMKD-Siax_200k.pth : https://civitai.com/models/147641/nmkd-siax-cx
Location: ComfyUI/models/ESRGAN
Note: Rename the file to 4x_NMKD-Siax_200k.pth

The workflow is separated into four steps:

1. Model Loader (Flux)
This is where you can change the Flux model, add LoRAs (optional) and type your prompt

2. ControlNet
This is where you load your lineart file (drag the file onto the Load Image node). You need to play around with the Canny parameters (low_threshold, high_threshold) until you have an image that has the contours you want, without too many extra lines that will make it harder for Flux to come up with a matching result. Hit Queue, then Cancel Current Run once the canny image appears in Preview Image. Once you are happy with the canny image, you can set the Apply ControlNet parameters: strength for how strong the contours should be applied, and end_percent for when the constraints can be released (ie. 50% means that after the first half of the generation, the model can do what it wants). If these parameters are too strict, Flux won't find a good solution and will be putting limbs all over the place. Sometimes if the constraints are too tight, the canny lines appear in the final image, like here.


3a. Generator
When you're happy with your group 2 settings, proceed to this step. You generate an image by pressing the Queue button in ComfyUI. Flux will generate a basic image based on your chosen settings. Parameters you can set here are Steps (more steps = more detail = longer generation time). The RandomNoise node contains the seed (an integer number) that initializes the noise generator (a long story but unimportant at this point). Same seed with identical settings = same image. Different seed = different generation.

4a. Save Basic Image
This saves your generated image in the ComfyUI output folder.

4b. HighRes "Fix"
This step performs what's called the HighRes Fix step. It takes a basic image, upscales it, and re-generates it in higher resolution. This adds detail. Drawback is that it takes a lot more time. Settings you can change here are Denoise (higher is more variation) and Steps. I usually get nice results with 0.2 Denoise and 10 steps.

3b. Load Basic Image
A trick you can use is to generate a batch of basic images first without having to highres all of them. To do this, deactivate HighRes by Rightclicking on group 4b and clicking Bypass Group Nodes. Generate a bunch of basic images with HighRes deactivated. Inside the output folder, delete the basic images you don't like, keeping only the basic files you want to highres. Now deactivate the generator, group 3a. Activate HighRes by Rightclicking on group 4b and clicking Set Group Nodes to Always. Connect the Image output of group 3b: click and drag the blue dot that says Image to the image input of group 4b. Now you can use group 3b to load the basic images you want to highres (drag from your folder onto the Load Image node). Hit Queue to HighRes your selected image.

The size of your lineart image matters a lot. Stable Diffusion models are trained on images with a certain size. Your lineart image is essentially an input element for Flux that Flux will try to replicate. If the dimensions are alien to Flux, the results won't be great. Standard dimensions are 1024x1024 (square), with landscape and portrait dimensions having a smaller width and greater height, or vice versa. There's quite a bit of tolerance around these values, but you will start getting poor results below 540px and above 2kpx. I tend to pick screen size proportions, ie. 768x1344 for my vertical screen. The highres step upscales to twice that size, so the end result is plenty big enough.

To illustrate, I initially halved the lineart image for a size of 1160x1680. On the large side. Some of the generated results were hotties, but I didn't get any swords.

ComfyUI generates PNG files which contain, next to the generated image, also the workflow that was used to generate it. This is great as it lets you drag and drop PNG files generated by ComfyUI onto a ComfyUI window, and ComfyUI will automatically load the workflow that was used to generate the image. If you don't want to share your workflow, then you should convert your .PNG files to JPEG instead, as JPEG images only contain image data. I use MS Paint for this as it's simple, fast and free. Open the .PNG in MS Paint, Save As -> JPEG picture. Done.

I have my weekend project set! Thanks a heap!

No worries, have fun.

xs70 wrote: ↑Fri Apr 25, 2025 9:21 am ...Done.

This was a fantastic series of posts.
Thank you xs70!

yw

One change I had to make is moving the upscaling model from location: ComfyUI/models/ESRGAN to ComfyUI\models\upscale_models since it couldn't find it in ESRGAN.