alibaba-pai

/

FLUX.2-dev-Fun-Controlnet-Union

like 13

Follow

Alibaba-PAI 1.8k

same here

same here

What workflow did you use?
Wasn't able to find controlnet workflow for Flux.2

Same here...there is a ControlNet for the new Zimage. Comfyui had to create some nodes to make that work. Probably the same situation. Looks like Zimage is getting all the attention.

Same here:
File "/home/user/app/ComfyUI/nodes.py", line 815, in load_controlnet
raise RuntimeError("ERROR: controlnet file is invalid and does not contain a valid controlnet model.")
RuntimeError: ERROR: controlnet file is invalid and does not contain a valid controlnet model.

I'm working on it,

but it comes out kind of wrong...

It should look like this...

Try to use ReferenceLatent node and place it:

1. after 'Apply VideoX ControlNet' and before 'BasicGuider'
2. after 'FluxGuidance' and before 'Apply VideoX ControlNet'
   nodes to see what will happen

I've about got it. I'm going to clean it up and get it in GitHub after verifying it with more control images and pinpointing is verified. But it's looking good, I think.

pinpointing! Thanks, Apple, I typed INPAINTING!

VAE patchify and RoPE worked out:

Canny seems to work well too

Thank Claude, I more or less vibe-coded this shiz. It seems to work as expected, though.

where do you get the VideoX controlNet Loader _V2

From me. When it finally works I'll share a link.

Depth is working

Inpainting is just meh. It's not going to be good until Black Forest Labs releases a proper fill checkpoint.

I need to get some sample workflows together, some notes, and proper naming because we also need to get a ZIT version as well.

I decided to just go ahead and share. It's not tested with any checkpoints besides the regular bf16 flux.2-dev weights. Hopefully this will work for some of you.

https://github.com/bryanmcguire/comfyui-flux2fun-controlnet

It doesn't work on fp8. I wish it worked on fp8. Also, the FLUX.2-dev-Turbo 8-step LoRa has been released. I wish it worked on 8-step.

Same result with Fp8 as above. Although, if you have the standard Flux2 workflow, you can put a depth map or a open pose in the reference image and that seems to work for me.

I checked again and it works on fp8 too. It didn't work because I connected multiple reference images. I wanted to put in the reference image of the character I wanted and have it be in the control net pose.

updated and I verified it works with fp8 scaled and gguf q4_k_m quants, supports reference latents, and it should support chaining controlnets.

If you find more issues, please put them on https://github.com/bryanmcguire/comfyui-flux2fun-controlnet/issues so I get notifications