Prompt Weighing and Blending using for SDXL 1.0 using Compel and 🧨 Diffusers

This notebook demonstrates the following:

• Performing text-conditional image-generations using 🧨 Diffusers.
• Using the Stable Diffusion XL Refiner pipeline to further refine the outputs of the base model.
• Manage image generation experiments using Weights & Biases.
• Log the prompts and generated images to Weigts & Biases for visalization.

Installing the Dependencies

Experiment Management using Weights & Biases

Managing our image generation experiments is crucial for the sake of reproducibility. Hence we sync all the configs of our experiments with our Weights & Biases run. This stores all the configs of the experiments, right from the prompts to the refinement technque and the configuration of the scheduler.

We can make the experiment deterministic based on the seed specified in the experiment configs.

Creating the Diffusion Pipelines

For performing text-conditional image generation, we use the diffusers library to define the diffusion pipelines corresponding to the base SDXL model and the SDXL refinement model.

Text-to-Image Generation

Now, we pass the embeddings and pooled prompts to the Stable Diffusion XL pipeline.

Logging the Images to Weights & Biases

Now, we log the images to Weights & Biases. This enables us to:

• Visualize our generations
• Examine the generated images across different images
• Ensure reproducibility of the experiments

Here's how you can control your prompts using Compel and manage them using Weights & Biases 👇