Stable Diffusion

StableDiffusionPipeline

StableDiffusionPipeline(
    hf_pretrained_model_path: str = "runwayml/stable-diffusion-v1-5",
    vae: str = "default",
    scheduler: str = "dpmsolver",
    model_id: Optional[str] = None,
)

Bases: FoundationalModel

Support for Stable Diffusion for text-to-image generation.

Examples:

Invoking default Stable Diffusion

from baseten.models import StableDiffusionPipeline
model = StableDiffusionPipeline()
image, url = model("A dog is running in the grass")

Using a scheduler

Schedulers can significantly influence how your generated image turns out and the quality of the image. Blueprint provides sensible defaults for a set of common schedulers but you can also define your own scheduler.

from baseten.models import StableDiffusionPipeline
from diffusers import LMSDiscreteScheduler

lms = LMSDiscreteScheduler()
model = StableDiffusionPipeline(scheduler=lms)

image, url = model("A dog is running in the grass")

Using a VAE

A variational auto encoder (VAE) is used within Stable Diffusion to help construct images from prompts. It can have a significant effect on the output of the model. Blueprint supports utilizing 3 VAEs as shown below. You may want to experiment with a new VAE if you want to improve the generation of hands and faces.

from baseten.models import StableDiffusionPipeline

model = StableDiffusionPipeline(vae="stabilityai/sd-vae-ft-mse")

images, url = model("A dog is running in the grass")

You can combine a scheduler and VAE for even more control.

from baseten.models import StableDiffusionPipeline

model = StableDiffusionPipeline(
    "runwayml/stable-diffusion-v1-5",
    vae="stabilityai/sd-vae-ft-mse",
    scheduler="pndm"
)

image, url = model("A dog is running in the grass", seed=2)

Parameters:

Name	Type	Description	Default
`hf_pretrained_model_path`	`str`	Name of the Hugging Face pretrained model to use for text-to-image generation. By default, we will use the v1.5 version provided by Huggingface.	`'runwayml/stable-diffusion-v1-5'`
`vae`	`str`	Name of the pretrained VAE model to use for image reconstruction. By default, we will use the "default" VAE. Users can also specify a custom VAE model either `stabilityai/sd-vae-ft-mse` `stabilityai/sd-vae-ft-ema`	`'default'`
`scheduler`	`str \| SchedulerMixin`	Name of the scheduler to use for the diffusion process. By default, we will use the "dpmsolver" scheduler. Users can also specify a custom scheduler that is either one of `dpmsolver` `ddim` `euler` `lms` `pndm` `eulera` A user can also specify a custom scheduler by passing in a scheduler object that is a subclass of the diffusers.scheduler Mixin.	`'dpmsolver'`
`model_id`	`Optiona;[str]`	ID of the finetuned model to use for text-to-image generation. By default, we will use the generic Stable Diffusion model.	`None`

blueprint_url `property`

blueprint_url: str

If this StableDiffusionPipeline is your fine-tuned model, use this property to get a link to the Blueprint page with your model information. Otherwise, it gives you the Stable Diffusion model page, same as StableDiffusionPipeline.url().

Example:

from baseten.models import StableDiffusionPipeline

model = StableDiffusionPipeline(model_id="MODEL_ID")
model.blueprint_url

call

__call__(
    prompt: str,
    height: int = 512,
    width: int = 512,
    guidance_scale: float = 7.5,
    eta: float = 0.0,
    seed: int = -1,
    num_inference_steps: int = -1,
) -> Tuple[Image.Image, str]

Generate image from a prompt. Supports all parameters of the underlying model from diffusers library.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	Text prompt to generate image from	required
`height`	`int`	Height of the generated image. Defaults to 512.	`512`
`width`	`int`	Width of the generated image. Defaults to 512.	`512`
`guidance_scale`	`float`	Guidance scale for the diffusion process. Defaults to 7.5.	`7.5`
`eta`	`float`	Eta value for the diffusion process. Defaults to 0.0.	`0.0`
`seed`	`int`	Seed for the diffusion process. Defaults to -1, which will use a random seed.	`-1`
`num_inference_steps`	`int`	Number of inference steps for the diffusion process. Defaults to None.	`-1`

Returns:

Name Type Description

generated_images

Tuple[Image.Image, str]

A tuple containing three values:

image: A PIL.Image object
url: An URL linking to the generated image on S3

url `staticmethod`

url()

Use this static method to get a URL to the Stable Diffusion model page in your Blueprint project, which contains information about the model.

Example:

from baseten.models import StableDiffusionPipeline

StableDiffusionPipeline.url()

Stable Diffusion