ffusionxlBASE_v10DirectmlONNX.zip

5.97 GB · Diffusers · fp16 · full

ffusionxlBASE_v10DirectmlONNX_119853.onnx

189 MB · Other · fp32 · full

ffusionxlBASE_v10DirectmlONNX_119852.onnx

65 MB · Other · fp16 · full

<ffusion1-xl>

🌌 FFusion/FFusionXL-BASE: Now Available in ONNX, DirectML, Intel OpenVINO Format This model serves as a foundational base, primed primarily for training purposes diffusers available at FFusion/FFusionXL-BASE. Beyond that, it also plays an instrumental role in inference and provides a benchmark for evaluating our LoRA extractions. 🌟 Overview - 🚀 Fast Training: Optimized for high-speed training, allowing rapid experimentation. - 🧩 Versatility: Suitable for various applications and standards, from NLP to Computer Vision. - 🎓 Train Your Way: A base for training your own models, tailored to your needs. - 🌐 Multilingual Support: Train models in multiple languages. - 🛡️ Robust Architecture: Built on proven technologies to ensure stability and reliability. 📜 Model Description FFusionXL "Base" is a foundational model designed to accelerate training processes. Crafted with flexibility in mind, it serves as a base for training custom models across a variety of standards, enabling innovation and efficiency. Available formats for training: - Safetensor checkpoints fp16 & fp32 - Diffusers(safetensors) FP 16 & FP32 - Diffusers(pytorch bin) FP16 & FP32 - ONNX un-optimzed FP32 - ONNX Optimized FP16 full DirectML support / AMD / NVIDIA - Intel® OpenVINO™ FP32 - unoptimized - Intel® OpenVINO™ FP16 - Trained by: FFusion AI - Model type: Diffusion-based text-to-image generative model - License: - Model Description: This is a trained model based on SDXL that can be used to generate and modify images based on text prompts. It is a that uses two fixed, pretrained text encoders ( and ). - Resources for more information: . 📊 Model Sources - Demo: Table of Contents - - - - - - - - - - - - - 1. 2. 3. 4. 5. 6. 7. - ### 📌 ONNX Version We are proud to announce a fully optimized Microsoft ONNX Version exclusively compatible with the latest DirectML Execution Provider. All the ONNX files are optimized (Quantization) to fp16 for fast inference and training across all devices. The Vae_Decoder is kept at fp32 with settings: "float16": false, "use_gpu": true, "keep_io_types": true, "force_fp32_ops": ["RandomNormalLike"] to avoid black screens and broken renders. As soon as a proper solution for a full fp16 VAE decoder arrives, we will update it. VAE encoder and everything else is fully optimized 🤟. Our ONNX is OPTIMIZED using ONNX v8: - producer: onnxruntime.transformers 1.15.1 - imports: ai.onnx v18, com.microsoft.nchwc v1, ai.onnx.ml v3, com.ms.internal.nhwc v19, ai.onnx.training v1, ai.onnx.preview.training v1, com.microsoft v1, com.microsoft.experimental v1, org.pytorch.aten v1, com.microsoft.dml v1, graph: torch_jit 🔖 ### 📌 ONNX Details NETRON Detrails: Install macOS: the .dmg file or run brew install --cask netron Linux: the .AppImage file or run snap install netron Windows: the .exe installer or run winget install -s winget netron -- NETRON browser version: --NETRON browser version: --NETRON browser version: --NETRON browser version: --NETRON browser version: 🔖 ### 📌 AMD Support for Microsoft® DirectML Optimization of Stable Diffusion AMD has released support for Microsoft DirectML optimizations for Stable Diffusion, working closely with Microsoft for optimal performance on AMD devices. 🔖 ### 📌 ONNX Inference Instructions 🔖 ### 📌 Text-to-Image Here is an example of how you can load an ONNX Stable Diffusion model and run inference using ONNX Runtime: from optimum.onnxruntime import ORTStableDiffusionPipeline model_id = "FFusion/FFusionXL-BASE" pipeline = ORTStableDiffusionPipeline.from_pretrained(model_id) prompt = "sailing ship in storm by Leonardo da Vinci" images = pipeline(prompt).images ### 📌 Intel® OpenVINO™ Version A converted Intel® OpenVINO™ model is also included for inference testing and training. No Quantization and optimization applied yet. ### 📌 OpenVINO Inference with FFusion/FFusionXL-BASE 🔖 ### 📌 Installing Dependencies Before using OVStableDiffusionXLPipeline, make sure to have diffusers and invisible_watermark installed. You can install the libraries as follows: pip install diffusers pip install invisible-watermark>=0.2.0 🔖 ### 📌 Text-to-Image Here is an example of how you can load a FFusion/FFusionXL-BASE OpenVINO model and run inference using OpenVINO Runtime: from optimum.intel import OVStableDiffusionXLPipeline model_id = "FFusion/FFusionXL-BASE" base = OVStableDiffusionXLPipeline.from_pretrained(model_id) prompt = "train station by Caspar David Friedrich" image = base(prompt).images[0] image.save("train_station.png") 🔖 ### 📌 Text-to-Image with Textual Inversion First, you can run the original pipeline without textual inversion: from optimum.intel import OVStableDiffusionXLPipeline import numpy as np model_id = "FFusion/FFusionXL-BASE" prompt = "charturnerv2, multiple views of the same character in the same outfit, a character turnaround of a beautiful cyber female wearing a black corset and pink latex shirt, scifi best quality, intricate details." np.random.seed(0) base = OVStableDiffusionXLPipeline.from_pretrained(model_id, export=False, compile=False) base.compile() image1 = base(prompt, num_inference_steps=50).images[0] image1.save("sdxl_without_textual_inversion.png") Then, you can load charturnerv2 textual inversion embedding and run the pipeline with the same prompt again: # Reset stable diffusion pipeline base.clear_requests() # Load textual inversion into stable diffusion pipeline base.load_textual_inversion("./charturnerv2.pt", "charturnerv2") # Compile the model before the first inference base.compile() image2 = base(prompt, num_inference_steps=50).images[0] image2.save("sdxl_with_textual_inversion.png") 🔖 ### 📌 Image-to-Image Here is an example of how you can load a PyTorch FFusion/FFusionXL-BASE model, convert it to OpenVINO on-the-fly, and run inference using OpenVINO Runtime for image-to-image: from optimum.intel import OVStableDiffusionXLImg2ImgPipeline from diffusers.utils import load_image

SHA256

2683382EB0880175A4E0047B2E62239A23C41A8B4206ED7AE4688FE56ACC9149

ModelTrends.ai-Modell-ID

#7529