Visualizations for OpenWebUI
How to use dynamic image URLs to let LLMs break free from the constraints of plain text and express themselves with the visual richness they deserve!
📌 TL;DR
I made an Open Source plugin for OpenWebUI that exposes dynamic image urls.
- 🔗 GitHub: https://github.com/oloflarsson/VisualizationsForOpenWebUI
- 🔗 OpenWebUI: https://openwebui.com/posts/9cafe09d-c27e-47a4-a019-0e0c2cc30162
This allows LLMs to create visualizations on the fly, such as charts, without any need for tool calls!
📖 Once upon a time...
... boring plain text and a sad markdown table inspired me to ask for a QuickChart 👇
The LLM output in the example above is just regular token output that happens to form a markdown image tag:
I'll create a chart showing the Stockholm temperature forecast using QuickChart. Let me prepare the properly URL-encoded data:
%22%2C%22backgroundColor%22%3A%22rgba(255%2C99%2C132%2C0.2)%22%2C%22fill%22%3Afalse%7D%2C%7B%22label%22%3A%22Low%20Temp%20%28%C2%B0C%29%22%2C%22data%22%3A%5B-5%2C-5%2C-4%2C-3%2C0%2C-2%2C-3%5D%2C%22borderColor%22%3A%22rgb(54%2C162%2C235)%22%2C%22backgroundColor%22%3A%22rgba(54%2C162%2C235%2C0.2)%22%2C%22fill%22%3Afalse%7D%5D%7D%2C%22options%22%3A%7B%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Stockholm%20Weather%20Forecast%20-%20Nov%2024-30%22%7D%2C%22scales%22%3A%7B%22yAxes%22%3A%5B%7B%22scaleLabel%22%3A%7B%22display%22%3Atrue%2C%22labelString%22%3A%22Temperature%20%28%C2%B0C%29%22%7D%7D%5D%7D%7D%7D)
The chart shows the daily high and low temperatures for Stockholm from Monday, Nov 24 through Sunday, Nov 30. You can see temperatures will gradually warm up toward the end of the week, with Sunday having the highest temperature at 3°C.
💡 This, I realized, was a fun concept to expand upon. Instead of using custom tool calls or MCP servers to generate visual content we might just make the LLM aware of some dynamic image URLs.
📊 Charts with Vega-Lite
Calling a third party service like https://quickchart.io is sure convinient, BUT:
- InfoSec considerations. It leaks your data to a third party.
- It will cost you money eventually, or free version gives you poor resolution charts.
Luckily, there's https://vega.github.io/vega-lite and https://github.com/vega/vl-convert that when exposed as an endpoint works about equally well.
So, I began with making an OpenWebUI filter function, see https://docs.openwebui.com/features/plugin/functions/filter, that exposes such a declarative url and appends a system instruction looking like this:
<visualization_chart>
To create a chart, simply respond with a Markdown image tag pointing to `/visualization/chart.png` with the Vega-Lite spec JSON in the `spec` parameter.
**Important:** No tool calling is required! Just type the Markdown image directly.
**URL Formatting:**
- URL-Encode the JSON spec: The JSON spec string must be URL-encoded to be passed as a parameter.
- URL-encode parentheses `(` as `%28` and `)` as `%29` within the spec. Unencoded parentheses will break the Markdown image syntax since they may be interpreted as URL termination.
**Best Practices:**
- **Use a Data URL:** Whenever possible, use `data.url` pointing to an exposed/public url with the CSV/JSON file.
- **Keep it Short:** `width`, `height`, and `$schema` are all optional and will be set automatically.
- **Add a title:** Always add a `title` property to the chart spec.
**Examples**
</visualization_chart>
Usable like this:
Here's another two examples based on data from https://github.com/datasets/five-thirty-eight-datasets/tree/main/datasets
🚀 But why stop there?
😱 So many possibilities for declarative visualizations when you think about it!
/visualizations/qr-code.png?data=https%3A//github.com/oloflarsson/VisualizationsForOpenWebUI
/visualizations/fancy-text.svg?text=%F0%9F%A6%A6%20Lorem%20ipsum%20dolor%20sit%20amet
/visualizations/color-swatch.svg?colors=%232e3440,%233b4252,%23434c5e,%234c566a,%238fbcbb...
😚 Getting excited yet?
Did I mentioned this is OpenSource and available for you to use in OpenWebUI today already:
- 🔗 GitHub: https://github.com/oloflarsson/VisualizationsForOpenWebUI
- 🔗 OpenWebUI: https://openwebui.com/posts/9cafe09d-c27e-47a4-a019-0e0c2cc30162
🤖 No? Okay, there's also the mandatory generative AI image:
/visualizations/generative-image.jpg?aspect_ratio=21:9&prompt=Photo%20of%20a...
There's multiple providers available for image generation configurable in the filter function valves:
🤡 Memelord
I also added such a url for meme generation:
It is composable. Takes any image url (even a previous dynamic one) and adds the black border and text at the bottom.
This is a great site to find public domain images with that old obscure flair: https://pdimagearchive.org
Here's a couple of fresh (old and stale) memes I made for you:
😔 Limitations
Two key things to keep in mind:
1️⃣ Must use state of the art model, such as latest Anthropic Claude.
Since this approach does not involve tool calls the URLs are fire-and-forget. No feedback loop back to the LLM. The LLM basically has one chance to get it right. Also, regular tools have schemas that can be used to avoid hallucination and formating errors. Luckily, state of the art models almost always succeed in typing out the urls.
2️⃣ Chart generation needs some CPU cores. Avoid the cheapest VPS.
The https://github.com/vega/vl-convert used to generate chart images actually requires some hardware. I had to beef up my Hezner VPS to mid tier and 8 cores to get a plesant experience.
🤩 Benefits
1️⃣ Quicker than tool calling
The visualization urls can just be typed out directly and then the LLM moves on to write other things.
I even implemented a placeholder loading state for charts and generative images where there can be a bit of waiting time. This placeholder helps avoids layout shift.
2️⃣ More fluid than tool calling
The LLM will naturally mix images and text in the same answer. Tool calling based solutions often result in a single visualization in the beginning and not as interleaved and fluid.
3️⃣ Locally rendered
Except for the generative AI images the visualizations are rendered locally. Great for information security.
🏎️ Cerebras Code - Crazy fast!
For this project I went all in on Cerebras Code with these vibe engineering components:
- Inference Provider: https://www.cerebras.ai
- AI Model:
GLM-4.6 - Agent Harness: https://kilocode.ai
So, 5 November Cerebras swapped out Qwen 3 Coder for GLM-4.6... Great move! https://www.cerebras.ai/blog/glm
I will say it is not as smart as Anthropic Claude Sonnet 4.5, but it's so damn fast I don't mind another feedback round.
❌☕ I can stay in the zone now. Focused. No need to go have a coffee break waiting for the LLM to finish responding.
✂️ OpenWebUI plugins are single python source files. AI did not hold up once that single file grew too large. I eventually had to split the single source file into multiple smaller files and add a build.py script that combines them. Then vibe engineering started working again. So, my take is that encapsulation and separation of concerns is key context engineering when vibe engineering.
🤔 Some final pondering
"LLMs just typing black text on white background is boring"
- Olof Larsson, 2025
📈 I've had great use of the charts visualization already. My deep research agent includes such charts in the reports. It actually helps me understand the content and makes for more fun-to-read reports.
🤡 Also, think of the poor LLMs. How would you feel about being forced to just type plain text all day long.
🫵 So, I encourage you:
