SambaNova for Open WebUI



I made a SambaNova provider manifold for Open WebUI that lists models automatically with web scraping.

Check it out: https://openwebui.com/f/olof/sambanova

💡 Listing models with web scraping

The SambaNova API is largely OpenAI-compatible. However, there is no model list API endpoint at this time.

Previous solutions have solved this by hard-coding the available models, such as this one https://openwebui.com/f/fabiofalopes/sambanova.

🙈 I figured out this alternative approach:

def get_sambanova_models(self):
    # Hey, wanna see something crazy?
    # SambaNova offers no model listing API endpoint at the moment.
    # However, we can scrape it from the documentation:
    # https://community.sambanova.ai/t/supported-models/193
    response = requests.get("https://community.sambanova.ai/t/supported-models/193")
    response.raise_for_status()
    soup = BeautifulSoup(response.text, "html.parser")
    code_texts = [code.get_text() for code in soup.find_all("code")]
    return code_texts

It is just like doing [...document.querySelectorAll('code')].map(el => el.textContent) in the browser console:

SambaNova Document Selector Screenshot

📝 Full Code

"""
title: SambaNova
description: https://cloud.sambanova.ai
author: Olof Larsson
author_url: https://olof.tech/sambanova-for-open-webui
version: 1.0.0
license: MIT
"""

import os
import requests
from typing import List, Union, Generator, Iterator, Optional
from pydantic import BaseModel, Field
from bs4 import BeautifulSoup

# Learn more here:
# https://olof.tech/sambanova-for-open-webui

class Pipe:
    class Valves(BaseModel):
        SAMBANOVA_API_KEY: str = Field(
            default=os.getenv("SAMBANOVA_API_KEY", ""),
            description="Get your key here: https://cloud.sambanova.ai",
        )

    def __init__(self):
        self.type = "manifold"
        self.id = "sambanova"
        self.name = "sambanova/"
        self.valves = self.Valves()

    def get_sambanova_models(self):
        # Hey, wanna see something crazy?
        # SambaNova offers no model listing API endpoint at the moment.
        # However, we can scrape it from the documentation:
        # https://community.sambanova.ai/t/supported-models/193
        response = requests.get("https://community.sambanova.ai/t/supported-models/193")
        response.raise_for_status()
        soup = BeautifulSoup(response.text, "html.parser")
        code_texts = [code.get_text() for code in soup.find_all("code")]
        return code_texts

    def pipes(self) -> List[dict]:
        sambanova_models = self.get_sambanova_models()
        return [{"id": name, "name": name} for name in sambanova_models]

    def pipe(
        self, body: dict, __user__: Optional[dict] = None
    ) -> Union[str, Generator, Iterator]:
        # Remove the "sambanova." prefix
        body["model"] = body["model"].partition(".")[2]

        headers = {
            "Authorization": f"Bearer {self.valves.SAMBANOVA_API_KEY}",
            "Content-Type": "application/json",
        }

        try:
            r = requests.post(
                url="https://api.sambanova.ai/v1/chat/completions",
                json=body,
                headers=headers,
                stream=True,
            )

            r.raise_for_status()

            if body["stream"]:
                return r.iter_lines()
            else:
                return r.json()
        except Exception as e:
            return f"Error: {e}"

🔗 Related Posts

I recently made Open WebUI plugins for all of the three big inference hardware companies: