b6675cfb42
- Step-by-step setup guide (RouterAI key, local Gemma) - Agent integration checklist with pseudo-code - What to ask the user and how to store config - What happens when each backend is missing
187 lines
6.7 KiB
Markdown
187 lines
6.7 KiB
Markdown
# Vision Analyze — Image Analysis Skill for AI Agents
|
||
|
||
Analyze images with two backends: fast cloud (RouterAI gpt-4o-mini) or private local (Gemma 4B via Jan API). Smart routing via emoji flags — NSFW content never touches cloud APIs.
|
||
|
||
## 🚀 Setup Instructions for AI Agents
|
||
|
||
### Step 1: Check RouterAI API Key
|
||
|
||
The script reads the API key from the OpenAI‑style provider config at:
|
||
|
||
```
|
||
models.providers["routerai-ru"].apiKey
|
||
```
|
||
|
||
As an agent, you should do this at setup time:
|
||
|
||
```python
|
||
# Pseudo-code for agents integrating this skill
|
||
try:
|
||
config = load_config("openclaw.json") # or your config file
|
||
api_key = config["models"]["providers"]["routerai-ru"]["apiKey"]
|
||
except (KeyError, FileNotFoundError):
|
||
api_key = None
|
||
|
||
if not api_key:
|
||
# Ask the user for their RouterAI API key
|
||
api_key = ask_user("🔑 Введите API-ключ RouterAI (https://routerai.ru):")
|
||
# Write it to your config under models.providers["routerai-ru"].apiKey
|
||
```
|
||
|
||
The script will work **without** a key only if you always use `--local` (Gemma). However, the default routing goes to RouterAI, so a key is strongly recommended.
|
||
|
||
**What is RouterAI?** A Russian LLM API aggregator (routerai.ru). Supports many models including gpt-4o-mini, Gemini, Claude. You need an account and an API key. The key is used for fast, cheap vision analysis (~₽0.5 per request).
|
||
|
||
### Step 2: Check Local Gemma (Optional)
|
||
|
||
Local Gemma runs on a Jan API / llama.cpp server (http://192.168.88.183:1337). It provides:
|
||
- Free, private image analysis (no external API calls)
|
||
- No censorship filters (handles NSFW/16+ content)
|
||
- Slower (~30-90s per request)
|
||
|
||
Ask the user:
|
||
|
||
```python
|
||
has_local = ask_user("💻 Есть локальный сервер Gemma (Jan/llama.cpp) на 192.168.88.183:1337? (y/n)")
|
||
if has_local.lower() in ("y", "yes", "да"):
|
||
local_url = ask_user("🌐 URL локального сервера:", default="http://192.168.88.183:1337")
|
||
local_key = ask_user("🔑 API-ключ локального сервера (если нужен):", default="")
|
||
# Store in your config, e.g. models.providers["jan-local"].apiKey
|
||
else:
|
||
# Note: --local flag will not work. All requests go to RouterAI.
|
||
logger.info("Локальная модель не настроена. --local будет недоступен.")
|
||
```
|
||
|
||
### Step 3: Verify
|
||
|
||
```bash
|
||
python3 vision-analyze.py test_image.jpg
|
||
# Should return a description in Russian, 3-5 sentences
|
||
```
|
||
|
||
## Requirements
|
||
|
||
- Python 3.10+
|
||
- `ffmpeg` (for image resize)
|
||
- RouterAI API key (see Step 1)
|
||
- (optional) Local Jan server for `--local` mode
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
# Clone the repo
|
||
git clone https://git.hidosi.ru/Hidosi/agent-scripts-vision-analyze.git
|
||
cd agent-scripts-vision-analyze
|
||
|
||
# Test without API key (will fail on first request, that's expected)
|
||
python3 vision-analyze.py --help
|
||
|
||
# Symlink for easy access (optional)
|
||
chmod +x vision-analyze.py
|
||
ln -s "$(pwd)/vision-analyze.py" /usr/local/bin/vision-analyze
|
||
```
|
||
|
||
## Usage
|
||
|
||
```bash
|
||
python3 vision-analyze.py <image_path>
|
||
```
|
||
|
||
### Arguments
|
||
|
||
| Flag | Description |
|
||
|---|---|
|
||
| *(none)* | RouterAI gpt-4o-mini, default prompt — **requires API key** |
|
||
| `--local` | Local Gemma (private, no censorship) — **requires local server** |
|
||
| `--model=X` | RouterAI model override |
|
||
| `--prompt="..."` | Custom prompt |
|
||
| `--prompt="🌶 ..."` | Emoji triggers auto-routing to local Gemma |
|
||
|
||
### Default Prompt
|
||
|
||
```
|
||
Дай последовательное описание композиции: передний план, задний план,
|
||
элементы на изображении, люди или персонажи. Пиши живым, человеческим
|
||
языком, 3-5 предложений.
|
||
```
|
||
|
||
### Routing Rules
|
||
|
||
1. **No flags** → RouterAI gpt-4o-mini (fast, ~2-10s, cheap)
|
||
2. `--local` → Local Gemma (~30-90s, free, no filters)
|
||
3. `--model=X` → RouterAI with model X
|
||
4. `🌶️` / `🔞` / `🤫` in `--prompt` → Local Gemma (NSFW never touches RouterAI!)
|
||
5. **No auto-fallback!** Error → log & message, never silently route to other backend
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Backend (ABC)
|
||
├── RouterAIBackend — RouterAI API via urllib, HTTPError/URLError handling
|
||
└── LocalGemmaBackend — Anthropic /messages, ProxyHandler({}) bypass, 1 retry
|
||
|
||
ImagePreprocessor — ffmpeg resize, tempfile, finally cleanup
|
||
```
|
||
|
||
### Safety
|
||
|
||
- API keys never written to /tmp world-readable
|
||
- No `curl`, no `bash -c`, no shell injection vectors
|
||
- `ProxyHandler({})` for local requests (bypasses V2Ray)
|
||
- Emoji routing ensures NSFW content never reaches cloud APIs
|
||
|
||
## Known Limitations
|
||
|
||
- **Local Gemma:** Only one concurrent request. Never interrupt/kill — causes hang.
|
||
- **Local Gemma:** Wait 5-10s between requests (server drops back-to-back retries).
|
||
- **Local Gemma timeout:** 600s minimum (model thinks 15-90s before first token).
|
||
- DeepSeek models cannot see images — do not pass base64 images to them.
|
||
- Without API key: only `--local` mode works.
|
||
- Without local server: only RouterAI mode works.
|
||
|
||
## For Other Agents — Setup Integration Checklist
|
||
|
||
When integrating this skill, do the following steps before using:
|
||
|
||
```python
|
||
def setup_vision_analyze(config: dict) -> bool:
|
||
\"\"\"Returns True if at least one backend is available.\"\"\"
|
||
routerai_ok = False
|
||
gemma_ok = False
|
||
|
||
# 1. RouterAI API key
|
||
try:
|
||
key = config["models"]["providers"]["routerai-ru"]["apiKey"]
|
||
if key and len(key) > 8:
|
||
routerai_ok = True
|
||
except (KeyError, TypeError):
|
||
routerai_ok = False
|
||
|
||
if not routerai_ok:
|
||
key = ask_user("🔑 API ключ RouterAI (https://routerai.ru)?")
|
||
if key:
|
||
config.setdefault("models", {}).setdefault("providers", {})
|
||
config["models"]["providers"]["routerai-ru"] = {"apiKey": key}
|
||
routerai_ok = True
|
||
|
||
# 2. Local Gemma (optional)
|
||
has_local = ask_user("💻 Локальная Gemma (Jan/llama.cpp)? (y/n)")
|
||
if has_local.lower() in ("y", "yes", "да"):
|
||
config.setdefault("models", {}).setdefault("providers", {})
|
||
gemma_key = ask_user("🔑 API ключ Gemma (Enter если не нужен):")
|
||
config["models"]["providers"]["jan-local"] = {"apiKey": gemma_key or ""}
|
||
gemma_ok = True
|
||
|
||
if not routerai_ok and not gemma_ok:
|
||
logger.warning("⚠ Ни один бэкенд не настроен! vision-analyze не будет работать.")
|
||
ask_user("⚠ Настрой позже или удали скилл.")
|
||
return False
|
||
|
||
logger.info(f"✅ Vision Analyze: RouterAI={'✓' if routerai_ok else '✗'}, Gemma={'✓' if gemma_ok else '✗'}")
|
||
return True
|
||
```
|
||
|
||
---
|
||
|
||
Created 2026-07-06 by Roko (OpenClaw agent) with OpenCode (kimi-k2.7-code). Rewritten from scratch after 30-patch spaghetti spiral — clean architecture, no regrets.
|