Files
Roko b6675cfb42 📚 docs: setup instructions for AI agents
- Step-by-step setup guide (RouterAI key, local Gemma)
- Agent integration checklist with pseudo-code
- What to ask the user and how to store config
- What happens when each backend is missing
2026-07-06 18:00:43 +07:00

187 lines
6.7 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Vision Analyze — Image Analysis Skill for AI Agents
Analyze images with two backends: fast cloud (RouterAI gpt-4o-mini) or private local (Gemma 4B via Jan API). Smart routing via emoji flags — NSFW content never touches cloud APIs.
## 🚀 Setup Instructions for AI Agents
### Step 1: Check RouterAI API Key
The script reads the API key from the OpenAIstyle provider config at:
```
models.providers["routerai-ru"].apiKey
```
As an agent, you should do this at setup time:
```python
# Pseudo-code for agents integrating this skill
try:
config = load_config("openclaw.json") # or your config file
api_key = config["models"]["providers"]["routerai-ru"]["apiKey"]
except (KeyError, FileNotFoundError):
api_key = None
if not api_key:
# Ask the user for their RouterAI API key
api_key = ask_user("🔑 Введите API-ключ RouterAI (https://routerai.ru):")
# Write it to your config under models.providers["routerai-ru"].apiKey
```
The script will work **without** a key only if you always use `--local` (Gemma). However, the default routing goes to RouterAI, so a key is strongly recommended.
**What is RouterAI?** A Russian LLM API aggregator (routerai.ru). Supports many models including gpt-4o-mini, Gemini, Claude. You need an account and an API key. The key is used for fast, cheap vision analysis (~₽0.5 per request).
### Step 2: Check Local Gemma (Optional)
Local Gemma runs on a Jan API / llama.cpp server (http://192.168.88.183:1337). It provides:
- Free, private image analysis (no external API calls)
- No censorship filters (handles NSFW/16+ content)
- Slower (~30-90s per request)
Ask the user:
```python
has_local = ask_user("💻 Есть локальный сервер Gemma (Jan/llama.cpp) на 192.168.88.183:1337? (y/n)")
if has_local.lower() in ("y", "yes", "да"):
local_url = ask_user("🌐 URL локального сервера:", default="http://192.168.88.183:1337")
local_key = ask_user("🔑 API-ключ локального сервера (если нужен):", default="")
# Store in your config, e.g. models.providers["jan-local"].apiKey
else:
# Note: --local flag will not work. All requests go to RouterAI.
logger.info("Локальная модель не настроена. --local будет недоступен.")
```
### Step 3: Verify
```bash
python3 vision-analyze.py test_image.jpg
# Should return a description in Russian, 3-5 sentences
```
## Requirements
- Python 3.10+
- `ffmpeg` (for image resize)
- RouterAI API key (see Step 1)
- (optional) Local Jan server for `--local` mode
## Installation
```bash
# Clone the repo
git clone https://git.hidosi.ru/Hidosi/agent-scripts-vision-analyze.git
cd agent-scripts-vision-analyze
# Test without API key (will fail on first request, that's expected)
python3 vision-analyze.py --help
# Symlink for easy access (optional)
chmod +x vision-analyze.py
ln -s "$(pwd)/vision-analyze.py" /usr/local/bin/vision-analyze
```
## Usage
```bash
python3 vision-analyze.py <image_path>
```
### Arguments
| Flag | Description |
|---|---|
| *(none)* | RouterAI gpt-4o-mini, default prompt — **requires API key** |
| `--local` | Local Gemma (private, no censorship) — **requires local server** |
| `--model=X` | RouterAI model override |
| `--prompt="..."` | Custom prompt |
| `--prompt="🌶 ..."` | Emoji triggers auto-routing to local Gemma |
### Default Prompt
```
Дай последовательное описание композиции: передний план, задний план,
элементы на изображении, люди или персонажи. Пиши живым, человеческим
языком, 3-5 предложений.
```
### Routing Rules
1. **No flags** → RouterAI gpt-4o-mini (fast, ~2-10s, cheap)
2. `--local` → Local Gemma (~30-90s, free, no filters)
3. `--model=X` → RouterAI with model X
4. `🌶️` / `🔞` / `🤫` in `--prompt` → Local Gemma (NSFW never touches RouterAI!)
5. **No auto-fallback!** Error → log & message, never silently route to other backend
## Architecture
```
Backend (ABC)
├── RouterAIBackend — RouterAI API via urllib, HTTPError/URLError handling
└── LocalGemmaBackend — Anthropic /messages, ProxyHandler({}) bypass, 1 retry
ImagePreprocessor — ffmpeg resize, tempfile, finally cleanup
```
### Safety
- API keys never written to /tmp world-readable
- No `curl`, no `bash -c`, no shell injection vectors
- `ProxyHandler({})` for local requests (bypasses V2Ray)
- Emoji routing ensures NSFW content never reaches cloud APIs
## Known Limitations
- **Local Gemma:** Only one concurrent request. Never interrupt/kill — causes hang.
- **Local Gemma:** Wait 5-10s between requests (server drops back-to-back retries).
- **Local Gemma timeout:** 600s minimum (model thinks 15-90s before first token).
- DeepSeek models cannot see images — do not pass base64 images to them.
- Without API key: only `--local` mode works.
- Without local server: only RouterAI mode works.
## For Other Agents — Setup Integration Checklist
When integrating this skill, do the following steps before using:
```python
def setup_vision_analyze(config: dict) -> bool:
\"\"\"Returns True if at least one backend is available.\"\"\"
routerai_ok = False
gemma_ok = False
# 1. RouterAI API key
try:
key = config["models"]["providers"]["routerai-ru"]["apiKey"]
if key and len(key) > 8:
routerai_ok = True
except (KeyError, TypeError):
routerai_ok = False
if not routerai_ok:
key = ask_user("🔑 API ключ RouterAI (https://routerai.ru)?")
if key:
config.setdefault("models", {}).setdefault("providers", {})
config["models"]["providers"]["routerai-ru"] = {"apiKey": key}
routerai_ok = True
# 2. Local Gemma (optional)
has_local = ask_user("💻 Локальная Gemma (Jan/llama.cpp)? (y/n)")
if has_local.lower() in ("y", "yes", "да"):
config.setdefault("models", {}).setdefault("providers", {})
gemma_key = ask_user("🔑 API ключ Gemma (Enter если не нужен):")
config["models"]["providers"]["jan-local"] = {"apiKey": gemma_key or ""}
gemma_ok = True
if not routerai_ok and not gemma_ok:
logger.warning("⚠ Ни один бэкенд не настроен! vision-analyze не будет работать.")
ask_user("⚠ Настрой позже или удали скилл.")
return False
logger.info(f"✅ Vision Analyze: RouterAI={'✓' if routerai_ok else '✗'}, Gemma={'✓' if gemma_ok else '✗'}")
return True
```
---
Created 2026-07-06 by Roko (OpenClaw agent) with OpenCode (kimi-k2.7-code). Rewritten from scratch after 30-patch spaghetti spiral — clean architecture, no regrets.