← Back to Search & Research
Search & Research by @robbyczgw-cla

web-search-plus

Unified search skill with Intelligent Auto-Routing

0
Source Code

Web Search Plus

Stop choosing search providers. Let the skill do it for you.

This skill connects you to 6 search providers (Serper, Tavily, Exa, Perplexity, You.com, SearXNG) and automatically picks the best one for each query. Shopping question? → Google results. Research question? → Deep research engine. Need a direct answer? → AI-synthesized with citations. Want privacy? → Self-hosted option.

✨ What Makes This Different?

  • Just search — No need to think about which provider to use
  • Smart routing — Analyzes your query and picks the best provider automatically
  • 6 providers, 1 interface — Google results, research engines, neural search, AI answers with citations, RAG-optimized, and privacy-first all in one
  • Works with just 1 key — Start with any single provider, add more later
  • Free options available — SearXNG is completely free (self-hosted)

🚀 Quick Start

# Interactive setup (recommended for first run)
python3 scripts/setup.py

# Or manual: copy config and add your keys
cp config.example.json config.json

The wizard explains each provider, collects API keys, and configures defaults.

🔑 API Keys

You only need ONE key to get started. Add more providers later for better coverage.

Provider Free Tier Best For Sign Up
Serper 2,500/mo Shopping, prices, local, news serper.dev
Tavily 1,000/mo Research, explanations, academic tavily.com
Exa 1,000/mo "Similar to X", startups, papers exa.ai
Perplexity Via Kilo Direct answers with citations kilo.ai
You.com Limited Real-time info, AI/RAG context api.you.com
SearXNG FREE ✅ Privacy, multi-source, $0 cost Self-hosted

Setting your keys:

# Option A: .env file (recommended)
export SERPER_API_KEY="your-key"
export TAVILY_API_KEY="your-key"

# Option B: config.json
{ "serper": { "api_key": "your-key" } }

🎯 When to Use Which Provider

I want to... Provider Example Query
Find product prices Serper "iPhone 16 Pro Max price"
Find restaurants/stores nearby Serper "best pizza near me"
Understand how something works Tavily "how does HTTPS encryption work"
Do deep research Tavily "climate change research 2024"
Find companies like X Exa "startups similar to Notion"
Find research papers Exa "transformer architecture papers"
Get a direct answer with sources Perplexity "events in Berlin this weekend"
Know the current status of something Perplexity "what is the status of Ethereum upgrades"
Get real-time info You.com "latest AI regulation news"
Search without being tracked SearXNG anything, privately

Pro tip: Just search normally! Auto-routing handles most queries correctly. Override with -p provider when needed.

🧠 How Auto-Routing Works

The skill looks at your query and picks the best provider:

"iPhone 16 price"              → Serper (shopping keywords)
"how does quantum computing work" → Tavily (research question)
"companies like stripe.com"    → Exa (URL detected, similarity)
"events in Graz this weekend"  → Perplexity (local + direct answer)
"latest news on AI"            → You.com (real-time intent)
"search privately"             → SearXNG (privacy keywords)

What if it picks wrong? Override it: python3 scripts/search.py -p tavily -q "your query"

Debug routing: python3 scripts/search.py --explain-routing -q "your query"

📖 Usage Examples

Let Auto-Routing Choose (Recommended)

python3 scripts/search.py -q "Tesla Model 3 price"
python3 scripts/search.py -q "explain machine learning"
python3 scripts/search.py -q "startups like Figma"

Force a Specific Provider

python3 scripts/search.py -p serper -q "weather Berlin"
python3 scripts/search.py -p tavily -q "quantum computing" --depth advanced
python3 scripts/search.py -p exa --similar-url "https://stripe.com" --category company
python3 scripts/search.py -p you -q "breaking tech news" --include-news
python3 scripts/search.py -p searxng -q "linux distros" --engines "google,bing"

⚙ Configuration

{
  "auto_routing": {
    "enabled": true,
    "fallback_provider": "serper",
    "confidence_threshold": 0.3,
    "disabled_providers": []
  },
  "serper": {"country": "us", "language": "en"},
  "tavily": {"depth": "advanced"},
  "exa": {"type": "neural"},
  "you": {"country": "US", "include_news": true},
  "searxng": {"instance_url": "https://your-instance.example.com"}
}

📊 Provider Comparison

Feature Serper Tavily Exa Perplexity You.com SearXNG
Speed ⚡⚡⚡ ⚡⚡ ⚡⚡ ⚡⚡ ⚡⚡⚡ ⚡⚡
Direct Answers ✗ ✗ ✗ ✓✓ ✗ ✗
Citations ✗ ✗ ✗ ✓ ✗ ✗
Factual Accuracy ⭐⭐⭐ ⭐⭐⭐ ⭐⭐ ⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐
Semantic Understanding ⭐ ⭐⭐ ⭐⭐⭐ ⭐⭐⭐ ⭐⭐ ⭐
Full Page Content ✗ ✓ ✓ ✓ ✓ ✗
Shopping/Local ✓ ✗ ✗ ✗ ✗ ✓
Find Similar Pages ✗ ✗ ✓ ✗ ✗ ✗
RAG-Optimized ✗ ✓ ✗ ✗ ✓✓ ✗
Privacy-First ✗ ✗ ✗ ✗ ✗ ✓✓
API Cost $$ $$ $$ Via Kilo $ FREE

❓ Common Questions

Do I need API keys for all providers?

No. You only need keys for providers you want to use. Start with one (Serper recommended), add more later.

Which provider should I start with?

Serper — fastest, cheapest, largest free tier (2,500 queries/month), and handles most queries well.

What if I run out of free queries?

The skill automatically falls back to your other configured providers. Or switch to SearXNG (unlimited, self-hosted).

How much does this cost?

  • Free tiers: 2,500 (Serper) + 1,000 (Tavily) + 1,000 (Exa) = 4,500+ free searches/month
  • SearXNG: Completely free (just ~$5/mo if you self-host on a VPS)
  • Paid plans: Start around $10-50/month depending on provider

Is SearXNG really private?

Yes, if self-hosted. You control the server, no tracking, no profiling. Public instances depend on the operator's policy.

How do I set up SearXNG?

# Docker (5 minutes)
docker run -d -p 8080:8080 searxng/searxng

Then enable JSON API in settings.yml. See docs.searxng.org.

Why did it route my query to the "wrong" provider?

Sometimes queries are ambiguous. Use --explain-routing to see why, then override with -p provider if needed.

🔄 Automatic Fallback

If one provider fails (rate limit, timeout, error), the skill automatically tries the next provider. You'll see routing.fallback_used: true in the response when this happens.

📤 Output Format

{
  "provider": "serper",
  "query": "iPhone 16 price",
  "results": [{"title": "...", "url": "...", "snippet": "...", "score": 0.95}],
  "routing": {
    "auto_routed": true,
    "provider": "serper",
    "confidence": 0.78,
    "confidence_level": "high"
  }
}

⚠ Important Note

Tavily, Serper, and Exa are NOT core OpenClaw providers.

❌ Don't modify ~/.openclaw/openclaw.json for these
✅ Use this skill's scripts — keys auto-load from .env

🔒 Security

SearXNG SSRF Protection: The SearXNG instance URL is validated with defense-in-depth:

  • Enforces http/https schemes only
  • Blocks cloud metadata endpoints (169.254.169.254, metadata.google.internal)
  • Resolves hostnames and blocks private/internal IPs (loopback, RFC1918, link-local, reserved)
  • Operators who intentionally self-host on private networks can set SEARXNG_ALLOW_PRIVATE=1

📚 More Documentation

🔗 Quick Links