Troubleshooting: Upstream Timeouts & Sluggish Server Targets
How to resolve 504 Gateway Timeouts, slow government and municipal portals, bot protection challenges, and configure exponential retry buffers.
Certain enterprise targets—particularly municipal procurement sites, patent registries, and legacy wholesale supplier databases—can take 15 to 30 seconds to respond, or experience intermittent connection drops during peak business hours.
When a target server fails to return data before the distributed runtime threshold, Tishtar records an UPSTREAM_TIMEOUT or GATEWAY_TIMEOUT (504).
Here is how to optimize your routine to handle sluggish, unreliable target servers.
3 Strategies for Sluggish Websites
┌─────────────────────────────────┐ ┌───────────────────────────────────┐
│ Strategy │ ──► │ Configuration in Tishtar Studio │
├─────────────────────────────────┤ ├───────────────────────────────────┤
│ 1. Exponential Retry Buffer │ ──► │ Max Retries: 3 / Backoff: 5m │
│ 2. Off-Peak Schedule Shift │ ──► │ Move run from 09:00 AM to 05:30 AM│
│ 3. Target Specific Section URL │ ──► │ Bypass heavy uncompressed images │
└─────────────────────────────────┘ └───────────────────────────────────┘1. Enable Automatic Retries with Exponential Backoff
If an upstream server is momentarily overwhelmed, immediately hammering it again will only trigger rate-limiting.
- Open your routine in Tishtar Studio.
- Click Advanced Settings -> Retry Policy.
- Set Max Retries to
3. - Set Retry Strategy to
Exponential Backoff (5 min, 15 min, 30 min). - If the first fetch times out, Tishtar quietly re-attempts execution after the target server recovers, without counting as a failed run.
2. Shift Execution to Off-Peak Hours
Government portals and tender gazettes often slow down between 09:00 AM and 11:00 AM as thousands of civil servants and citizens log into their systems simultaneously.
- Pro Tip: Shift your routine’s schedule to 05:30 AM or 06:00 AM. Server load is minimal, response times are 5x faster, and your intelligence digest is still ready before your workday begins.
3. Bypass Heavy Media & Target Specific HTML Endpoints
If a competitor product page is bloated with 10 megabytes of unoptimized hero video and product images:
- Target their lighter, text-only sub-pages (e.g.
example.com/item/specificationsrather than the heavy media-rich gallery page). - Point your routine to their RSS feed, sitemap index, or public API endpoint if available.