Set meta["proxy"] on Scrapy requests with console credentials (URL-encode them). This is not the same as a one-off requests call. Do not treat crawl concurrency as a product rate limit. Not for mainland China.
Copy protocol, host, port, and auth from the console, then put them on Scrapy requests. A common pattern is a full proxy URL in `request.meta["proxy"]`, which the default `HttpProxyMiddleware` honors. That is not the same as one `requests.get(..., proxies=...)`: a crawl issues many requests, and exit cadence follows the rotating or static product you bought—not a Scrapy setting that “turns into static.” One-off Python calls: Python residential proxy.
ZTProxy sells rotating residential (per GB) and static residential (per IP·month). Examples use fake placeholders—not production hosts. Authorized targets only. Not for mainland China.
What to have ready
| Ready item | Source |
|---|---|
| Protocol, host, port, auth | Console values only (not the fake placeholders) |
| Python and Scrapy | Your local environment |
| Check URL | A page you may access |
| Rotating or static order | Rotating vs static |
Placeholders (not production): host `proxy.example.invalid`, port `10000`, user `USER_EXAMPLE`, password `PASS_EXAMPLE_NOT_REAL`. Strings: proxy format. Protocol: HTTP / SOCKS5 setup. Commands follow Docs EN quickstart. Confirm the same credentials with curl before starting a crawl.
Recommended pattern: meta["proxy"]
URL-encode the username or password if it contains `@` or `:`. Do not invent country or session fields in the username. Do not assume you must install a “rotating middleware” unless your own scheduling needs it and you understand how it interacts with the product you ordered.
import os
from urllib.parse import quote
import scrapy
user = quote(os.environ.get("ZT_PROXY_USER", "USER_EXAMPLE"), safe="")
password = quote(os.environ.get("ZT_PROXY_PASS", "PASS_EXAMPLE_NOT_REAL"), safe="")
host = os.environ.get("ZT_PROXY_HOST", "proxy.example.invalid")
port = os.environ.get("ZT_PROXY_PORT", "10000")
PROXY = f"http://{user}:{password}@{host}:{port}"
class AuthorizedCheckSpider(scrapy.Spider):
name = "authorized_check"
def start_requests(self):
yield scrapy.Request(
"https://example.com",
meta={"proxy": PROXY},
callback=self.parse,
)
def parse(self, response):
self.logger.info("status=%s", response.status)`CONCURRENT_REQUESTS` and `DOWNLOAD_DELAY` are client pacing. They reduce pressure on the target and help you avoid turning retries into extra billed traffic. They are not a ZTProxy rate limit or SLA. Billing units: GB vs IP. Use SOCKS5 only when the console returns SOCKS and your downloader can speak SOCKS. Do not assume UDP is available.
Sessions and common issues
On rotating residential, later requests may see different exits; that is not automatically a failure. For a short-lived same exit, use sticky fields from the console; for a term-long address, use static. See sticky ≠ static.
| Symptom | Check first |
|---|---|
| 407 | Username/password, URL encoding, missing `meta["proxy"]`; order still valid |
| Some requests bypass the proxy | Proxy set only on the first Request; later `follow` calls dropped meta |
| Timeouts or unusually high traffic | Retries repeatedly downloading failures; recheck credentials with curl first |
| Wrong region | The order region; do not invent username syntax |
Fuller symptom table: 407 and timeouts.
Compliance and scope
Collect only public or authorized targets you may access, and respect site rules. This is not a bypass tutorial. Prices follow the console and pricing (rotating about $0.40/GB, about $0.20/GB on a 100 TB full bundle; static about $1.50 shared and about $5 dedicated per IP/month). USDT/USDC available; Visa/Mastercard in application. Not for mainland China.
FAQ
How is this different from the requests page?
That page is a single HTTP call. This page is about putting the proxy on every Scrapy Request and not dropping meta on follow-up requests.
Do I have to enable HttpProxyMiddleware myself?
It is usually on by default. Check that a custom middleware has not disabled or overridden it.
Can I use the example host as-is?
No. Replace it with console values.
If I raise CONCURRENT_REQUESTS, is that a higher plan?
No. That is only client concurrency. Plans and billing follow the console. See GB vs IP.
Must I install a third-party rotating plugin?
No. Start with the console product (rotating or static) and the credentials it issues. Do not stack an unverified middleware just to look like rotation.
Can I use this to break a login wall?
No. Authorized targets only.
If I reuse the same proxy URL, do I keep the same IP?
Not necessarily, especially on rotating residential. Reusing a URL is not static.
Where are prices confirmed?
In the console and on pricing.