I am writing a proxy server. After I use curl to fetch a webpage, say http://world wide web.foo.com/pants, curl helps make the following request:
GET /pants HTTP/1.1
After I have curl send that request through my local proxy, curl changes the GET request to:
GET http://world wide web.foo.com/pants HTTP/1.1
This transformation causes the foo.com server return a 404. Is foo.com damaged? Or perhaps is the properly accredited domain title only significant to proxy servers? Must I always strip http://domain in the demands I send?
Because this proxy needs to result in the upstream link with the server, curl (along with other compliant user agents) transmits the entire URL.
Your proxy needs to strip the hostname, resolve it to IP, connect upstream after which send
GET /pants HTTP/1.1.
Whenever a client transmits a request with an HTTP-based proxy, it needs to specify the entire URL since the proxy may support multiple target methods that may be symbolized with Web addresses (ie: HTTP, FTP, etc). It's the proxy's responsibility to massage the URL and ask for data right into a more appropriate format when passing it onto the following server.
Estimating from rfc 2616, section 5.1.2 inchAsk for-URI", emphasis by me:
The absoluteURI form is Needed once the request has been designed to a proxy. The proxy is asked for to forward the request or service it from the valid cache, and return the response. Observe that the proxy MAY forward the request onto another proxy or straight to the server per the absoluteURI. To be able to avoid request loops, a proxy MUST have the ability to recognize all its server names, including any aliases, local versions, and also the number Ip. A good example Request-Line could be:
GET http://world wide web.w3.org/pub/World wide web/TheProject.html HTTP/1.1
To permit transition to absoluteURIs in most demands later on versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI form in demands, despite the fact that HTTP/1.1 clients is only going to generate them in demands to proxies.