v0.28 plusses for spaces causing errors and _urlparse suddenly being ignored

[shenanigansd]

Good morning, I have requests like this:

httpx.request("https://impressdc/fmi/odata/v4/Data_Orders/Order", params={
  "$filter": f"ID_Order eq {order_number}",
  "$select": "ExtSource,ExtOrderID,ID_Order",
})

And I use this to keep the special characters intact:

from httpx import _urlparse as httpx_urlparse 
httpx_urlparse.UNRESERVED_CHARACTERS += "$,"

After upgrading from 0.27.2 to 0.28.1, the resulting URLs have changed:

- https://impressdc/fmi/odata/v4/Data_Orders/Order?$filter=ID_Order%20eq%20341589&$select=ExtSource,ExtOrderID,ID_Order
+ https://impressdc/fmi/odata/v4/Data_Orders/Order?%24filter=ID_Order+eq+341589&%24select=ExtSource%2CExtOrderID%2CID_Order

This is causing the remote to respond with "400: + is not a valid character"

I see #3371 and #3373 linked to in #3394 talking about the spaces, though I'm not sure about why _urlparse is suddenly being ignored.
How can I restore the previous behavior?

[tomchristie]

Thanks @shenanigansd, really good question.

• _urlparse is not public API and direct usage is not currently supported.
• Using the complete URL string, or httpx.URL("https://www.example.com/", query=b"test=$%24 %20+")^* is a good approach if you need to be precise about what escaping style you want. This will only percent encode the query portion where spec-required. (Using params={...} will always give you HTML form-like behavior. There's some docs work to be done here.)

---

^* Actually I'm unkeen on having chosen bytes for that interface, but there's a more involved discussion there to be had at a later date.

[aw-was-here]

Is there a reason this isn't just a flag to flip? We're actually debating monkeypatching httpx (or maybe even just replacing it with something else) because we're using an API that actually expects spaces to be percent encoded. Keeping it as bytes seems like an odd solution to the problem.

[tomchristie]

Is there a reason this isn't just a flag to flip?

Yes.

It's the same reason your internet browser doesn't have a flag to flip how it urlencodes your form submissions. That's the behaviour we have when you use params=...

You can include the URL querystring directly if you need more control here.

[aw-was-here]

Most do encode outside of form submissions however, as was pointed out in #2460 and #2536 .

[tomchristie]

😌👍