[p2-core] WebSocket Gateway compression

[bitfl0wer]

To have zstd compression be usable and flexible in polyproto, a handshake negotiating compression parameters needs to be documented.

This sort of hit me "last minute" in the beta.1 release cycle, since I have never worked with compression in this way before, and assumed it'd just "magically" work. :P

[MathMan05]

I know with discord/spacebar it's just something included in the paremiters and if the server doesn't want to the server may just ignore it.

[bitfl0wer]

Interesting! I had in mind the selecting the highest possible settings which both parties (want to) support, then using those for further gateway communications. Unsure yet though, as I have yet to look into how to best do this.

[MathMan05]

could just have it in the perimeter, or have the server advertise what it supports in the request to get the URL for the socket and have the client choose one

[bitfl0wer]

It will likely be something like that.
I imagine the Gateway would include supported encoding parameters in the Hello payload, which the client would parse and use to send the parameters it chose as part of the Identify/Resume payload

[MathMan05]

I meant why not just have it not be in the gatway at all, instead when getting the URL of the gateway from /gateway so it doesn't complicate the gateway itself

[bitfl0wer]

Makes the gateway less flexible. Especially when load peaks are higher, gateway servers would be able to reduce their compression parameters to save CPU time, and could increase compression parameters at lower load times. Especially when one would like to deploy multiple gateway servers globally, the parameters might not always be the best fit for all deployments. Making it part of the hello payload is a better idea imo

[MathMan05]

Like how exactly though? Like turning it on/off or reducing how much they tell the libary to compress things? If it's the first, then clients can handle that, or at least I know jank does, it just figures out the type of the stream and decodes that, if it's the second, there's no need to signal that.

[bitfl0wer]

zstd requires parameters to be known before attempting to decompress a payload. As far as I know, they cannot be determined dynamically.

[MathMan05]

couldn't one just adjust the compression levels when it starts or something, cause it'd be the effort, unless I'm wrong, then disregard this. Though if it were an issue, you could just fall back to uncompressed

[bitfl0wer]

I think restarting the server to set different compression levels is more effort than implementing this. You just receive parameters in the Hello event, parse them and apply them; it is not that hard

[MathMan05]

It's just kinda wonky, where the first few messages aren't compressed, like not a huge deal, but it's still an odd thing to handle instead of just saying what you want. (you could also make an event if the parameters are wrong that it sends uncompressed json telling the client what to reconnect with or something)

[MathMan05]

wait, would this only be fore zstd, the title makes it sound like its for in general

[bitfl0wer]

I don't really see any tangible benefits from your suggested approach, but thank you for the suggestion!