ARK Core uses WebSockets for P2P communication because it scales well for large networks, at least compared to traditional TCP. Depending on how you make use of WebSockets it can reduce the traffic on a network by a significant amount because data is being pushed between peers when it becomes available rather than having to pull periodically to see if there is any new data to download and process.
Before continuing - WebSockets are better for scaling. It was a conscious decision to pick traditional HTTP communication for Odysseia to keep the architecture simple to support reasonably sized networks. If scaling issues would be hit then a WebSocket plugin could easily be swapped in without much complications.
Odysseia chose Fastify because it is a fast and low overhead HTTP server that comes without a lot of bells and whistles out of the box but is highly extensible through a plugin system that allows you to hook into various parts of the request cycle to modify input and output along the way. This plays well with the goals of ARK Core and Odysseia - being a highly extensible blockchain framework.
Fastify powers multiple different servers that are run as part of the P2P communication layer. The main use-case is of course the gossiping server which is responsible for ensuring that all nodes on the network get the information they need to maintain a full copy of the blockchain.
The validator server is responsible for exposing everything that a validator needs to forge new blocks. This ranges from triggering a synchronisation with the network to getting a list of transactions to include in an upcoming block to proposing a new block to other validators.
The operations server is responsible for accepting operation requests for things like replacing or removing pending transactions. Its requirements are fairly simple because it only takes inputs and executes and action based on those and finally responding with an HTTP status code. Rate-limiting is the most important part for this server because too many operations could cause the node to stall while the event loop is busy processing previous requests that are still executing operations.
Accessibility & Compression
Both of these servers have different needs in terms of external accessibility, rate limiting and compression to prevent becoming unresponsive and thus not being able to participant in the network anymore.
The gossiping server for example needs high compression to reduce traffic for public network requests and rate-limiting to avoid being flooded with requests which could result in becoming unresponsive. Many nodes becoming unresponsive can slow down the network because it takes longer to relay new information to all nodes that should be informed about it.
The validator server doesn't have such strict requirements because it will only ever be used for local requests. There is no need for high levels of compression or rate-limiting because we are in control of how many requests are being sent and what kind of data is being responded with.
The extensibility of Fastify makes it easy to spin up those dedicated servers without much hassle and allow plugin developers to hook into those servers before they are launched.