Databases are supposed to be used for persisting long-lived data, not for ephemeral data. Using a database for storing short-lived chat messages just to delete them seconds later is quite an anti-pattern. It was common in the early years of PHP where databases were pretty much the only way to exchange data between sessions. But more modern web application frameworks have ways to communicate between sessions using in-memory techniques without having to abuse a database.
However, when your architecture supports offline messes which get delivered as soon as the player goes back online, then you are not dealing with ephemeral data. It's data which can last for days, probably forever. That means it belongs into a database so it doesn't hog RAM and doesn't get lost in case of a server reboot.
So my recommendation would be to conceptually separate chat messages from offline notifications. The first are ephemeral. They just get sent to all receivers and then the server forgets about them. The latter are stored in the database until the recipient connects to the server. Deciding what's a chat message and what's an offline notification would be up to the server. If the recipient is online, it's always a chat message. If the recipient is offline, the server should decide whether it's important enough to turn into an offline notification or if it should just be discarded.
On the other hand, there is one argument for storing all chat messages permanently in the database: You want to log them. That can be useful if you want to investigate reports of abusive behavior. But the question is if it is worth the cost. Large communities can generate a lot of pointless chatter. Is it really worth it to store all of that forever? And then there is the legal problem of data protection. Are you actually allowed to permanently store all that communication, probably some of which players consider confidential? Or maybe you are even required to log communication in case law enforcement wants access? That's a question for your local lawyer.