I am wondering about how public chats can be saved and archived for future reference by developers? Let's say I was a developer for a game with a public chat system and I wanted to make sure I could look up and read a chat from ten years ago?

  • \$\begingroup\$ Chat messages go through your server and get saved somehow (SQL database, logs, etc). What seems to be the problem? \$\endgroup\$ – Kromster Mar 9 '17 at 18:20

The actual archiving is the same as with any other data. Find a format to serialize it in, convert it to that format, then write it to disk. This could be just writing it out as text, or writing it out in some structured way. A few things to keep in mind:

  1. Ensure you have unique user IDs or a corresponding backup of account data to go with the chat log, so you can associate a message with whoever posted it, even across a user renaming their account or changing their e-mail address.

  2. If you need to keep statistics, maybe record them when you write out the data, so you don't need to recalculate them for the whole archive every time.

  3. Familiarize yourself with the data protection, data retention and required disclosure laws of the countries your game is used in. In particular, keep in mind that even in-game chat may contain information that could be security relevant (even if it's just someone accidentally posting their password).

    So make sure you encrypt your logs appropriately and restrict access to them. The easiest way to avoid data leaks is usually to not keep data very long. I.e. keep data for a week or so (whatever is enough time for you to access abuse/fraud/harassment reports), and then get rid of it if it is not flagged as involved in a dispute. Alternately, anonymize your data after a while so it is still usable for your statistics, but doesn't contain any e-mail addresses or user IDs anymore.

  4. Make backups of data you need to keep. It's easy to lose a server or whatever.

  • \$\begingroup\$ Comments are not for extended discussion; this conversation has been moved to chat. \$\endgroup\$ – user1430 Mar 14 '17 at 14:12

Simply dump the chat records to a text file and archive them like you would any other important piece of system data. This probably means rotating and redundant backups of decreasing temperature - maybe the last month of logs stays resident in a SQL database (that is backed up), but the logs are eventually dumped to text or some other format and then archived to tape which is "cold stored" in an off-site location.

Text is the most-basic form the chat logs can be archived in, although if you want to retain the ability to perform complex queries on them it may be better to keep them in a structured, database-like format (perhaps going as far as storing them in a separate actual SQL database).

Your strategy will ultimately depend on the scale of the data (how much chat volume you get on an average daily basis) and how much you actually care about being able to go back and cherry-pick a random conversation from a decade ago at a moment's notice. Realistically speaking I doubt you'll need to do that on-demand very often.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.