Use sqlite. Really. I know it's more work than just using a serializer, but that's the way all normal android apps do it, and its the right way to do it, as I'll explain below.
See https://github.com/codecoding/SQLite4Unity3d for how to do that.
If that's too much work, checkout out custom serialization using https://github.com/jacobdufault/fullserializer
The format that you save your data in is an orthogonal concern to the way in which you save large data sets.
There are two issues here:
When I save a record, I need to be able to restore that record, even if the data model has potentially changed in code compared to when I saved the record.
I need to save a large number of records, and it's too slow.
Lets address these problems one by one:
Deserializing when the model has changed
An XML/json serializer will solve this problem to some extent, but ultimately, if your data model changes you need to use a migration to migrate old data to the new schema.
There is no way to avoid this problem.
Using a weakly typed serializer (eg. json) will allow you to receive null values for missing keys, but ultimately, if a field changes type, you'll still have broken data.
Don't ever use
The standard solution to this problem is to use a database, and migrate data on a per-table basis using migrations, as though you were writing any other kind of database driver application.
Although android has built-in support for sqlite and migrations, unity doesn't; you'll have to write some custom code for it.
You could also use a custom json deserializer to apply your migrations 'on the fly' as you deserialize objects, but the inbuilt
JsonUtility doesn't not support this; you'll have to use a custom serializer like https://github.com/jacobdufault/fullserializer
Saving large data sets
Now, regardless of the storage format for individual records, the key to saving large datasets is to do it in parts.
In sqlite, partition your data per table.
If serializing manually, split the data model into a series of small objects and save them one by one.
The reason for this is that the UI thread will block while you perform slow operations like saving. The correct approach here is to do something logically equivalent to this:
- Split data into records
- Show save progress indicator
- Start a coroutine
-- While there are records left to save:
--- Save a few records
--- `yield` to wait for the next frame and allow the UI to update
This is where sqlite really shines; sqlite is naturally partitioned into tables, and tables into rows. Inserting a few rows or updating a few rows is a simple atomic operation that is easy to split into parts.
Doing this by hand with serialized objects means manually looking after folders full of multiple distinct serialized objects; it's all the house keeping that sqlite does for you.
Remember that as with any UI driven application monolithic (large, slow, do-all-the-things) operations cannot be split onto multiple cores and cannot be deferred; to be responsive, tasks on the UI threads should periodically return control to avoid locking the UI. See this for more background on the topic on android: https://developer.android.com/training/articles/perf-anr.html
Unless the amount of data you're saving is trivial, saving objects using direct serialization is almost certainly not going to scale well (many unity games have ANR issues on android for this reason).
Basically, if you take anything away from this, it should be this:
- You should really invest in learning and using sqlite, even if you're using android. It's the 'right' solution for the problem you describe.