# How to save large data in Unity?

Can anyone please tell me how to save data for Android.

I have too much data to save (many npc-enemy stats). Also, it should work after updating: I read on stack overflow that, with BinaryFormatter, when you update or change the class, your old settings might be lost since the classes no longer match. Sometimes, you get an exception when reading the saved data due to this.

I also read about JSON/XML. Does JSON/XML throw an error if I change class?

• Definitely not JSON, nor XML. Just think about it. How much wasted space they contain. Jan 26, 2017 at 9:32
• then binaryformatter ? Jan 26, 2017 at 9:33
• @Bálint Drive space is cheap, even on mobile devices. And you can further reduce the size of JSON or XML by zipping your savegames. Jan 26, 2017 at 10:25
• NPC & enemy stats are one of the last things I'd expect to take up a lot of disc space, so this makes me think you might be doing something very unusual. Can you tell us more about the specific data you need to store (what parameters & types, etc) and how much of it you have? Since you're concerned about changing formats, it would help to share an example of the format or structure of the data you're using now, and some ideas for what kinds of future changes you want to be able to handle compatibly. Jan 26, 2017 at 13:46
• I am making tycoon game, so data is Ai info, player stats, market calculations, etc. most of the data is stored in classes. Jan 26, 2017 at 15:05

If I want my data to be readable I would go with JSON. With JSON.NET for example, it's easy to set default values for fields or ignore unknown fields. So if you either add or remove fields from your class it might be out of balance but it won't break.

But what exactly is too much data? Do you need to serialize billions of large objects without cutting corners? If this is the case, JSON might be too sow for you. But be careful when you say large, it's a relative term and my intuition tells me you are talking about serializing a relatively small amount of data.

If it really is a huge amount of data and it does not need to be humanly readable you can use a binary writer. This is a lot faster but without all the handy functions from JSON libraries, you need to tell it what fields to serialize and de-serialize.

### Short version

Use sqlite. Really. I know it's more work than just using a serializer, but that's the way all normal android apps do it, and its the right way to do it, as I'll explain below.

See https://github.com/codecoding/SQLite4Unity3d for how to do that.

If that's too much work, checkout out custom serialization using https://github.com/jacobdufault/fullserializer

### Long version

The format that you save your data in is an orthogonal concern to the way in which you save large data sets.

There are two issues here:

• When I save a record, I need to be able to restore that record, even if the data model has potentially changed in code compared to when I saved the record.

• I need to save a large number of records, and it's too slow.

Lets address these problems one by one:

### Deserializing when the model has changed

An XML/json serializer will solve this problem to some extent, but ultimately, if your data model changes you need to use a migration to migrate old data to the new schema.

There is no way to avoid this problem.

Using a weakly typed serializer (eg. json) will allow you to receive null values for missing keys, but ultimately, if a field changes type, you'll still have broken data.

Don't ever use BinarySerializer.

The standard solution to this problem is to use a database, and migrate data on a per-table basis using migrations, as though you were writing any other kind of database driver application.

Although android has built-in support for sqlite and migrations, unity doesn't; you'll have to write some custom code for it.

You could also use a custom json deserializer to apply your migrations 'on the fly' as you deserialize objects, but the inbuilt JsonUtility doesn't not support this; you'll have to use a custom serializer like https://github.com/jacobdufault/fullserializer

### Saving large data sets

Now, regardless of the storage format for individual records, the key to saving large datasets is to do it in parts.

In sqlite, partition your data per table.

If serializing manually, split the data model into a series of small objects and save them one by one.

The reason for this is that the UI thread will block while you perform slow operations like saving. The correct approach here is to do something logically equivalent to this:

- Split data into records
- Show save progress indicator
- Start a coroutine
-- While there are records left to save:
--- Save a few records
--- yield to wait for the next frame and allow the UI to update


This is where sqlite really shines; sqlite is naturally partitioned into tables, and tables into rows. Inserting a few rows or updating a few rows is a simple atomic operation that is easy to split into parts.

Doing this by hand with serialized objects means manually looking after folders full of multiple distinct serialized objects; it's all the house keeping that sqlite does for you.

Remember that as with any UI driven application monolithic (large, slow, do-all-the-things) operations cannot be split onto multiple cores and cannot be deferred; to be responsive, tasks on the UI threads should periodically return control to avoid locking the UI. See this for more background on the topic on android: https://developer.android.com/training/articles/perf-anr.html

Unless the amount of data you're saving is trivial, saving objects using direct serialization is almost certainly not going to scale well (many unity games have ANR issues on android for this reason).

Basically, if you take anything away from this, it should be this:

• You should really invest in learning and using sqlite, even if you're using android. It's the 'right' solution for the problem you describe.

you can use either SQLite(also if you need key-value pair/Doucument database can use UnQLite istead) or JSON(json have less waste characters included and is much smaller if you remove all \r\n in it compare to XML). for json use this : http://www.newtonsoft.com/json

• How would they use it? Just listing technologies they can use is as good as a Google search... Jan 26, 2017 at 11:14
• Why the downvote? This answer is helpful (the only criterium for an upvote) because the asker might not know about these technologies to do a search in the first place.
– Eric
Jan 26, 2017 at 21:49
• @aliagamon Welcome to the GD.SE community! Thanks for your first answer. Could you perhaps elaborate on the reasons for choosing either SQLite or (whitespace-stripped) JSON, preferably within the context of Unity publishing to Android? I agree with Alexandre that this would definitely improve your answer.
– Eric
Jan 26, 2017 at 21:56

You should give a try to Easy Save. Best suitable for large projects having number of files or large files to manage. Variety of primitive and unity types are supported. Cloud storage, encryption and spreadsheets are also supported. Which may help a lot for large project's save system development.