For example, a SQLite DB would be faster than reading text files on startup.
That's an interesting assumption. Opening it might be faster, depending on the size of the text file. But reading it is another matter altogether. If all your startup code does is open the file and read a couple of entries out, then it would be faster. But if you're reading the entire thing into memory, then a DB is going to be slower than the text file.
Not unless you write your own parser, or use a bad parser framework.
And of course, once it's in memory, finding the data for monster X will be faster than an SQLite DB. Assuming you use an appropriate data structure.
This is the only requirement that would suggest using an SQLite DB, so you probably shouldn't use that.
Lastly, remember that SQL databases are designed and optimized for complex searching for stuff. If all you're searching for in your DB is a monster by name or something, you're really using an over-designed tool for the job.
How to maintain data integrity. If it's just a text-file, you can put invalid values in it.
If you are going to allow your users to directly modify game data without forcing them to use a specific tool, then you're going to have to accept that users can do the wrong thing. With great power, comes great opportunity to abuse that power.
So your options are to "force" users to use a specific tool (which would require some form of encryption on your data file. It won't stop dedicated hackers, but the hoi polloi will be warded off), or to do integrity testing at load time. Or you could split the difference.
Let users edit text files however they please, but you introduce a "build mod" step between the text and your game. They have to run some command line tool that converts their text files into your game data. That's where integrity testing goes. You could also optimize your data in such cases, if text parsing is really that big of a turn-off for you. Of course, this makes it a bit more difficult to use, thus slightly slowing down turnaround time between editing the game data and seeing a change.
If possible, I would like to allow users to make their own tools to crunch my data.
That's a lot less viable for SQLite DBs. Especially if you add integrity check expressions. And it becomes much harder if those integrity expressions require functions that only your code exposes; that last part makes it impossible for someone to use their own SQLite tool without linking to a library of yours for the interface.
I would also like input on the file format (eg. YAML vs. XML vs. JSON vs. whatever)
That's dealer's choice. However, XML has one significant advantage (beyond being the most prevalent and de facto lingua franca of textual data storage): XML schemas can do integrity testing for you, for simple cases. There are even schema-guided editors that can help prevent users from doing the wrong thing at edit time.
I'd use whatever is easier for you as a developer. Pick the syntax you like best; there will be tools for reading it. Personally, I'd use Lua, as I would also be using Lua for scripting, so I would have it handy.