It seems fairly obvious that exclamation marks, brackets and square brackets are legal characters in application names, on both the Google Play Store and Apple iTunes Store. We find evidence of this in applications that are already available on the two stores.
Examples from both markets
!! Exclamation Marks !!
Very easy to find across both stores, as they are use quite regularly. Android has the likes of Angry Birds GO!, and over at Apple we find examples such as Mahjong!!.
(( Circular Brackets ))
Still very common across both stores. Android has examples such as Tsumego Pro (Go Problems), while Apple features applications such as Piano Tiles (Don't Tap The White Tiles).
[[ Square Brackets ]]
While not as common, we find evidence of such use on Android, in applications such as Random Reader [Prototype]. Apple provides us an interesting example, showing off various other allowed characters with (:3 )+[____].
Google character use
While Google is not as friendly in allowing mass application search, there is theoretically no real reason that any character would be disallowed from application names. The name you see, and use to find the application via searches, is not the same name too which it is actually listed.
Google list the applications under the package name, which is typically of the format name.domain.com. We can see that formatting and special characters are often omitted from the package name, for example Angry Birds GO! has the package name of angrybirdsgo. This is the name used in the physical URL for the application, and the actual name in which it is saved to your device.
This makes it substantially easier to differentiate between two applications that have the same name. If two companies make an application of the same name, the two applications could theoretically coexist on the same system, because the inclusion of the unique company names would provide enough differentiation to prevent a computer system from overwriting one with the other. In fact, if you change the name of an application you are making but not the package name, it will still overwrite its predecessor version.
Apple character use
On the Apple side of things, I can only infer that special character use is greatly permitted, as I have had no experience in publishing to an Apple device. That said, they provide far greater allowance in performing searches of their application marketplace.
Of great importance, you can effectively browse all games (as opposed to categories on the Play store), sorted alphabetically. Of particular interest, we have the standard "other" option to select, which lists all applications starting with a special character or number. Roughly the first 10 pages show applications starting with a special character.
We can infer that if a character is permitable, there are good chances that someone will have published an application that starts with that character. While this is not a definitive way to identify characters that are not allowed, this makes it much easier to find evidence of characters that are allowed.