Although I'm not a lawyer, my understanding is: The illegality depends on a number of factors, including, but not limited to, the following:
Jurisdiction (e.g., copyright laws aren't enforceable globally because many countries don't recognize intellectual property rights that were not granted within their borders, plus there are also some countries don't even have copyright laws)
Permission (when using someone else's copyrighted material, you must first obtain their permission if you don't want to open yourself up to potential litigation {getting sued})
Court judgement (a judgement must be provided by a court to determine if a violation of someone's intellectual property rights occurred, after which point court-sanctioned penalties may be applied)
In essence, the owner of the intellectual property has to take steps to protect their rights. If apparent violations are permitted, but no attempts are ever made to stop these violations, then others may [reasonably] conclude that these are not really violations (e.g., they were used with permission that was granted privately, which is normally how these things are handled).
So, the answer is: Maybe -- it might be illegal in that it could be a violation of intellectual property laws, and you'd need to ask someone qualified with the relevant legal expertise (such as a lawyer), on a case-by-case basis, if you were unable to get an answer from the copyright holder.