If an open source game's resources and source code appear to be governed by the same license, I would suggest you contact one of the project leads and verify that this is actually the case. The LGPL, and other open source licenses, are generally intended to govern the use of source code. Many open-source games adopt separate licenses for the source code and the content. In such arrangements, the content is often distributed under one of the many Creative Commons licenses. The CC licenses are relatively short and concise, outlining in plain English exactly how you may or may not reuse the content. Most of the open source projects I have worked with have used CC licenses which allow you to reuse any content you like, as long as you attribute the original author(s). To fulfill this stipulation, simply include the author(s) in your game's credits, along with a description of the borrowed content. Note that some licenses include a non-commercial stipulation, which bars reuse or redistribution with any commercial product. In such cases, you can still borrow content for freeware or open source projects. However, if you use the content in an open-source project which you later decide to sell, you will (as I understand it) need to remove/replace that content in the version you sell.
If the assets of an open source game are governed by an open source license, or a license such as the Creative Commons, you generally do not need to ask permission. The only exception would be if you do not fully understand the terms of the license. This shouldn't be the case with a CC license, but some open source licenses are more verbose and filled with legalese. If your redistribution rights are unclear, it never hurts to ask the author(s).
As to whether or not you should use open source game assets, I would say "absolutely". If you do not have the talent yourself (or on your team) to produce better assets yourself, using open source assets will enable you to deliver a better game whilst giving the original content author(s) greater exposure.
One last note: if you find that a game's assets are licensed under a FOSS license, be wary of stipulations concerning the relationship between your game's license and the source game's license. Some licenses only allow code and assets to be used in other projects that are governed by the same license or a similar license (i.e. both licenses are permissive or both licenses are reciprocal). Be careful not to back yourself into a license that you're not completely comfortable using, all for the sake of using someone else's assets. If you detect a possible license conflict, you can always ask the original author(s) for permission to redistribute the assets under a different license.
