I want to work on sound in games, but I don't have any experience in scripting, only in sound design for film. Some friends that do know scripting are starting to learn in Unity 5, and my part would be working on sound. So my question is do I need to learn the whole C#, as it's used in Unity or can I learn only the part that affects my part of work? Or does this work in this way anyway?
That depends on how you distribute the workload on your team and how much you want to add to the project.
Someone who does nothing except producing audio files and never even touches the unity editor can still add value to the team. Just ask the other team members what sound effects they need, provide them with the files and let them worry about getting the engine to play them. (this is actually the only task described here I, as a programmer, feel completely clueless about and wouldn't even know where to start).
Someone who understands what all the settings in the AudioSource component actually do and what all those filters in the audio mixer are good for could add even more value because that person could then be responsible for using the Unity editor to set up appropriate audio sources for everything. Someone with a background in sound design might be able to make better creative choices in this regard than a layman.
Someone who also knows some scripting can control when and how individual audio clips are played and can use that to take full control of the audio experience of the game while others can concentrate on other aspects.