Well, you'll have to use a little bit of physics, but you don't need to simulate any physics. There are formulas for pendulum motion you can easily use to set the rotation of your pendulum. For small swings, the motion can be approximated with simple harmonic motion.
The angular displacement overat a specific time can be approximated with:
This is most accurate for a small maximum θ, but will likely be accurate enough for your purposes. Create a function that takes the current time, and outputs the angle for which your pendulum should be rotated, and rotate your sprite by that amount.