I'm new to LibGDX but I am making a menu with some basic animation, I would like to start with a sprite scaled up and then it would transition to the normal size.
Is it possible with LibGDX or do I have to use a third party library?
Take a look at Actions, I'd say they're what you need: https://github.com/libgdx/libgdx/wiki/Scene2d#actions
I don't know LibGDX or if it has a built in animation library, but you can be certain that it does give you the ability to scale a sprite.
All you have to do is use a function of the desired graph shape to pick the size you'll render the button on each frame. For example, a linear interpolation for the scaling values will produce a perfectly smooth(err, linear) growing or shrinking effect. Periodic functions are a special class of function which repeat. Sine(sin) is such a function. You can use this to produce a continuous animation.
However, I will suggest that you only use this scaling temporarily. This is an animation that will always look the same; take the time to have an animation made in a proper animation tool and simply change sprites each frame. This approach can be combined with the math above.