Game AI will encompass all those types of math. Making an intelligent AI system is a very challenging task. This task will rely heavily on machine learning, statistics, probability theory and graph theory.
Another area would be in procedural generation. Creating a world and most if not all the items in it intelligently, without much manual content generation is an excellent goal. Simulating the creation of goods, their use and resulting location in the world would be very interesting.
More procedural generation tasks may include:
- Generation of a detailed history of a world. Wars fought, the naming of lands, the rise and fall of leaders, plagues, etc.
- Character creation for NPC characters. With the traits of their ancestor y and personal histories.
There are a number of mini-games you can make with this kind of math background as well. For example:
- Simulating the spread of disease and the efforts to combat it
- Generating efficient travel networks for transportation between nodes (cities, planets, etc.)
- Simulation of evolution of player made organisms.
- and so on.
Most of these things you can start small with and develop solo. They may not have fancy graphics or spectacular effects, but they'll demonstrate your potential and develop your skills.
You do it gradually like any other task you're learning in life. Start very simple, create something small that can be improved over time with more and more complexities.