To determine an auction bid, you need the AI to be able to estimate the value it expects to be able to earn from that property.
First then, we need an estimate for how many rounds the game is likely to last. You could get this by playing a large number of games and calculating the average number of rounds they go for. Or for a more dynamic estimate, compute the average rate of money loss for a losing player at each turn. The estimated turns remaining can then be found by taking the current net worth of all players and subtracting this value for each projected future turn until all but one is reduced to zero. If you don't have the data gathering to support this, you could also just take the average rental cost on the board currently and divide the second-highest net worth by this value as a crude estimate.
Next, it needs to compute how many times an opponent is likely to land on that property in that remaining time. To a first approximation, players land on 1 space per round out of 40 spaces on the board, so we'd expect roughly
remainingOpponents * remainingRounds / 40.0 opportunities to collect rent from this property in future. Multiply this by the rent and you have a guess at how much income this property will generate.
At any time this property may also be mortgaged, so add the mortgage value to the expected gain to get our first estimate of its net worth to the AI. (So even a property with negligible future rent prospects may still be worth picking up if we can buy it at/below its mortgage value).
This estimate can now be used as the upper limit for the AI's bid. It should start bidding below that value and above any opponents' bids (you can use your heuristic of choice to choose a starting bid/increment), and stop bidding if the leading bid goes above this threshold. At that point we'd estimate that buying this property would be a net loss by endgame.
We can refine this estimate in various ways:
If the AI collects all properties of this colour, or improves this property with houses/hotels in future, the rent may increase.
If the AI already owns other properties of this colour, getting this property could increase the rent they can charge at those properties too, or enable improvements via houses/hotels.
If the AI is short on cash, it may want to factor in the opportunity cost of spending its liquid funds at auction. You could evaluate the expected value of buying any of the available properties the AI is likely to land on next turn — if the expected value there is greater, cap your bid at what you can spend while still affording this future purchase.
You could add in the expected savings in rent that you would otherwise pay in the future if the opponent with the current leading bid were to win. This can also include savings from landing on their *other" properties of the same colour, whose rents may increase if they complete their set with this one, and increase further if they then build houses...
You can add an "overspending budget" per lap, equal to the $200 passing GO bonus plus the AI's average rate of earning/spending on rent in a lap (measured from recent laps, or estimated by current property ownership). The AI can over-bid by this amount per lap (or per round if you divide it by the number of rounds per lap), and still stay in the black — still not have a net drain on its coffers. If you spread this over the lap then you'll tend to have leftover budget as you turn the last corner, so you can bid more aggressively on the high-value properties at the tail end of the board.
If the AI has more cash in hand than it's likely to need to pay rent in the short term, then it can similarly increase this budget.
You could run this estimator from each opponents' perspective, and if it has high value to the leading bidder, bid up to a factor of their valuation, to drive up the price they pay or deny them the ability to secure it.
If the supply of available properties is dwindling or the AI is behind its opponents in terms of properties in hand, you may want to increase this estimation by a "hunger factor" to guard against the AI being shut out of the property game while waiting for the optimal purchase to come along.
Each of these refinements can make the AI more strategically savvy and might improve numerical results over a large number of games. But they also increase the complexity of the algorithm and might not significantly improve the perceived intelligence of the AI from the players' perspective — overcomplicating this could actually make the AI look less intelligent, if it makes a bidding choice for non-obvious reasons that look to the players like a bug. So exercise judgement in deciding just how nuanced this feature needs to be for your goals guiding this project.