When I worked in conferences, we rarely paid speakers a fee at all. Occasionally we would ‘go big’ and hire a well-known name from the business world, and we would agree a fee with their agency (usually based on a fixed price advertised upfront).
Personally I never once thought we got value from them. Generally they won’t know your industry / audience, and usually won’t tailor their talk either (even if they claim they will). We even paid over £10k for one guy - a so called ‘internet pioneer / tech guru’ - who spent the whole talk speaking as if it were a new phenomena…I’d have been better paying my Grandad, so it’s also worth checking they’re still actually on-top of things and remain relevant, and aren’t just dining off their legacy.
Anyway, back to agreeing fees! I would say just ask what they want, offer them X% less (don’t accept their first offer!), come to an agreement if they’re within your budget then sign a contract. I would never entertain renegotiating after you’ve agreed (and signed a contract). That’s really unprofessional on their part.