I don't understand all this fuss over speaking fees. A lot of people who've been "on the circuit" gave themselves away and even paid for the right for years before getting their heads above water - I know I sure did. Still do, really.
It is the nature of a capitalist society for businesses to enter into contracts seeking the most advantageous terms available. If you signed up for something but didn't nail down all the details up front and don't like how it turns out, it's your fault. If you don't get offered what you want, don't take the deal. If you need the gig to get ahead, suck it up and pay your dues (so to speak). The rest of us have, it's just a rite of passage. Just try to be better later on when you're the one in charge.
One of my mentors says quite eloquently that you need to approach each job with a clear idea of what you need out of it and what you're willing to put into it, and be clear about where you have to draw the line for each individual gig - and then not cross that line. You might draw the line at one end of the spectrum for a particular event because of special circumstances, whatever they might be, and then draw it at the other end for other events. But you draw the line, each time, and stick to it, period, or you've got nobody to blame but yourself.
There might be something wrong with your regional library association, but this isn't it.