This is an issue virtually every lawyer faces at one time or another. What you're referring to, though you may not be aware of it, is the Pareto Principle—the general rule that 80 percent of a given outcome can be traced to 20 percent of possible causes. In the best case, you can identify the 20 percent of your practice's clients who give you the majority of your aggravation, and fire them—and retain the 20 percent that produce 80 percent of your profits!
The problem arises when these percentages overlap—that is, when one of your clients is difficult to work with, yet also contributes a fair amount to your bottom line. In this case, you have to sit down and really think about the consequences of cutting this client loose. Sure, you will lose some money in the near term, but if this person has made you so crazy that you can't even think straight, you may find that getting rid of him frees up the mental space you need to find and service new paying clients.
On the other hand, if firing this client would be an extreme financial hardship, you need to come up with a strategy by which you can retain his business and reduce your angst. A patient intermediary (a paralegal or administrative assistant) might be one solution, or you can come to an agreement with the client that you will only be available via phone or email during specific times, or address the other issues (late payments, missed appointments) that make him difficult to handle.