Should we sue?

ok here's the scoop...my ex boyfriend is calling these chatlines and giving out my home phone # on them, having a female pretend to be me and saying that I'm "down for whatever" and basically making me look like a slut. Now, he's printing up these flyers with pics of me and my new boyfriend saying that we're a "swinger" couple, and we DON'T even get down like that! Meanwhile, I'm getting all these weird phone calls all hours of the day AND night from different guys saying that they talked to me on the chatline and on the flyers, and I NEVER go on them or to any swinger's clubs, and I have never put any flyers up about my new boyfriend and I. My parents, new boyfriend, and I are thinking about taking him to court and suing him for defamation of character, both slander and libel. Will there be enough to go on?