COLUMBIA, S.C.—South Carolina Gov. Mark Sanford said Tuesday that he “crossed lines” with a handful of women other than his mistress—but never had sex with them.

The governor said he “never crossed the ultimate line” with anyone but Maria Belen Chapur, the Argentine at the center of a scandal that has derailed his once-promising political career.

“This was a whole lot more than a simple affair, this was a love story,” Sanford said. “A forbidden one, a tragic one, but a love story at the end of the day.”

During an emotional interview at his Statehouse office with The Associated Press on Tuesday, Sanford said Chapur is his soul mate but he’s trying to fall back in love with his wife.

He said that during the encounters with other women he “let his guard down” with some physical contact but “didn’t cross the sex line.” He wouldn’t go into detail.

Sanford, 49, said the casual encounters happened outside the U.S. while he was married but before he met Chapur, on trips to “blow off steam” with male friends.