His first spouse, Ms Holderness, said Mr Porter had kicked her on their 2003 honeymoon in the Canary Islands.
The US government analyst also alleged he punched her in the face while they were on holiday a couple of years later in Florence, Italy.
Ms Willoughby, a motivational speaker, told US media she was married to Mr Porter from 2009-13.
She wrote about her experiences in a blog post entitled Why I Stayed .
Ms Willoughby said she filed a protective order against him in June 2010 after he allegedly punched the glass of the door at their Alexandria, Virginia, home.
She told the Washington Post : "He has never faced repercussions that forced him to confront his issues."
A third woman who says she was Mr Porter's girlfriend contacted his ex-wives with allegations of "repeated abuse", according to CNN.
The unnamed woman was reportedly seeking advice on how to leave Mr Porter.
"I work in politics, and despite Rob's repeated abuse, some of which I think many know about, he continues to rise and I'm afraid to go against him," the woman reportedly wrote to Ms Holderness in 2016.
Mr Porter attended Harvard with President Trump's son-in-law, Jared Kushner, and was a Rhodes Scholar at Oxford University.
He formerly worked for Utah Senator Orrin Hatch, who said he was "heartbroken" by the allegations.
According to US media reports, Mr Porter has recently been romantically linked with White House communications director Hope Hicks.
She was reportedly involved in drafting the initial White House statement about Mr Porter on Tuesday.
