Speaking from an unnamed hospital where he reportedly was being treated for stress-related complications, the longest-serving member of Congress maintained that the allegations against him "are not true," and that he could not explain their origin.
Mr Conyers, 88, told a Detroit radio station "I am retiring today," as he became the first to step down among four serving US lawmakers facing sexual misconduct allegations.
The lawmaker was abandoned by party leadership when the seriousness of the allegations against him became apparent, and as Congress grapples with how to address the issue of sexual misconduct.
Congresswoman Sheila Jackson Lee took to the House floor to read a letter by Mr Conyers to his colleagues, saying he notified House Speaker Paul Ryan and Democratic leader Nancy Pelosi that he was vacating his seat.
"I was taught by a great woman, my mother, to honour women," Mr Conyers wrote, according to the congresswoman.
"Given the totality of the circumstance of not being afforded the right of due process in conjunction with current health conditions and to preserve my legacy and good name, I am retiring."
A Conyers lawyer, Arnold Reed, said the departure was effective immediately.
Mr Conyers endorsed his son, John Conyers III, to replace him in his congressional seat.
It marked a stunning and rapid fall for one of America's civil rights pioneers, a man who worked with Martin Luther King Jr and hired rights icon Rosa Parks on his congressional staff, where she worked for 23 years.
But Mr Conyers, who took his Michigan seat in 1965 and was a founding member of the Congressional Black Caucus, insisted that his record would not be tarnished by the accusations.
"My legacy can't be compromised or diminished in any way by what we're going through now. This too shall pass," he said, speaking on the Mildred Gaddis Show.
Several female former staffers have come forward in the past two weeks with damning allegations.
One of them, Marion Brown, last week broke a non-disclosure agreement that she signed after receiving a settlement when she left her job in Mr Conyers's office, saying she "felt compelled" to speak out.
"It was sexual harassment, violating my body, propositioning me, inviting me to hotels with the guise of discussing business and then propositioning me for sex," Ms Brown told NBC.
Zero tolerance
Wounded by the mounting accusations, and well aware of the potential implications for Democrats' efforts to win congressional seats in next year's mid-term elections, the party's congressional leadership last week called for Mr Conyers to resign.
Jerrold Nadler, who replaced Mr Conyers when he stepped down from his leadership role on the House Judiciary Committee, said he was "saddened" by the developments.
"With that said, there can be no tolerance for behaviour that subjects women to the kind of conduct that has been alleged," Nadler added.
Mr Conyers is one of four sitting US lawmakers hit by allegations of sexual misconduct in the wake of the accusations targeting movie mogul Harvey Weinstein, which triggered a torrent of accusations against powerful figures in the worlds of entertainment, media and politics.
The spectre of sexual misconduct is also overshadowing a current US Senate race.
Republican Roy Moore, a former Alabama supreme court chief justice, has been accused by several women of sexually assaulting or touching them when they were teenagers – including one who has 14 at the time – and he was in his thirties.
Several Republican senators called for Moore to quit the special election race. But President Donald Trump, who himself faced sexual misconduct accusations when he was a candidate, has endorsed Moore.
