Hailemariam's announcement comes amid a political crisis and lingering unrest in the Horn of Africa country, which has been releasing thousands of political prisoners to ease tensions.
"Unrest and a political crisis have led to the loss of lives and displacement of many," Hailemariam said in a televised address on Thursday.
"I see my resignation as vital in the bid to carry out reforms that would lead to sustainable peace and democracy," he said.
Hailemariam added, however, that he will stay on as prime minister in a caretaker capacity, until the ruling Ethiopian People's Revolutionary Democratic Front (EPRDF) and the country's parliament accept his resignation and name a new premier.
Al Jazeera's Mohammed Adow, reporting from Doha, said parliament will meet on Friday to choose Hailemariam's successor and noted that Ethiopian Foreign Minister Workneh Gebeyehu is considered to be a leading candidate for the position.
"If they choose a candidate from either of the two main groups who have been protesting for most of the past three years, the Oroma and the Amhara, then it will be interesting to see how they are going to appease the other group that they leave out of this coalition," he said.
Hundreds of people have died in a wave of violence across Ethiopia, initially sparked by an urban development plan in the capital, Addis Ababa, in 2015.
The unrest spread as demonstrations against political restrictions and human rights abuses broke out.
Demonstrations demanding greater freedoms began in late 2015 and engulfed much of the restive Oromia and Amhara regions, Ethiopia's most populous areas, before spreading to other parts of the country.
The unrest led to a months-long state of emergency that has since been lifted.
The government has long been accused of arresting critical journalists and opposition leaders.
Human rights organisations and opposition groups have called for their release, saying they were arrested on trumped-up charges and punished for their points of view.
In January, Hailemariam announced that his government would release detainees and so far 7,000 people either saw their charges dropped or were pardoned.
An academic-turned-politician, Hailemariam has led Ethiopia since 2012, after the death of former leader Meles Zenawi.
He served as deputy prime minister and foreign minister under Meles before assuming power, and he was also elected chairman of the African Union in 2013.