Moussa Traore, who led Mali from 1968 until he was ousted in a coup in 1991, died at his home aged 83 in the capital Bamako on Tuesday, his family said.
As a young lieutenant in 1968, Traore was the main instigator of a coup that overthrew Modibo Keita, the country's first post-independence president.
He became president the following year and ruled with an iron fist.
His 22 years at the helm of the West African country were marked by the arrests of opponents, quashed demonstrations, suspicious deaths such as that of former president Keita in detention, and accusations he embezzled international aid money.
But he was also known for his diplomatic skills. As chair of the then Organisation of African Unity (now the African Union), he played a key role in the 1989 Senegal-Mauritania crisis and the Chadian-Libyan conflict, as well as Liberia's first civil war.
Traore reached a peace agreement in 1990 with Mali's Tuareg armed rebel groups after making significant concessions.
But in 1991, the soldiers he had sent to quell pro-democracy demonstrators turned against him and overthrew him in a bloody insurrection, which officially resulted in more than 200 dead and 1,000 wounded.
He was sentenced to death for "political crimes" in 1993 and, along with his wife, for economic crimes in 1999.
The sentences were commuted to life in jail, and Traore was pardoned in 2002.
"When you embark on a military career, you prepare yourself for the idea of not dying in your bed," he said upon hearing his first verdict in 1993.
'Patriots among us'
In recent years, Traore was increasingly seen as an elder statesman, with politicians soliciting his advice.
Former prime minister Soumeylou Maiga expressed on Twitter his "great dismay" over Traore's death, in a tribute in which he referenced the "friendship and respect that we have developed over the past several years".
Traore's death comes just four weeks after another putsch -- Mali's fourth since independence from France in 1960 -- after rebel army officers overthrew president Ibrahim Boubacar Keita on August 18.
Twice-elected Keita still had three years left in his second five-year term in office.
He had been facing mounting protests over his failure to curb a bloody eight-year-old jihadist insurgency, cure Mali's sickly economy and root out corruption.
Traore had recently met with the country's military rulers.
"These young colonels are children... I tell them the mistakes made and what to avoid and I hope, I hope they understood," he said.
Asked about the state of the country, often described as close to collapse, he replied: "I have never despaired of my country, (there is) such a mess but there are still patriots among us."
Talks were taking place in Ghana on Tuesday between the junta leader, Colonel Assimi Goita, and the 15-nation West African bloc Ecowas on determining a timetable for restoring civilian rule.