Kenyan President William Ruto is the first African leader in more than 15 years to make an official state visit to the US.

This is an opportunity for President Joe Biden to demonstrate commitment to Africa at a time when Washington appears to be playing catch-up in its engagement with the continent.

But relations with other African allies are under strain, as strategic rivals including Russia and China challenge traditional areas of Western influence.

At one time Mr Ruto would have been an unlikely candidate to be feted at the White House with the pomp and ceremony granted to only a handful of close allies a year.

The International Criminal Court charged him with crimes against humanity related to the violence that followed Kenya’s 2007 election. But the case collapsed and Mr Ruto has since reinvented himself as an indispensable partner to the US.

Lingering suspicions about his democratic credentials are not the reason that congress decided against inviting him to address a joint session, says US ambassador to Kenya, Meg Whitman. As far as she knows, it is a scheduling issue.

Ms Whitman, a former CEO of companies such as eBay and Hewlett Packard Enterprises, is a champion of Kenya and its investment potential as a tech hub, the so-called Silicon Savannah.

“If you really want to lean into Africa, then who would be the right choice to come to a state dinner?” she asks.

“Kenya has been a long-standing 60-year ally of the United States. It is certainly the most stable democracy in East Africa. President Ruto has stepped up and he’s a real leader.”

Under Mr Ruto Kenya has developed its role as the region’s diplomatic and business centre, an “anchor state” for the US in a tough neighbourhood.

Although domestically he has faced protests over his handling of the struggling economy, globally he has become an advocate for Africa on issues related to climate change and debt relief.

Kenya is also an important security partner in East Africa, and has pleased Washington by pledging to send Kenyan police to Haiti.

The only phone call President Biden made to a leader in sub-Saharan Africa last year was to Mr Ruto, about Nairobi’s promise to lead a multinational force to the troubled country.

Analysts suspect the state visit is partly meant to compensate for the fact that Mr Biden has failed to keep his own promise to visit Africa.

He made the pledge at a grand summit of African leaders in Washington two years ago, in which he assured his guests he was “all in” for the continent. But since then, he has been distracted by crises elsewhere, such as the wars in Ukraine and Gaza.

The summit followed the administration’s announcement of a new strategy aimed at turning its relationships with African countries into more equal partnerships that advance the strategic interests of both.

In some ways Mr Ruto is the poster child of that approach, but as he arrives in Washington the focus has been on US setbacks in West Africa.

If there’s one country which best captures the challenges facing the US in Africa, perhaps it is Niger.

For years it was home to more than 1,000 US troops, stationed on two bases from where they launched security operations against Islamist militants in the region.

But a coup last year changed the nature of the relationship – with Niger’s military rulers growing closer to Russia and Iran.

American efforts to find a way to continue security co-operation broke down in March.

The junta’s prime minister told the Washington Post a senior US delegation had taken a “condescending tone” and shown “a lack of respect”. He accused it of trying to dictate Niamey’s relationships with other countries.

This week the Pentagon confirmed a full withdrawal of its troops by September – opening the door to even closer ties between Niger and Moscow.

Molly Phee, the State Department’s top African affairs official, says it was impossible to square America’s interests and values, which also included a timeline to return to civilian rule, with the junta.

“We shared legitimate concerns about the trajectory of [Niger’s] discussions with Russia and Iran,” she told the BBC.

“In the end, we were not able to reach an understanding that addressed our top priorities,” she said, noting that the relationship should be reciprocal.

“We intend to maintain a diplomatic partnership, as well as other aspects of our relations.”

The breakdown follows Niger’s expulsion of the French, the old colonial power.

It highlights the tensions as the US tries to balance security partnerships with democratic values, limitations the Russians do not share.

What has happened in Niger has been echoed in other Sahel countries – with Moscow happy to offer protection to those who seized power in a string of coups, often in exchange for access to natural resources.

In recent weeks a small contingent of American troops were forced to leave Niger’s neighbour Chad, when officials there questioned the future of the US presence.

America is also facing increased competition from other nations on the continent. China has been investing in Africa for two decades, but there is a host of new middle-power players.

A Gallup poll last year found that the US had lost its soft power edge while China had gained fans. But the biggest change was Russia’s rise in popularity.

“Historically the West has seen Africa as a problem to be solved. Actors like China and Turkey, and other Arab Gulf players, they see it as an opportunity to be seized,” says Muritha Mutiga, the Africa programme director for the International Crisis Group.

“So, the way in which China, Turkey and the Gulf have engaged has been welcomed, because it is seen as a long-term bet, it is seen as taking the continent seriously.”

The Biden administration points to some success in its efforts to treat Africa as a strategic partner.

A stream of high-level visits has framed the importance of Africa as “the continent of the future”, with its young fast-growing population, abundance of natural resources and increasing influence on the international stage.

American backing has helped African nations win better representation at global forums, such as the G20, IMF and World Bank, although the US has struggled to get African support for its positions on Israel’s war in Gaza, and Russia’s war with Ukraine.

The administration has also won plaudits for investing in the Lobito Corridor, a rail line snaking through Angola, the Democratic Republic of Congo and Zambia that will be used to transport critical raw materials.

“With that Lobito Corridor, [the Americans] decided to speak in the language that Africans understand,” says Kingsley Moghalu, a Nigerian political economist and former central bank governor.

“If you are seen to be delivering major projects that are beneficial to African economies, and to African people, then off the back of that you have leverage to talk about democracy and things like that.”

Alex Vines, head of the Africa Programme at the Chatham House think-tank in London, pushes back on the perception that Western power is fading in Africa.

“One African leader said to me: ‘We get tired of a Chinese buffet, we’d like to go a la carte, we want choice,’ ” he says.

“So, I do think what we’re increasingly seeing is [that] plenty of African countries want a bit of the United States, but they will want a bit of Russia or UAE or Turkey.”

The challenge is “patchy African leadership” with ambitious, long-term vision that can make the most of the competition.

President Ruto is seen as one of the figures who can, but all, including Niger, have options.

“There is a game of chess going on,” Dr Vines says. “There is a new scramble for Africa. The difference is that the chess board, the African continent is alive, it’s not passive. It can suck people in and really surprise them.”