The acting Haitian prime minister has accused former President Jean-Bertrand Aristide of "disturbing Haiti's fragile order" by visiting nearby Jamaica.
Gerard Latortue told the BBC Mr Aristide wanted to "encourage armed groups to destabilise the country".
He said he had recalled Haiti's ambassador to Kingston in protest at what he called Jamaica's "insult".
Mr Aristide, who left Haiti on 29 February amid a rebellion, made a call for peace as he arrived in Jamaica.
He says he is on the island to see his two daughters.
Speculation is rife that Mr Aristide's proximity to Haiti - Jamaica is just 200km (125 miles) away - could revive the violence that forced him into exile.
Mr Aristide was taken by helicopter from Jamaica's main airport to a gated, heavily guarded house near the northern resort town of Ocho Rios.
In a brief arrival statement, he said he wished for peace in Africa and in the Caribbean.
Mr Aristide, his wife Mildred and his small party were greeted by a team of middle-ranking government officials.
It did not include Prime Minister PJ Patterson or any of his senior ministers.
Mr Aristide is travelling with a US-Jamaican delegation and is expected to spend about eight to 10 weeks on the island.
Speaking to the BBC's World Today programme, Mr Latortue said: "All of us, we know - [Mr Aristide] is calling all the time, everybody... All the money he took with him, he is using that money to encourage armed groups to destabilise the country."
He accused the Jamaican government of "a kind of insult to the Haitian population at a very difficult time when we are trying to lay down the basis for a new democracy in the country" by hosting the ousted leader.
Haiti's envoy to Kingston was being recalled, he said, until normal relations resumed.
The BBC's Stephen Gibbs, who is in Port-au-Prince, says news of Mr Aristide's journey to Jamaica has divided the city between his supporters and opponents.
The official purpose given for Mr Aristide's journey is for him to be reunited with his young daughters.
But earlier, on leaving his brief exile in the Central African Republic, Mr Aristide failed to make clear what his true plans were.
"For the time being, I'm listening to my people," he said before boarding a plane in Bangui.
"The more we listen to them the more we serve them, the more we will know what to do at right time. We all have to do what we can to promote peace," Mr Aristide said.
The Jamaican government is aware of the sensitivity of the planned visit; it says it has made it clear to Mr Aristide he cannot use the visit as a launching pad for a campaign to be reinstated as president.
The Jamaicans say he has agreed to this condition, say correspondents.
A US peacekeeper injured by gunfire has become America's first casualty on its mission to Haiti.
The soldier was on patrol in the Belair district of the capital Port-au-Prince, the US military revealed.
Tensions are high in the Belair area, where residents accused US soldiers of shooting dead two unarmed men on Friday; the Americans say they were responding to firing.
At least six Haitians have died in fire by US troops in the past week or so. http://news.bbc.co.uk/1/hi/world/americas/3511676.stm