A similar substance found Tuesday in an envelope addressed to Obama tested positive for ricin in a field test, it says.
The three letters all contained "the same verbiage, font, style and paper color," it says.
The letters were postmarked Memphis, Tennessee, which is typically the postmark that letters mailed from northern Mississippi bear, it says.
On his Facebook page, Curtis posted the same quote: "To see a wrong and not expose it, is to become a silent partner in its continuance," the affidavit says.
Sgt. Corrie Robbins of the Booneville Police Department in Mississippi told investigators that Curtis had been investigated several times since 2007, the affidavit says. It adds that Curtis' ex-wife reported to police in 2007 that he was "extremely delusional, anti-government, and felt the government was spying on him with drones."
If convicted, Curtis will face a maximum of 15 years in prison, $500,000 in fines and three years of supervised release.
Wicker said Thursday that he met Curtis about a decade ago. "He's an entertainer," the senator said. "He's an Elvis impersonator, and he entertained at a party that my wife and I helped give for a young couple that was getting married. He was quite entertaining."
The Clarion-Ledger of Jackson, Mississippi, posted photographs of a man it identified as Curtis. In one photograph, he is shown under an "Elvis" sign holding a microphone as he appears to be singing. He is wearing a white suit and sporting long sideburns and swept-back hair.
A Kevin Curtis Live Facebook page describes him a "Master of Impressions performing 'Tribute to the Stars' for audiences of all ages!"
The FBI arrested him on Wednesday at his home in Corinth.
Letters put focus on Texas chiropractor's words
The line in the letters about exposing "a wrong" comes from John Raymond Baker, a longtime Texas chiropractor, his wife said. It's been widely quoted online, but Tammy Baker sounded surprised that it was used in the letters under scrutiny in Washington.
When contacted by CNN, she said that she was not aware of the letters and that the phrase refers to her husband's general philosophy of care.
She said their office phone rang frequently Wednesday afternoon, which was "kind of freaking out our other employee."
A 2006 post on a blog for Baker's office said the comment originally was a criticism of insurance companies. Since then, the site said, it "has been a quote that has been picked up and quoted (sometimes without attribution) around the net" and "people are using it about all kinds of injustices."
Mail for members of Congress and the White House has been handled at off-site postal facilities since the 2001 anthrax attacks, which targeted Sen. Patrick Leahy, D-Vermont, and then-Majority Leader Tom Daschle, D-South Dakota.
On heightened alert
Suspicious letters in Michigan and Arizona, too
Investigators are trying to determine whether suspicious letters found at Senate offices elsewhere in the country came from the same source, federal law enforcement sources said.
Sen. Carl Levin, D-Michigan, said one of his home-state offices received a "suspicious-looking" letter and alerted authorities. "We do not know yet if the mail presented a threat," said Levin, the chairman of the Senate Armed Services Committee.
A staffer for Arizona Sen. Jeff Flake flagged "suspicious letters" at the freshman Republican's Phoenix office, Flake spokeswoman Genevieve Rozansky said in a statement, but "no dangerous material was detected in the letters."
Phoenix Fire Department spokesman Jonathan Jacobs said the envelope contained a powder. The person who found the envelope was being treated at a Phoenix-area hospital for a pre-existing condition and stress from the event, and others in the immediate vicinity were examined as well.
In a statement issued Wednesday, the FBI said it has no indication of a connection between the letters and Monday's bombings at the finish line of the Boston Marathon.
Ricin is easily made
Ricin is a toxic substance that can be produced easily and cheaply from castor beans. As little as 500 micrograms, an amount the size of the head of a pin, can kill an adult. There is no test for exposure and no antidote.