Allen hurt his shoulder on the play and placekicker Stephen Gostkowski punted the rest of the game. Brady took over Allen's holding duties.
Indianapolis got the ball on the free kick, but the drive ended with Luck's second interception.
Vinatieri's 21-yard field goal early in the third quarter made it 21-15.
Then the Patriots took a 29-15 lead on Ridley's 3-yard run and his carry for a 2-point conversion.
But Luck came right back with his second touchdown pass to Brazill.
"We didn't accomplish the ultimate goal," Colts running back Donald Brown said, "but there are definitely some positives to look back on."
Brady completed 13 of 25 passes for 198 yards with no touchdowns or interceptions. Luck was 20 for 41 for 331 yards.
Notes: Former NBA star Charles Barkley was in the Patriots' locker room after the game. "I'm a big fan of his," Belichick said. ... A moment of silence was held for 17-year-old Sam Berns of Foxborough, whose battle with a rare and fatal genetic condition that accelerates the aging process inspired Patriots players and was the subject of an HBO documentary "Life According to Sam." He died Friday. He had been invited by Patriots owner Robert Kraft to be the team's honorary captain Saturday.