Celtic failed to press home their advantage though, with the champions restricted to shots from long-range, and they suffered a defensive lapse of their own at the other end.
Nicke Kabamba had the better of Christopher Jullien physically, and was hauled to the ground after turning the centre-back on the byline.
Alan Muir pointed to the spot and Chris Burke made no mistake as he slotted the ball past Vasilis Barkas from 12 yards.
Edouard went close with a free-kick that fizzed just past the far post, but from open play the visitors were limited to shots from range.
Callum McGregor sent one such effort onto the roof of the goal shortly after the hour mark, but Lennon had seen enough.
The Celtic boss sent on Patryk Klimala and Hatem Elhamed as he looked to inject some spark into his side with a switch to 3-5-2.
Burke hit an early contender for goal of the season in the opening day loss to Hibs, but a free-kick from a similar range went harmlessly wide after Scott Brown had fouled Aaron Tshibola.
Lennon went even more attacking late on as Boli Bolingoli replaced Greg Taylor at wing-back and Killie now camped very firmly in their own half.
He teed one up for Forrest in stoppage time, but the winger was crowded out by the Killie defence.
That was the last half-chance of the game, as Celtic dropped their first points of the new campaign.
Jullien’s physical problem
Christopher Jullien has been impressive for Celtic in some high-profile games, but there’s no question that he can struggle against some of the Premiership’s more physical strikers.
It was evident last season against Lyndon Dykes, and the Frenchman was given a torrid afternoon by Nicke Kabamba. The striker turned Jullien with ease and was hauled to the ground for the first half penalty from which Chris Burke equalised.
Kabamba created another half-chance for himself after brushing Jullien off the ball as half-time approached, and it was clear that Killie were looking to exploit the centre-back as often as possible.
Other Premiership managers will have been watching on with interest.
Having taken an early lead, most observers would have expected Celtic to take control of the game and ensure there was no way back for Killie.
While they dominated the ball though the visitors struggled to make Rogers work in the Kilmarnock goal.
Credit must go to Dyer’s men, who were compact and organised with Alan Power and Dicker breaking up play expertly in front of the defence.
However, it was a strangely toothless performance from Celtic that showed why Lennon has been so keen to bring in another striker this summer.
Promising signs for Killie
It’s been an up-and-down 12 months or so for the Rugby Park club, with the failed Angelo Alessio experiment followed by some unconvincing performances under Dyer.
Here though Killie looked more like the obdurate outfit we saw under Steve Clarke, limiting Celtic to very few chances and looking dangerous on the break.
Despite both picking up bookings Power and Dicker strolled through the 90 minutes, while Burke continues to defy time on the right hand side.
The hosts were without most of their summer signings, with last season’s top-scorer Eamonn Brophy left on the bench for the 90 minutes.
The fact they have players who can come into a side that performed so impressively will give Killie plenty of optimism for the season ahead.