Hooker Penny made his Tigers debut on 4 January 1896 at Gloucester. He would appear in ten Midland Counties Cup finals with Leicester as the club rose to national prominence.

Presented with a scroll to mark 500 appearances, it would emerge he was actually nine short. He did however have the honour of the most Tigers appearances until David Matthews (502) beat his total in 1974.