Ally began his senior career with St Mirren and during his time at Paisley he also had a short spell on loan with Stenhousemuir. Following his departure from Saints, Ally played for a short period of time with East Stirlingshire and Annan Athletic. He signed for Albion Rovers in 2014 and spent two seasons at Cliftonhill, in the second of which he was the club's top scorer. Ally joined up at Glebe Park at the beginning of the 2016-17 season and scored a number of crucial goals as City achieved promotion via the play-offs