The issue is that, with so much time remaining, the liklihood that a successful 2pt conversion would make a major difference in the game was actually less than the risk that failing to convert would change the way FIU had to play the remainder of the game.
This wound up being the case, with FIU failing its 2pt try, necessitating that they go for 2 after their next TD, which they also failed. This opened the door for UCF to win on its final drive with only a FG. Had they just kicked extra points on their TDs, a UCF field goal would have only tied the game and forced overtime.
Things worked out in the end, thanks to a little luck, but that's what the term "chasing points" means - once you start, it changes the way you have to play, forcing you to incur additional risk if you fail.