This guy and I had rendez vous plans. How do i back out without him being mad?

We decided that on Friday we were going to go out and then come back to his place and..yeah. But I really cannot go through with this given that I've started having strong feelings for a coworker. How do I back out without it looking like I was leading him on for fun? He is also a coworker and I don't want him to reveal these plans out of spite to people.