Honestly the best thing to do if you’d rather let him down gently is just say something along the lines of “I’ve been thinking and I don’t want to ruin our friendship. Being in a relationship can cause a lot of complications and that could ruin the friendship we already have.” Best of luck OP