The actor said he "can't look back and go, 'Oh, what if?'" on the missed opportunities.