It almost did happen. Last January as Trump was scrambling to hold on to the presidency, he considered refusing to leave the White House. Then he was told that once Biden was sworn in, the Secret Service would physically remove him.