It's been billions of years since Pangea brook apart. Think of all of the forces that have acted upon those continents since. Erosion from water is a big part. Islands have broken off; earthquakes have shifted continents; glaciers have formed, mountains have formed by continents being smashed together. It wasn't a clean cut. Imagine having week old birthday cake left overs. Already when it was fresh and you cut it, crumbs fell off and the cutting wasn't neat. Now a week later as you're trying to pick up an already cut piece, the frosting is falling off and taking pieces of cake with it and it's crumbling. It isn't a perfect fit because time has altered the shapes of the continents.
