More important things to do. This way of saying that you don't want to waste your time with something (or someone) goes back to 17th-century England. Its French equivalent is “other dogs to whip.” Variations are “other fish to fry” and “better fish to fry.”