there's (or here's) the rub that is the crucial difficulty or problem. literary

☞ This expression comes from Shakespeare's Hamlet: 'To sleep: perchance to dream: ay, there's the rub; For in that sleep of death what dreams may come When we have shuffled off this mortal coil, Must give us pause'. In the game of bowls, a rub is an impediment that prevents a bowl from running smoothly.

2004Cheryl HoltMore than Seduction But there's the rub, you see. I can't be linked to a scandal.