let (one) pass by
To allow one to pass by or around (someone or something). I need to get into the building to retrieve my briefcase—please let me pass by! I didn't want to drive any faster on the highway, so I moved closer to the curb to let the cars behind me pass by.