In Chronicles Dylan says, “Sometimes you know things have to change, are going to change , but you can only feel it . . .  but you don’t know it in a purposeful way.”  Maybe this is why the farewell in “Restless Farewell” is a restless one.  The speaker is uneasy, agitated, regretful, but finger-pointing, insinuating in his targets.  The “fast”/”past” rhyme, maybe the first time Dylan used it, helps express the break to come, the need for change, the inevitability of it:

Oh ev’ry girl that ever I’ve touched
I did not do it harmfully
And ev’ry girl that ever I’ve hurt
I did not do it knowin’ly
But to remain as friends
And make amends
You need the time and stay behind
And since my feet are now fast
And point away from the past
I’ll bid farewell and be down the line

Robert Shelton says that “[t]ime is crucial in this song.”  Time to go, time to leave the past behind, time to move on, time to cut ties, time to make the break. It’s also the last song of ten on this memorable album; maybe Bob was restless about leaving it and the work and joys that went behind creating it.

Dylan chose to sing this song at Frank Sinatra’s 80th birthday celebration, a restless farewell of sorts, to a man whose time was to come (he died at the age of 82).  Time goes by fast . . . the last time Dylan would sing this song live was near his own birthday, May 24, on May 21, 1998.  Birthdays are celebrations of our years.  “Restless Farewell” to them Dylan seems to be saying, so we can “point away from the past.”