This song is exclusively a series of  terminal rhymed couplets, aa/bb all the way through, except for the end of the verse word “bully,” appearing 11 times, not once rhymed.  I see it as a thematic anti-rhyme, bullying its way through the song having the final say 11 times in conflict with the rhymes that dominate.  It’s tough being the misunderstood bully, Dylan expresses throughout it, and what’s so tough about it is echoed in the discordance of “bully” vs the world of the rhymes living in the song.

I also would like to give attention to the thematic rhyme in verse 10, “for”/”war” appearing in lines that label the bully as a cause for war:

What’s anybody indebted to him for?
Nothin’, they say. He just likes to cause war

Here’s the studio cut from Infidels with lyrics:

http://www.popmodal.com/video/17195/Neighborhood-Bully-Bob-Dylan-Regarding-Israel–30th-Anniversary

Well, the neighborhood bully, he’s just one man
His enemies say he’s on their land
They got him outnumbered about a million to one
He got no place to escape to, no place to run
He’s the neighborhood bully

The neighborhood bully just lives to survive
He’s criticized and condemned for being alive
He’s not supposed to fight back, he’s supposed to have thick skin
He’s supposed to lay down and die when his door is kicked in
He’s the neighborhood bully

The neighborhood bully been driven out of every land
He’s wandered the earth an exiled man
Seen his family scattered, his people hounded and torn
He’s always on trial for just being born
He’s the neighborhood bully

Well, he knocked out a lynch mob, he was criticized
Old women condemned him, said he should apologize.
Then he destroyed a bomb factory, nobody was glad
The bombs were meant for him. He was supposed to feel bad
He’s the neighborhood bully

Well, the chances are against it and the odds are slim
That he’ll live by the rules that the world makes for him
’Cause there’s a noose at his neck and a gun at his back
And a license to kill him is given out to every maniac
He’s the neighborhood bully

He got no allies to really speak of
What he gets he must pay for, he don’t get it out of love
He buys obsolete weapons and he won’t be denied
But no one sends flesh and blood to fight by his side
He’s the neighborhood bully

Well, he’s surrounded by pacifists who all want peace
They pray for it nightly that the bloodshed must cease
Now, they wouldn’t hurt a fly. To hurt one they would weep
They lay and they wait for this bully to fall asleep
He’s the neighborhood bully

Every empire that’s enslaved him is gone
Egypt and Rome, even the great Babylon
He’s made a garden of paradise in the desert sand
In bed with nobody, under no one’s command
He’s the neighborhood bully

Now his holiest books have been trampled upon
No contract he signed was worth what it was written on
He took the crumbs of the world and he turned it into wealth
Took sickness and disease and he turned it into health
He’s the neighborhood bully

What’s anybody indebted to him for?
Nothin’, they say. He just likes to cause war
Pride and prejudice and superstition indeed
They wait for this bully like a dog waits to feed
He’s the neighborhood bully

What has he done to wear so many scars?
Does he change the course of rivers? Does he pollute the moon and stars?
Neighborhood bully, standing on the hill
Running out the clock, time standing still
Neighborhood bully