God was angry because Balaam went when He had specifically told him, "Don't go unless they come to you and ask you." Nothing in God's Word says that they did. Instead, it says that Balaam got up in the morning and saddled his donkey, and off he went.
God gave conditional permission. The condition was only if he was asked again, but he was not asked again yet went anyway. Balaam was one of those people who, if you give him an inch, he takes a mile. If he was not specifically told, "You shall not go," then he thought that meant he could go ahead and leave.
In like manner, there are those who think, "Well, because the Bible does not say 'Thus saith the Lord,' it is okay!" We can see many things in Balaam's character that are similar to what many people today mimic due to the fact that they are not listening to God either. God was very specific with Balaam, but all he heard was, "Go ahead!" He tuned out the part that began with if.
This is why God was angry with him. He was so angry that He came out against him, to stand in his way. Maybe the most intriguing detail here is that the word adversary is, in Hebrew, satan, which means generally "adversary, enemy, foe." God came out against Balaam the same way that Satan comes out against us, when God allows him to do so. God set Himself up as Balaam's enemy.
In reality, by leaving without fulfilling the conditions, Balaam chose to join Satan's side. God, then, visibly to the donkey but invisibly to Balaam, set Himself up as the adversary to Balaam.
Balaam showed God that he would do what Balak wanted him to do. In counterpoint, God will do something to try to get Balaam to change, to turn. God does not come out against Balaam as a normal enemy would—to do him harm—but to turn him around and give him a chance to repent. But Balaam would have nothing to do with that. He had set himself up as an enemy of God, and he never turns himself around.