Dean wasn't really sure want he was trying to accomplish when he went to go say yes to Michael.
It could have been a number of things. It could have been that he want the angels on his side for once. He could have wanted to punish his brother in some stupid, self-destructing way. It could have been that he just wanted to end. But he knew deep down that the answer was much darker.
He just wanted it to end. All of it. He wanted someone, anyone to take control, whether it be Lucifer or Michael, he didn't want to be around to see the end result. He wanted to die.
He tried so hard to push the thought down. He tried to be optimistic about getting rid of the devil and avoiding the angels, but he couldn't help the small glimmer of relief he had knowing that giving Michael his consent was always an option.
The thing that really scared him however, was the fact that he thought about Sam not more than three seconds before he made his decision. Do I care that this will hurt Sam? No. He could care less that Sam, the guy who now insisted they sleep in the same bed with Dean being the little spoon, would feel absolutely devastated and betrayed if he said yes to Michael. But that didn't matter. Not anymore.
If Castiel hadn't come when he had, he would have gone through with it.