Why are you adding a favicon in your template?
Joomla automatically detects the favicon from /templates/yourtemplate/favicon.icon or /favicon.icon.

@Bakual its done by many templates as a way of allowing the user to load their template from the admin

else why would J! even have that function?

Before commenting everyone should read this helpful article which explains a lot about the intricacies of this subject as there is a lot more to it than a lot of people understand!: http://www.jonathantneal.com/blog/understand-the-favicon/

Having multiple favicon's is perfectly acceptable. However can you provide an example in code where you are trying to replace the default favicon with your own and that Joomla then just adds yours as well as the default? Is that what you are saying the "issue" is?

put your favicon in the template root folder, then in the index.php call the load favicon statement.
you would run into this if your template provides for you to supply your favicon in the admin as a param, and they have their ( or any ) favicon in the template root.

What will happen is the param favicon will load first ( via $this->addFavicon(JURI::root() . $favicon); ) and then the favicon in the template root will be loaded second. You can clearly see this with the element inspector.

As Phil says loading multiple favicons is actually required for use cases and the addFavicon is used for that purpose. The same URL favicon can't be loaded twice - so you are loading different URL's for your favicon. There's nothing that can be done in that use case other than ensuring that favicon.ico is deleted properly.