If the control navigates to a URL, use an anchor. If the control performs an action, use a button. Get this right and middle-click, copy-link, and right-click menus all behave correctly.
Style does not change the semantics — a styled button can look like a link, an anchor can look like a button. Pick the element that matches the behaviour.
The shortcut: ‘Can you bookmark this destination?’ If yes, it is a link.