Why does Linux answer to ARP on incorrect interfaces?

Why do you call the ARP reply “wrong”? The system’s IP address can certainly be reached via that interface. This is why the ARP reply is sent to begin with. Not doing so might cause some traffic to flow via less optimal paths, or not at all. For example, tap0 might be a VPN connection, and this ARP reply helps to ensure that traffic to the other IP address will flow correctly through the VPN.

If you really want to do this, you can set the sysctls arp_ignore and arp_announce to desired values.

arp_announce - INTEGER
  Define different restriction levels for announcing the local
  source IP address from IP packets in ARP requests sent on
  interface:
  0 - (default) Use any local address, configured on any interface
  1 - Try to avoid local addresses that are not in the target's
  subnet for this interface. This mode is useful when target
  hosts reachable via this interface require the source IP
  address in ARP requests to be part of their logical network
  configured on the receiving interface. When we generate the
  request we will check all our subnets that include the
  target IP and will preserve the source address if it is from
  such subnet. If there is no such subnet we select source
  address according to the rules for level 2.
  2 - Always use the best local address for this target.
  In this mode we ignore the source address in the IP packet
  and try to select local address that we prefer for talks with
  the target host. Such local address is selected by looking
  for primary IP addresses on all our subnets on the outgoing
  interface that include the target IP address. If no suitable
  local address is found we select the first local address
  we have on the outgoing interface or on all other interfaces,
  with the hope we will receive reply for our request and
  even sometimes no matter the source IP address we announce.

  The max value from conf/{all,interface}/arp_announce is used.

  Increasing the restriction level gives more chance for
  receiving answer from the resolved target while decreasing
  the level announces more valid sender's information.

And arp_ignore is described as:

arp_ignore - INTEGER
  Define different modes for sending replies in response to
  received ARP requests that resolve local target IP addresses:
  0 - (default): reply for any local target IP address, configured
  on any interface
  1 - reply only if the target IP address is local address
  configured on the incoming interface
  2 - reply only if the target IP address is local address
  configured on the incoming interface and both with the
  sender's IP address are part from same subnet on this interface
  3 - do not reply for local addresses configured with scope host,
  only resolutions for global and link addresses are replied
  4-7 - reserved
  8 - do not reply for all local addresses

  The max value from conf/{all,interface}/arp_ignore is used
  when ARP request is received on the {interface}

So, you would want to set arp_ignore to 1 (or possibly 2) and arp_announce to 2.

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

For testing, it is probably fine to do this. But a real production system will likely behave in the manner you experienced, and your program needs to be able to deal with that.