The MX records for your domain name determine the names of your incoming email servers (whether or not they also handle outgoing email is completely irrelevant) as well as the priority that determines in which order a sender should be trying them.
The A and AAAA records for those email server names provide the IPv4 and IPv6 addresses that are actually used to establish the connection to the email server. In addition the A and AAAA records are used as a fallback if a domain name doesn't have any MX records associated with it.
The same applies in the outgoing direction with the important difference being that it is the MX, A and AAAA records of the destination domain that matter instead of your own DNS records. -- Got some spare cpu cycles ? Join Team Helix or Team Starfire!