Is your load-balancer looking at the healthcheck endpoint on the brokers?
Default port 5550 for non-TLS, 5553 for TLS:
HEALTHCHK TCP N N - Mgmt 5550 U U HEALTHCHK TCP Y N - Mgmt 5553 U D No Cert