Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Route53 support #172

Merged
merged 11 commits into from
Oct 15, 2020
Merged

Route53 support #172

merged 11 commits into from
Oct 15, 2020

Conversation

ytsarev
Copy link
Member

@ytsarev ytsarev commented Oct 15, 2020

This PR adds support for AWS Route53 as EdgeDNS.

Notable differences comparing on-prem setup with Infoblox:

  • GSLB Zone Delegation is fully controlled by external-dns through DNSEndpoint CRDs which are dynamically created by k8gb controller.
  • Upstream PR with NS Record support Add NS record support kubernetes-sigs/external-dns#1813 is pending so we are running our own external-dns build for a time being
  • Single DNSEndpoint for NS record in k8gb namespace and also single NS record per k8gb multicluster setup(also simplifies Infoblox based setup)
  • CoreDNS 53/udp is exposed via NLB instead of nginx-controller based exposure on each worker node
  • Workloads/workers in the reference setup are also behind NLB so k8gb now capable to resolve LoadBalancer Hostname in Ingress status and populate Gslb enabled DNS record accordingly

Note:
TXT based split brain control is skipped for now - the mechanism might be rethought given we are moving from direct API calls to edgeDNS to full utilization of external-dns capabilities ( Custom TXT creation is also lacking there as of now )

* We have common delegated zone per k8gb clusters deployment
* Multiple Gslb objects should not override delegated zone
  config with different NS server names
* Thus consolidate the NS server names over multiple Gslbs
* We don't need to duplicate NS records over multiple k8gb instances
  as it is DNS zone wide configuration common for all Gslb objects
* Drawback: multiple Gslb objects will be writing the same data
  into this DNSEndpoint
* Advantage: we avoid creating additional Reconciliation loop
  and additional configuration CRD to handle this situation
Additionally simplify external-dns params
* Introduce Dig function
* Resolve AWS NLB fqdn which is associated with
  coredns service exposure
* Populate NS Server A record with resolution results
To avoid potential collisions and to be more clear
and explicit in NS server FQDN
* Fixing full local setup
* Fixing Terratest
Copy link
Collaborator

@k0da k0da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

reconcileRequeueSeconds: 30

externaldns:
image: k8s.gcr.io/external-dns/external-dns:v0.7.4
image: absaoss/external-dns:v0.7.1-401-gede9767c # temporary image containing NS record support before the merge of https://github.com/kubernetes-sigs/external-dns/pull/1813
interval: "20s"
expose53onWorkers: true # open 53/udp on workers nodes with nginx controller
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably can use known port name instead of number, like exposeUdpOnWorkers

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 cat /etc/services |grep 53/udp|head
domain           53/udp     # Domain Name Server

53 is more clear and explicit imho. Like in Route53 naming itself lol

@ytsarev ytsarev merged commit 5749966 into master Oct 15, 2020
@ytsarev ytsarev deleted the delegate branch October 15, 2020 14:00
@ytsarev ytsarev mentioned this pull request Oct 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants