Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics (4/4) #572

Merged
merged 1 commit into from
Aug 25, 2021
Merged

Metrics (4/4) #572

merged 1 commit into from
Aug 25, 2021

Conversation

kuritka
Copy link
Collaborator

@kuritka kuritka commented Aug 23, 2021

related to #124

This PR contains several specific metrics

	Health records *prometheus.GaugeVec
	StatusPerIngressHosts *prometheus.GaugeVec
	StatusFailover *prometheus.GaugeVec
	StatusRoundRobin *prometheus.GaugeVec
	StatusGeoIP *prometheus.GaugeVec
	ZoneUpdateTotal *prometheus.CounterVec
	ErrorTotal *prometheus.CounterVec
	ReconciliationTotal prometheus.Counter

Demo

make destroy-full-local-setup deploy-full-local-setup upgrade-candidate deploy-prometheus
# wait
make stop-test-app
# wait
make start-test-app
# wait

k8gb_gslb_error_total

Screenshot 2021-08-19 at 14 38 53

Screenshot 2021-08-20 at 10 01 11

k8gb_gslb_healthy_records

Screenshot 2021-08-19 at 14 38 10

k8gb_gslb_reconciliation_total

Screenshot 2021-08-19 at 14 39 16

k8gb_gslb_status_failover

Screenshot 2021-08-19 at 14 44 53

Screenshot 2021-08-19 at 14 45 40

k8gb_gslb_status_roundrobin

Screenshot 2021-08-19 at 15 37 41

Screenshot 2021-08-19 at 15 38 00

k8gb_gslb_status_per_ingress_hosts

Screenshot 2021-08-19 at 14 49 21

Screenshot 2021-08-19 at 14 49 45

Infoblox metrics:

  • k8gb_infoblox_heartbeats_total
  • k8gb_infoblox_heartbeat_errors_total
  • k8gb_infoblox_zone_updates_total
  • k8gb_infoblox_zone_update_errors_total

Signed-off-by: kuritka [email protected]

k0da
k0da previously approved these changes Aug 24, 2021
Copy link
Collaborator

@k0da k0da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

somaritane
somaritane previously approved these changes Aug 24, 2021
@kuritka kuritka dismissed stale reviews from somaritane and k0da via b0c81e6 August 24, 2021 10:28
@kuritka kuritka force-pushed the prometheus-metrics-add branch from c96953b to b0c81e6 Compare August 24, 2021 10:28
k0da
k0da previously approved these changes Aug 24, 2021
Copy link
Member

@ytsarev ytsarev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rebase with master

@kuritka kuritka force-pushed the prometheus-metrics-add branch 4 times, most recently from 41468b4 to 5fb1cdb Compare August 24, 2021 13:22
@kuritka
Copy link
Collaborator Author

kuritka commented Aug 24, 2021

After today talk with @somaritane , @k0da I extended this PR with new infoblox metrics + tests:

  • k8gb_infoblox_heartbeats_total

  • k8gb_infoblox_heartbeat_errors_total

  • k8gb_infoblox_zone_updates_total

  • k8gb_infoblox_zone_update_errors_total

  • k8gb_gslb_zone_update_total - REMOVED

@somaritane , naming changed by your recommandations
@ytsarev , merged with master, I made CONTRIBUTION.md changes

@kuritka kuritka requested a review from ytsarev August 24, 2021 13:57
ytsarev
ytsarev previously approved these changes Aug 24, 2021
Copy link
Member

@ytsarev ytsarev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

return result.RequeueError(fmt.Errorf("error reading the object (%s)", err))
}

err = r.DepResolver.ResolveGslbSpec(ctx, gslb, r.Client)
if err != nil {
m.ErrorIncrement(gslb)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using VerbNoun naming for the function in order to retain the consistency with other functions in the code (e.g. RequeError()):

Suggested change
m.ErrorIncrement(gslb)
m.IncrementError(gslb)

Copy link
Collaborator Author

@kuritka kuritka Aug 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@somaritane , than would be nice to follow RequeueError patter, will rename all of them?

ErrorIncrement -> IncrementError
ReconciliationIncrement -> IncrementReconciliation
InfobloxZoneUpdateIncrement -> InfobloxIncrementZoneUpdate
InfobloxZoneUpdateErrorIncrement -> InfobloxIncrementZoneUpdateError
InfobloxHeartbeatIncrement -> InfobloxIncrementHeartbeat
InfobloxHeartbeatErrorIncrement -> InfobloxIncrementHeartbeatError

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@somaritane , amended for all increment funcs

Copy link
Contributor

@somaritane somaritane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kuritka looks good, just a question about strategy related status metrics

m.metrics.K8gbGslbFailoverStatus = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: K8gbGslbFailoverStatus,
Help: "K8GB Failover status.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can find a better description and maybe a name as well for this metric.
Because "K8GB Failover status." sounds a bit misleading to me, we're not talking about k8gb operator status here, also this is GaugeVec metric, not Scalar.
Maybe it should be "Gslb status count for Failover strategy"?
With "K8gbGslbStatusCountForFailover" and "k8gb_gslb_status_count_for_failover" respectively?

Same goes for Roundrobin and GeoIP metrics below

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@somaritane , amended, thx.

Screenshot 2021-08-25 at 11 06 21

related to #124

This PR contains several specific metrics
```gaolang
	Health records *prometheus.GaugeVec
	StatusPerIngressHosts *prometheus.GaugeVec
	StatusFailover *prometheus.GaugeVec
	StatusRoundRobin *prometheus.GaugeVec
	StatusGeoIP *prometheus.GaugeVec
	ZoneUpdateTotal *prometheus.CounterVec
	ErrorTotal *prometheus.CounterVec
	ReconciliationTotal prometheus.Counter
```

```shell
make destroy-full-local-setup deploy-full-local-setup upgrade-candidate deploy-prometheus
make stop-test-app
make start-test-app
```

![Screenshot 2021-08-19 at 14 38 53](https://user-images.githubusercontent.com/7195836/130069631-6ad73421-9a76-4327-a856-7873615fba86.png)

![Screenshot 2021-08-19 at 14 38 10](https://user-images.githubusercontent.com/7195836/130069543-f446fa68-5364-4ff3-a1de-9ffc7c9a8bc2.png)

![Screenshot 2021-08-19 at 14 39 16](https://user-images.githubusercontent.com/7195836/130069680-026e5ec4-f66a-4ea0-9ebd-6a6c5eaa2d60.png)

![Screenshot 2021-08-19 at 14 44 53](https://user-images.githubusercontent.com/7195836/130070434-712c9583-e176-46bd-9994-6059de5d64a1.png)

![Screenshot 2021-08-19 at 14 45 40](https://user-images.githubusercontent.com/7195836/130070537-d46e410b-6a8e-4d27-86ce-474b803a988c.png)

![Screenshot 2021-08-19 at 14 49 21](https://user-images.githubusercontent.com/7195836/130071022-b82942f8-e469-48b8-946a-583f49c54497.png)

![Screenshot 2021-08-19 at 14 49 45](https://user-images.githubusercontent.com/7195836/130071080-3b4a4d63-a86f-47c1-bc64-211beb819749.png)

![Screenshot 2021-08-19 at 14 50 26](https://user-images.githubusercontent.com/7195836/130071169-ddc065de-dd21-47a3-a35b-3f0a1be9c416.png)

Infoblox metrics are:
 - k8gb_infoblox_heartbeats_total
 - k8gb_infoblox_heartbeat_errors_total
 - k8gb_infoblox_zone_updates_total
 - k8gb_infoblox_zone_update_errors_total

Signed-off-by: kuritka <[email protected]>
@kuritka kuritka force-pushed the prometheus-metrics-add branch from dd119bc to 2fd01ee Compare August 25, 2021 09:05
Copy link
Contributor

@somaritane somaritane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kuritka LGTM!

@kuritka kuritka merged commit 07fb011 into master Aug 25, 2021
@kuritka kuritka deleted the prometheus-metrics-add branch August 25, 2021 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants