[doc]: High frequency counter HLD #1795
Conversation
Pterosaur
force-pushed
the
hfc
branch
3 times, most recently
from
bc0e247 to
c14e643
Compare
| ### Architecture Design | ||
|
|
||
| This section covers the changes that are required in the SONiC architecture. In general, it is expected that the current architecture is not changed. | ||
| This section should explain how the new feature/enhancement (module/sub-module) fits in the existing architecture. |
There was a problem hiding this comment.
same here. these "guidance" words are better to be removed. very confusing to read.
There was a problem hiding this comment.
Sorry, Yes, there are many template "guidance" words I haven't removed them.
|
|
||
| #### Modules #### | ||
|
|
||
| ##### Netlink Module ##### |
There was a problem hiding this comment.
i know the numbered prefix are not there in the template, but the doc will be much more readable with it. we should consider add them here.
There was a problem hiding this comment.
Hi @zhangyanzhao , Do you have any suggestions or concerns about adding a number prefix to each title in the template?
There was a problem hiding this comment.
Hi @zhangyanzhao , Do you have any suggestions or concerns about adding a number prefix to each title in the template?
Good suggestion. Updated the template. Github markdown does not support auto number yet, need community member to update it manually when creating HLD doc with the template.
|
|
||
| Pin CPU? | ||
|
|
||
|  |
There was a problem hiding this comment.
are we still uses the chunk size concept in our API?
There was a problem hiding this comment.
Or leverage the existing solution, time-based reporting?
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
Pterosaur
force-pushed
the
hfc
branch
8 times, most recently
from
290f7f9 to
0604b27
Compare
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
|
|
||
| - For high-frequency counters, the native IPFIX timestamp unit of seconds is insufficient. Therefore, we introduce an additional element, `observationTimeNanoseconds`, for each record to meet our requirements. | ||
| - The enterprise bit is always set to 1 for stats records. | ||
| - The element ID of IPFIX is derived from the object index. For example, for `Ethernet5`, the element ID will be `0x5 | 0x8000 = 0x8005`. |
There was a problem hiding this comment.
Having a variable element ID per SAI object will blow up the number of templates in the report (separate template ID per port, queue etc.).
There was a problem hiding this comment.
In my mind, I assume all stats(per port, queue etc) in one profile will be encoded into ONE template. I will generate a unique template ID for a profile.
|
|
||
| ### Data format | ||
|
|
||
| We will use IPFIX as the report format, with all numbers in the IPFIX message in network-order (Big-endian). |
There was a problem hiding this comment.
You cannot make the assumption that data will be Big- or Little-endian. If it is collected by the ASIC and DMAd to a ring buffer, then kernel driver would need to correct the endianness of each data set. We need to have a bit in the element ID or Enterprise number that would indicate endianness.
There was a problem hiding this comment.
Are you worried that endian conversion will consume too many CPU clocks?
| ``` | ||
| STREAM_TELEMETRY_GROUP|{{profile_name}}|{{group_name}} | ||
| "object_names": {{list of object name}} | ||
| "object_counters": {{list of stats of object}} |
There was a problem hiding this comment.
How do you plan to handle the update of the counter list or object list from the report POV? is it going to be a new template ID?
There was a problem hiding this comment.
Yes, I assume we will use a new template ID.
We treat any configuration update as a new profile. The previous session will be interrupted. The SAI report object will get a new SAI_TAM_REPORT_ATTR_REPORT_IPFIX_TEMPLATE_ID
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
|
/azp run |
|
No pipelines are associated with this pull request. |
|
/azp run |
|
No pipelines are associated with this pull request. |
|
/azp run |
|
No pipelines are associated with this pull request. |
Signed-off-by: Ze Gan <[email protected]> Move SAI section to OCP reference Signed-off-by: Ze Gan <[email protected]>
Signed-off-by: Ze Gan <[email protected]>
|
/azp run |
|
No pipelines are associated with this pull request. |
|
/azp run |
|
No pipelines are associated with this pull request. |
Signed-off-by: Ze Gan <[email protected]>
|
/azp run |
|
No pipelines are associated with this pull request. |
|
Hi @r12f , Please help to review or approve it if everything is good to you |
|
Looks good! But somehow I could not approve this due to a GitHub bug... |
| @@ -0,0 +1,647 @@ | |||
| # Stream telemetry high level design <!-- omit in toc --> | |||
There was a problem hiding this comment.
"Stream telemetry" is a term already used by sonic gnmi based telemetry implementation. Suggest you replace it with another term like "high frequency counters telemetry" (just an example).
Pterosaur
changed the title
Pterosaur
changed the title
High frequency counter HLD initial version