Transformer Infra memory optimization and enhancements #80
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s for following: *Transformer infra memory and startup time optimizations *Support for gNMI style PUT/REPLACE on leaf-list. *Sort DB keys for tables having mutli-element key with optional elements(sonic-yang container representing the table has multiple lists) before DB operation. *Modifications to improve debuggability/logging.
…cations to incorporate following: *Support gNMI style PUT/REPLACE on leaf-list in SET flow *Conditionally invoke subtree only once to improve SET request performance *Support for key-transformer on sonic-yang GET request. *Improve debuggability/logging.
*Add utility functions in transformer infra for CRUD AND GET cases to retrieve yang node info *Key-transformer support on Sonic-Yang list for GET flow *Call back error message packaging *Changes to Improve debuggability and logging
*Json based table sorting for non CVL table ordering *Changes to Improve debuggability and logging.
…n_app to handle leaf-list target PUT/REPLACE to swap contents of leaf-list.
Bring in transformer memory optimization and enhancements into branch created from community master
kwangsuk
approved these changes
May 4, 2023
ayelrod
reviewed
May 18, 2023
ndas7
approved these changes
May 23, 2023
tomek-US
approved these changes
May 23, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What are the changes
1.) Transformer infra memory usage & startup time optimizations –
• optimize yang annotations caching at startup/yang load time by avoiding cache members that can be derived from other members .
• Move dumping of debug files containing transformer specifications for yang-data translation to on-demand signal based debug files dump.
• Use of optimized yang schema pre-generated by ygot and goyang during ocbinds build time instead of loading all the sonic and openconfig yang modules and generating the full yang schema on the run for the transformer infra use during initialization/startup.
2) Improve debuggability and logging
3) Enhance transformer Infra to support key-transformer on Sonic-Yang list for GET flow to give control to apps to generate Sonic-yang list key-leaf values as per the app need, when the Db-key string has values containing Db key separator.
4) Infra enhancement for PUT/REPLACE on leaf-list, to support as per GNMI Spec to swap contents of leaf-list.
5) Infra optimization in SET flow to conditionally invoke subtree only once at parent node and skip for child nodes during infra traversal of payload.
6) Infra support to restrict top-level yang container REPLACE operation.
7) Infra enhancement to Sort DB keys for tables having more than one list entry(leafref from key leaf of one list to keyleaf of another list in same table/container) in sonic yang during DELETE operation
8)Json based table sorting for non CVL table ordering for DELETE flow
Why the changes are required
To support openconfig yang to sonic yang translation through the common app. Includes optimizations and enhancements based on applications requirements when integrating with transformer infra.
How did I verify the changes
Used a test transformer yang both in openconfig style and sonic style along with transformer annotations and callbacks to exercise unit test cases to perform translation on the test yang.
The results of the unit tests exercised are available in the PR #81 . Unit test infra and cases are a part this PR.