Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds FSIO R/W sync script for stormond implementation #18927

Closed

Conversation

assrinivasan
Copy link
Contributor

@assrinivasan assrinivasan commented May 9, 2024

Why I did it

This is part of a larger feature implementation: Storage Monitoring Daemon. It adds a script to sync the FSIO reads and writes from the STATE_DB to the disk, which is called by the various reboot scripts during planned reboot.

Work item tracking
  • Microsoft ADO (number only): 17468992

How I did it

  1. Added the sync script to image_config folder

  2. Made changes in sonic_debian_extension.j2 that copies the above file to the appropriate location in the image.

  3. Made changes that instruct the build_template to copy 'fsio-rw-sync' into the appropriate location in the pmon container, so that the script can be called by the daemon from within pmon.

How to verify it

  1. Flash image with these changes onto a DUT

  2. Write sample values of the appropriate fields to STATE_DB

  3. Erase the contents of the fsio-rw-stats.json file and replace with empty JSON

  4. Reboot the device with this sister sonic-utilities change: Modified reboot scripts to sync FSIO reads/writes to disk before OS-level reboot sonic-utilities#3171

  5. On reboot, verify that the values from Step 2 are in the JSON file.

if the sonic-utilities PR is not yet merged, you can verify the presence of the sync script in /usr/local/bin and call it manually, after which you can verify that the empty JSON file has been populated with the contents of the STATE_DB values we previously posted to the database.

root@str3-s6100-acs-6:~# redis-cli -n 6 HSET "STORAGE_INFO|sda" fs_io_reads 1000 ; redis-cli -n 6 HSET "STORAGE_INFO|sda" fs_io_writes 2000 ; redis-cli -n 6 HSET "STORAGE_INFO|sda" latest_fs_io_reads 10 ; redis-cli -n 6 HSET "STORAGE_INFO|sda" latest_fs_io_writes 20
(integer) 1
(integer) 1
(integer) 1
(integer) 1
root@str3-s6100-acs-6:~# echo "{}" > /host/pmon/stormond/fsio-rw-stats.json
root@str3-s6100-acs-6:~#
root@str3-s6100-acs-6:~#
root@str3-s6100-acs-6:~#
root@str3-s6100-acs-6:~# sudo reboot
requested COLD shutdown
/host: 9.3 GiB (9988685824 bytes) trimmed on /dev/sda4
Thu 09 May 2024 06:50:47 PM UTC Issuing OS-level reboot ...
root@str3-s6100-acs-6:~# client_loop: send disconnect: Broken pipe
assrinivasan@assrinivasan-dev-vm-1:/data/sonic$ ash str3-s6100-acs-6
10.3.146.155
Warning: Permanently added '10.3.146.155' (RSA) to the list of known hosts.
Linux str3-s6100-acs-6 5.10.0-23-2-amd64 #1 SMP Debian 5.10.179-3 (2023-07-27) x86_64
You are on
  ____   ___  _   _ _  ____
 / ___| / _ \| \ | (_)/ ___|
 \___ \| | | |  \| | | |
  ___) | |_| | |\  | | |___
 |____/ \___/|_| \_|_|\____|

-- Software for Open Networking in the Cloud --

.
.
.
admin@str3-s6100-acs-6:~$ cat /host/pmon/stormond/fsio-rw-stats.json
{"sda": {"fs_io_reads": "1000", "fs_io_writes": "2000", "latest_fs_io_reads": "10", "latest_fs_io_writes": "20"}}admin@str3-s6100-acs-6:~$

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211
  • 202305

Tested branch (Please provide the tested image version)

  • 20230531.22

Description for the changelog

Added logic to sync stormond STATE_DB values to disk in planned reboot scenarios

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

A service 'fsio-sync' that calls a shell script 'fsio-sync.sh' that
parses the latest fsio reads/writes from /proc/diskstats and saves it
to disk using 'fsio-rw-sync' script, as well as updates the StateDB
upon successful parse and sync.

Changes that copy these files to the appropriate locations in the image.

Changes that instruct the build_template to copy 'fsio-rw-sync' into
the appr. location in the pmon container
… boot

Changed fsio-sync.sh so that sleep interval can by dynamically changed per loop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant