Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken stat scheduler_loop_duration #42886

Merged
merged 3 commits into from
Oct 17, 2024
Merged

Conversation

0vj00
Copy link
Contributor

@0vj00 0vj00 commented Oct 10, 2024

The stat scheduler_loop_duration is broken since this change. The nesting of the stats context statement results in it not capturing the full scheduler loop duration. This PR fixes the nesting.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@0vj00 0vj00 requested review from ashb and XD-DENG as code owners October 10, 2024 04:52
@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Oct 10, 2024
Copy link

boring-cyborg bot commented Oct 10, 2024

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

@0vj00
Copy link
Contributor Author

0vj00 commented Oct 10, 2024

@howardyoo - I'd appreciate if you can review as well, since this follows up on your original PR #40802

@howardyoo
Copy link
Contributor

howardyoo commented Oct 10, 2024 via email

@potiuk
Copy link
Member

potiuk commented Oct 13, 2024

I approved the run - @howardyoo, would be great to get the review. Also - if we could have some unit test for that (can we?) it would be great.

@howardyoo
Copy link
Contributor

howardyoo commented Oct 14, 2024

I approved the run - @howardyoo, would be great to get the review. Also - if we could have some unit test for that (can we?) it would be great.

I don't think this can be unit-tested..?
The fix seems to be related to some changes that happened outside of Airflow OTEL implementation, but the code change makes sense. So I'd say let's approve and merge it.

@potiuk potiuk merged commit 60b8056 into apache:main Oct 17, 2024
52 checks passed
Copy link

boring-cyborg bot commented Oct 17, 2024

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
@sunchao
Copy link
Member

sunchao commented Oct 29, 2024

@potiuk @0vj00 any chance this can be added to Airflow 2.10.3 release? we are seeing the same issue as well

@potiuk potiuk added this to the Airlfow 2.10.4 milestone Oct 29, 2024
@potiuk
Copy link
Member

potiuk commented Oct 29, 2024

@potiuk @0vj00 any chance this can be added to Airflow 2.10.3 release? we are seeing the same issue as well

No. Not unless 2.10.3 rc1 is cancelled and we attempt to do 2.10.3rc2 (cc: @utkarsharma2 @ephraimbuddy ) -> I marked it provisionally for 2.10.4 - next time it's a good idea to mention it before we prepare a new release, that things are good candidates and would be great to back-port.

@sunchao
Copy link
Member

sunchao commented Oct 29, 2024

I see, thanks @potiuk

potiuk pushed a commit to potiuk/airflow that referenced this pull request Oct 31, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)
@potiuk potiuk modified the milestones: Airlfow 2.10.4, Airflow 2.10.3 Oct 31, 2024
@potiuk
Copy link
Member

potiuk commented Oct 31, 2024

We are going to have rc2 - > backporting that one in #43544

potiuk added a commit that referenced this pull request Oct 31, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)

Co-authored-by: Venkat VJ <[email protected]>
@utkarsharma2 utkarsharma2 added the type:bug-fix Changelog: Bug Fixes label Nov 1, 2024
utkarsharma2 pushed a commit that referenced this pull request Nov 1, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)

Co-authored-by: Venkat VJ <[email protected]>
(cherry picked from commit 842c60a)
utkarsharma2 pushed a commit that referenced this pull request Nov 1, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)

Co-authored-by: Venkat VJ <[email protected]>
(cherry picked from commit 842c60a)
utkarsharma2 pushed a commit that referenced this pull request Nov 1, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)

Co-authored-by: Venkat VJ <[email protected]>
(cherry picked from commit 842c60a)
utkarsharma2 pushed a commit that referenced this pull request Nov 1, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
(cherry picked from commit 60b8056)

Co-authored-by: Venkat VJ <[email protected]>
(cherry picked from commit 842c60a)
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
* wip

* wip

* fix lint err

---------

Co-authored-by: venkat <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Scheduler including HA (high availability) scheduler type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants