Code Quality Improvements Needed in LG._link_drops Function

[sourcery-ai]

The Sourcery AI bot has identified several code quality issues in the LG._link_drops function that need to be addressed to improve the overall quality and maintainability of the codebase. The issues found include:

1. Replace identity comprehension with call to collection constructor: This involves converting list/set/tuple comprehensions that do not change the input elements into direct calls to collection constructors. For example, instead of using [item for item in coll], use list(coll).

2. Use named expression to simplify assignment and conditional: This can help in reducing redundancy and improving readability by using the new assignment expression feature (the walrus operator :=).

3. Simplify dictionary access using default get: This involves using the get method with a default value to simplify dictionary access and avoid potential KeyError exceptions.

4. Low code quality in LG._link_drops: The function has a quality score of 13%, which is below the acceptable threshold of 25%. This score is based on factors such as method length, cognitive complexity, and working memory.

Suggested Actions:

• Refactor the LG._link_drops function to make it shorter and more readable. Aim to reduce the function length by extracting pieces of functionality into their own functions.
• Reduce nesting by introducing guard clauses to return early where possible.
• Ensure that variables are tightly scoped, so that related code is grouped together within the function.

These improvements should be addressed as part of a larger code quality improvement effort. A separate ticket will be raised to track these changes and ensure they are implemented effectively. This will help in maintaining a clean and efficient codebase.

For more details on the specific issues, refer to the Sourcery documentation.

---

I created this issue for @myxie from #300 (comment).

Tips and commands

Interacting with Sourcery

• Generate a plan of action: Comment @sourcery-ai plan on this issue.
• Generate a pull request for this issue: Comment @sourcery-ai develop to
  generate a PR that addresses this issue.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[myxie]

@sourcery-ai develop

[sourcery-ai]

Objectives

• Refactor LG._link_drops function to improve its code quality score above 25%
• Replace identity comprehensions with direct collection constructor calls
• Implement walrus operators (:=) to simplify assignment and conditional statements
• Simplify dictionary access using .get() method with default values
• Extract complex logic into smaller, focused helper functions to reduce cognitive complexity
• Reduce nesting depth by introducing guard clauses for early returns
• Group related code together to improve variable scoping
• Ensure changes comply with project's Python coding standards using black formatter
• Verify changes pass pylint checks with score above 9
• Add unit tests for any new helper functions created during refactoring

File changes

daliuge-engine/dlg/dask_emulation.py

• Replace list comprehension in _Listifier.run() with direct constructor call
• Extract complex linking logic from _DelayedDrop._add_upstream() into helper methods
• Add guard clauses in _DelayedDrop._add_upstream() for early returns
• Simplify dictionary access in _AppDrop.make_dropdict() using .get() with defaults
• Use walrus operator in _DelayedDrop._to_physical_graph() for cleaner dependency handling
• Extract graph building logic from _DelayedDrop.get_graph() into a separate method
• Simplify producer/pydata validation in _DataDrop.__init__() using walrus operator
• Extract complex serialization logic from _AppDrop.__call__() into helper methods
• Reduce nesting in compute() function by introducing guard clauses
• Group related code in _AppDrop.__call__() by moving argument processing to separate method

Verify changes

• black daliuge-engine/dlg/dask_emulation.py
• pylint daliuge-engine/dlg/dask_emulation.py --fail-under=9
• pytest daliuge-engine/test/test_dask_emulation.py -v

---

This plan was automatically generated.
Please review the plan carefully and make any necessary adjustments.