<Devtools/> dailyToolsCheatsheetsBlog
Ctrl + K

Building gradle dependency graph using Python and GraphViz

2023-03-03

Problem

In this article we'll demonstrate how to build graph visualization for a gradle project structure, to see which modules depend on other modules.

Some Gradle projects can become big and it becomes hard to make sense of the structure, by visualizing it you can easily see which modules are unnecessary.

Let's say we have a project with 3 subprojects inside api, shared and services (from https://docs.gradle.org/current/userguide/declaring_dependencies_between_subprojects.html)

api/build.gradle

dependencies {
    implementation project(':shared')
}

shared/build.gradle

dependencies {
  implementation "some:opensource:2.13"
}

services/build.gradle

dependencies {
    implementation project(':shared')
    implementation project(':api')
}

Implementation

Here is a task break down

  1. Python Command line tool that takes project location as input
  2. Find all Gradle files and their module names
    • We'll scan a directory for all build.gradle files
    • Project name will always be a parent folder name
  3. We'll parse each gradle file looking for some keywords like "project" and <other module name>
  4. Construct a structure {name -> [dependencies]}
  5. Convert that structure into DOT markup
  6. Preview result

Create python command line tool

this is fairly standard:

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)

output

python3 generate_graph.py /Users/devtoolsdaily/src/gradleproj
checking project:  /Users/devtoolsdaily/src/gradleproj

Find all Gradle files and their module names

for that we'll use pathlib.Path.rglob function that can find all files by name, then we'll parse their parent folder using os.path.split

def find_modules(project_path):
  project_to_gradle_file = {}
  for full_path in pathlib.Path(project_path).rglob("build.gradle"):
      folder = os.path.split(full_path)[0]
      if folder == project_path:
          # skipping root build.gradle
          continue
      module_name = os.path.split(folder)[1]
      # initializing our result with empty dependencies first
      module_to_gradle_file[module_name] = full_path
  return module_to_gradle_file

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)

  modules_to_gradle_files = find_modules(project_path)
  print(modules_to_gradle_files)

output

{'shared': PosixPath('/Users/devtoolsdaily/src/gradleproj/shared/build.gradle'),
'api': PosixPath('/Users/devtoolsdaily/src/gradleproj/api/build.gradle'),
'services': PosixPath('/Users/devtoolsdaily/src/gradleproj/services/build.gradle')}

We'll parse each gradle file looking for some keywords like "project" and <other module name>

for each line in gradle file, we'll check for other project names and for keyword project

def find_internal_dependencies(gradle_file_path, all_modules):
  dependencies = []
  with open(gradle_file_path) as f:
      for line in f:
          print(line)
          for module in all_modules:
              if module in line and "project" in line:
                  dependencies.append(module)
  return dependencies

We'll construct a final structure out of that

if __name__ == "__main__":
  project_path = sys.argv[1]
  print("checking project: ", project_path)
  modules_to_gradle_files = find_modules(project_path)
  
  module_dependencies = {}
  for module_name, gradle_file in modules_to_gradle_files.items():
      dependencies = find_internal_dependencies(gradle_file, modules_to_gradle_files.keys())
      module_dependencies[module_name] = dependencies
  print(module_dependencies)

that outputs

{'shared': [], 'api': ['shared'], 'services': ['api', 'shared']}

Convert that structure into DOT markup

def convert_to_dot(module_dependencies):
  result = ["digraph {"]
  for module, dependencies in module_dependencies.items():
      for dependency in dependencies:
          result.append(f"  {module} -> {dependency}")
  result.append("}")
  return "\n".join(result)

digraph {
  api -> shared
  services -> api
  services -> shared
}

Preview result

Paste the output into Online Graphviz Playground

the result should look like this result

How to get started with Graphviz

  1. read the official documentation
  2. read our getting started with python and graphviz guide
  3. play with graphviz examples in our Graphviz playground
  4. follow the interactive tutorial to learn Graphviz
FAQContact Us