Line-by-line code profiling in Python

Oct 1st, 2015 in  by Michael Cho

How to find slow sections in your Python code.

An easy way to find the slow sections of your python code.

Add the profiler library


pip install line_profiler

Add @profile decorator

Above the function to profile, e.g.:


@profile
def parse_teams_cache(teams, include_search=False, user=None):
    team_items = []
    .... etc ...

Enable the function / file to be called from the command line

By adding something like the following to the bottom of the file:


if __name__ == "__main__":
    teams = Group.leaders()
    user = User.query.get(5347)
    parse_teams_cache(teams, include_search=False, user=user)

Run the profiler


kernprof -l -v pages/helpers/team_helper.py

Results

Results show the slowest code is in loading the cache, ie:


Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   143                                           @profile
   144                                           def parse_teams_cache(teams):
   145         1            2      2.0      0.0      team_items = []
   146        27           27      1.0      0.0      for team in teams:
   147        26          185      7.1      0.2          team_cache_key = ...
   148        26        99097   3811.4     98.5          team_cache = ...
   149        26           47      1.8      0.0          if team_cache:
   150        26         1154     44.4      1.1              team_item = ...


Other articles you may like

Method delegation in Python
Jul 11th, 2018
Using Python enums in SQLAlchemy models
May 16th, 2018
Python command-line scripts with argparse
Feb 15th, 2018
SQLAlchemy commit(), flush(), expire(), refresh(), merge() - what's the difference?
Nov 2nd, 2017
Prioritized Code Review Checklist - what to look for first, second, and last
Sep 21st, 2017