Line-by-line code profiling in Python

Oct 1st, 2015 in  by Michael Cho

How to find slow sections in your Python code.

An easy way to find the slow sections of your python code.

Add the profiler library


pip install line_profiler

Add @profile decorator

Above the function to profile, e.g.:


@profile
def parse_teams_cache(teams, include_search=False, user=None):
    team_items = []
    .... etc ...

Enable the function / file to be called from the command line

By adding something like the following to the bottom of the file:


if __name__ == "__main__":
    teams = Group.leaders()
    user = User.query.get(5347)
    parse_teams_cache(teams, include_search=False, user=user)

Run the profiler


kernprof -l -v pages/helpers/team_helper.py

Results

Results show the slowest code is in loading the cache, ie:


Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   143                                           @profile
   144                                           def parse_teams_cache(teams):
   145         1            2      2.0      0.0      team_items = []
   146        27           27      1.0      0.0      for team in teams:
   147        26          185      7.1      0.2          team_cache_key = ...
   148        26        99097   3811.4     98.5          team_cache = ...
   149        26           47      1.8      0.0          if team_cache:
   150        26         1154     44.4      1.1              team_item = ...


Other articles you may like

SQLAlchemy commit(), flush(), expire(), refresh(), merge() - what's the difference?
Nov 2nd, 2017
Prioritized Code Review Checklist - what to look for first, second, and last
Sep 21st, 2017
Many to many relationships in SQLAlchemy models (Flask)
Jul 28th, 2017
Bash script to relink alembic migrations
Jun 12th, 2017
Using model callbacks in SQLAlchemy to generate slugs
Feb 7th, 2017