PyQt excepthook

Written by Michael

Today I learned the hard way that exception tracebacks are sometimes cut off by PyQt. Say you have a Python function f() that calls a Qt function g() which in turn ends up calling another Python function of yours, h(). If an error happens in h(), you'd expect to get the following traceback:

Traceback (most recent call last):
  File ..., line ..., in f()
  File ..., line ..., in g()
  File ..., line ..., in h()
Exception

Instead, you only get:

Traceback (most recent call last):
  File ..., line ..., in g()
  File ..., line ..., in h()
Exception

The fact that f() is missing can sometimes make debugging very difficult.

The following code can be used to reproduce the problem:

from PyQt5.QtWidgets import *
from PyQt5.QtCore import Qt

class Window(QWidget):
    def __init__(self):
        super().__init__()
        b1 = QPushButton('1')
        b2 = QPushButton('2')
        b1.clicked.connect(self.f1)
        b2.clicked.connect(self.f2)
        layout = QVBoxLayout(self)
        layout.addWidget(b1)
        layout.addWidget(b2)
        self.setLayout(layout)
    def f1(self, _):
        self.inputMethodQuery(Qt.ImAnchorPosition)
    def f2(self, _):
        self.inputMethodQuery(Qt.ImAnchorPosition)
    def inputMethodQuery(self, query):
        if query == Qt.ImCursorPosition:
            self.h()
        else:
            return super().inputMethodQuery(query) # Call 'g()'
    def h(self):
        raise Exception()

app = QApplication([])
window = Window()
window.show()
app.exec()

When you run it, you get the following app:

Clicking on any of the two buttons results in the error below. Note how we don't get any indication as to which button was clicked:

Traceback (most recent call last):
  File "<stdin>", line 19, in inputMethodQuery
  File ""<stdin>", line 21, in h
Exception

If however you also type in the following code:

import sys
import traceback

from collections import namedtuple

def excepthook(exc_type, exc_value, exc_tb):
    enriched_tb = _add_missing_frames(exc_tb) if exc_tb else exc_tb
	# Note: sys.__excepthook__(...) would not work here.
	# We need to use print_exception(...):
    traceback.print_exception(exc_type, exc_value, enriched_tb)

def _add_missing_frames(tb):
    result = fake_tb(tb.tb_frame, tb.tb_lasti, tb.tb_lineno, tb.tb_next)
    frame = tb.tb_frame.f_back
    while frame:
        result = fake_tb(frame, frame.f_lasti, frame.f_lineno, result)
        frame = frame.f_back
    return result

fake_tb = namedtuple(
    'fake_tb', ('tb_frame', 'tb_lasti', 'tb_lineno', 'tb_next')
)

sys.excepthook = excepthook

Then you get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 13, in f1
  File "<stdin>", line 18, in inputMethodQuery
  File "<stdin>", line 19, in inputMethodQuery
  File "<stdin>", line 21, in h
Exception

Note how the extra output indicates which button was clicked.

This requires further explanation. The reason why Qt's inputMethodQuery(...) was used is that it is a virtual function. This lets us override it from Python. When you look at Qt's source code, you find that inputMethodQuery(ImAnchorPosition) results in a recursive call to inputMethodQuery(ImCursorPosition). So, if you think of ImAnchorPosition as our g(), then the code mirrors the structure f() -> g() -> h() described above.

When an exception occurs in Python, sys.excepthook(...) is called with an exc_tb parameter. This parameter contains the information for each of the lines in the Tracebacks shown above. The reason why the first version of our code did not include f() in the traceback was that it did not appear in exc_tb.

To fix the problem, our additional excepthook code above creates a "fake" traceback that includes the missing entries. Fortunately, the necessary information is available in the .tb_frame property of the original traceback. Finally, the default sys.__excepthook__(...) does not work with our fake data, so we need to call traceback.print_exception(...) instead.

If you found this helpful, then you will likely also be interested in fbs. It's a framework that deals with common problems to let you create PyQt apps in minutes, not months. At the time of this writing, it has 1400 stars on GitHub. Check it out!