Bug 1169

Summary: Incomplete sys.path is set for python module
Product: unbound Reporter: Eugene Shatsky <eugene>
Component: serverAssignee: unbound team <unbound-team>
Status: ASSIGNED ---    
Severity: enhancement CC: cathya, wouter
Priority: P5    
Version: 1.5.8   
Hardware: x86_64   
OS: Linux   

Description Eugene Shatsky 2016-12-01 19:01:36 CET
I've got "pythonmod: can't parse Python script" error when trying to set up unbound with my python script. After some investigation I've narrowed the problem to importing a module which I installed with pip system-wide. Chroot is disabled.
Here is what I get with interactive python:

>>> import sys
>>> print(sys.path)
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']

Here is what I get with log_info in my script:

['/usr/lib/python2.7/', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '.', '/etc/unbound', '/etc/unbound', '/etc/unbound', '/usr/lib/python2.7/dist-packages']

Why is it that paths are different when the script is launched via unbound-python? Missing '/usr/local/lib/python2.7/dist-packages' is the one that pip installed the module into.
Also, I wonder why unbound python-related error messages are so obscure in the syslog. It's just "can't parse Python script" if exception is thrown by module-level code, or "Exception occurred in function" if exception is thrown by function-level code. Any way to make it print full python errors with traces?
Comment 1 Wouter Wijngaards 2016-12-02 08:55:35 CET
Hi Eugene,

In pythonmod/pythonmod.c, line 157, the python module appens to sys.path, the config directory: value, and the configure-time RUN_DIR and SHARE_DIR.  It also appends the distutils.sysconfig.get_python_lib(1,0) directory.

It does not remove entries, so I don't know why there are missing entries.
the dist-packages one seems to be present at the end of the list, by the way?

For errors it calls PyErr_Print() and that prints to stderr.  Which is not there anymore when unbound is not running from the commandline with -d (or -dd).

Best regards, Wouter