Bug 4140 - Python mod is not called on cached queries
Python mod is not called on cached queries
Status: RESOLVED FIXED
Product: unbound
Classification: Unclassified
Component: server
1.7.3
x86_64 FreeBSD
: P5 minor
Assigned To: George Thessalonikefs
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-07-31 16:21 CEST by Michael
Modified: 2018-10-29 12:03 CET (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2018-07-31 16:21:16 CEST
We are an ISP and we need to redirect our customers to our portal, when they are out of money.
One of the options is sending portal IP on each query(except main site and payment system sites). I've decided, that Unbound is a good choice to modify DNS answer on the fly.
Documentations says that modules are called in the same order as they are mentioned in module-config parameter.
Well, while testing i've figured out, that python-module(operate function) is called only when there is no cached answer. It looks like that:
(1) - client, that should be redirected
(2) - client, that should get normal answer
(s) - unbound server with module-config "validator python iterator"

(1)->(s) looks for ya.ru
(s)->(1) answers with portal IP(logs show validator-python-iterator-python-validator) Python does not store in cache+invalidates cache. Answer is not cached
(2)->(s) looks for ya.ru
(s)->(2) answers with Yandex IP(logs show validator-python-iterator-python-validator) python does not invalidate answer, nor asks to exclude from caching. Answer is cached.
(1)->(s) looks for ya.ru
(s)->(1) answers with Yandex IP(i don't see Python module in the log). But should answer with portal IP.

I tried also module config without validator, that obviously didn't help.
I've found an example python script  insource distribution with callbacks and new init procedure for Python module.
I've added callbacks for recursive answer and for cached answer with logging, which one is called.
Well, i see that after caching correct answer only callback function is called from python module. Is that correct? Neither I see validator nor iterator to be called in logs.
If that is correct - can I modify answer in callback based on  DNS-client IP and resolved ip for the query? If i can - where can i read about it?
IP addresses of the customers to redirect are held in local memcached, so the IP-addresses of the payment systems are held in memcached, so the check shoud not slow down our server.
Comment 1 Wouter Wijngaards 2018-07-31 16:34:41 CEST
Hi Michael,

There are separate callbacks for the cache returns.

inplace_cb_reply_servfail_call when the query is a failure (SERVFAIL).  That you probably do not need to change.

inplace_cb_reply_cache_call when the query is answered from the cache.
The callback is of the type inplace_cb_reply_cache.  I think you register with the function register_inplace_cb_reply_cache
pythonmod/examples/inplace_callbacks.py has example code that registers such a function.

So, callbacks already exist for cached responses.  If there is a good spot to document, apart from the examples already there, maybe that could be a way to solve this?

Best regards, Wouter
Comment 2 Wouter Wijngaards 2018-07-31 16:40:27 CEST
Hi Michael,

Let me add some more.  The position of the python plugin, eg. after validator and before iterator does not matter for the in-place callbacks.  Those are registered by the plugin, and it can then intercept and modify responses at those moments.  For the interaction with the validator and the iterator the order is important, but for the in-place callbacks that does not matter so much, just register them, and then that python function is called at that time.  (There is a small list of places you can register callbacks for).

It is possible to add new callback moments and callbacks functions if one is missing, but I closed the bug because I thought the existing cache response call seems to be what you need to use.

Best regards, Wouter
Comment 3 Michael 2018-07-31 17:02:00 CEST
Thank you, Wouter for your reply.
Seems like, i was not that clear.
I have 2 thing to check before answering the query:
1 - client IP. If it is in memcache - the second check should be performed
2 - true answer from recursion - if A-record is in whitelist(e.g. - 3d-Secure page of a bank, or something like that) - answer should not be altered. If it isn't in whitelist - answer should be altered -> have only one A-record with portal IP, where it states, that customer should pay for the service(and links to payment systems).
Yes, "def inplace_cache_callback(qinfo, qstate, rep, rcode, edns, opt_list_out,                        region)" seems like the function, i need. The best place to alter the query_reply, but... I need to find out client IP. In the "resip.py" example we can use "qstate.mesh_info.reply_list", but in this callback seems like there is no mesh_info in qstate. Where do I get client IP, while in this callback?
Will the following mnemocode work for this callback?
"
msg=DNSMessage...
msg.answer.append
msg.set_return_msg
"
Getting to know client IP in the callback function seems to me to be the main difficulty.
Could you clarify that question?
Comment 4 Wouter Wijngaards 2018-07-31 17:11:54 CEST
Hi Michael,

Yes, of course.  That information is sitting in a variable called "repinfo" of type "comm_reply" at the point where the callback is being called.  But the variable is not passed to the python function.

Not sure if we can easily add it.

Best regards, Wouter
Comment 5 George Thessalonikefs 2018-08-02 17:23:20 CEST
Hi Michael,

I will try to expose the "repinfo" variable that contains the client IP to Python but I want to do that for all callbacks that are called prior to the mesh state.

Will update here when I have made progress.

-- George
Comment 6 George Thessalonikefs 2018-08-22 13:06:41 CEST
Hi Michael,

The changes to expose the `repinfo` variable to the inplace_callbacks are now on the repository.
You can revisit the example code in pythonmod/examples/inplace_callbacks.py and check the inplace_servfail_callback() function for an example.
Note that you can use parts of that code in the inplace_cache_callback() for your specific case.
Also note that the callbacks' signature is now changed and requires an extra **kwargs parameter.

-- George
Comment 7 Michael 2018-10-29 12:03:34 CET
Hello, George and Wouter
Sorry for bumping up, but still can't figure out how to replace answer message in callbacks. Examples only show how to add EDNS options, not change answer itself. Could you help me with this question?