Caching RiveScript Brain for a Web Application #72

Since it's Python, have you tried pickle?

Also, how are you managing your RiveScript instance in your Flask app? If you keep one globally accessible instance, the requests can reuse that instead of needing to load the bot for each message. Here's an example. Depending on your use case it might be acceptable to initialize the bot on server startup and keep it in memory for the lifetime of your application, and not worry about serializing and storing the bot.

Thanks for your reply!

I had run across pickle but didn't end up trying it in favor of json / rapidjson, just gave it a try and looks like the time to unpack the binary is still a bit lengthy.

The latter is an option I tried and seemed to work well, but had seen that it was considered bad practice to declare global variables across requests? But to your point, it seems like the best option instead of having to serialize and store the bot.

Any feedback on the latter point is appreciate, but will be trying to implement that solution and follow up. Thanks again for this library and your help!

I haven't battle-tested it (read: sending tons of simultaneous requests to a Flask app configured this way), so it might take a little tweaking to get it to work in a thread-safe way if you find that to be a problem.

Some ideas I have for thread safety:

• Put the RiveScript instance on an attribute of your Flask app object. Your endpoints can use from Flask import current_app and access it by doing current_app.bot.reply(...) for example. This seems to be Flask's preferred way to handle globals from what I've seen being used by Flask extensions.
• Implement thread locking yourself: create your own get_reply() function that uses a mutex to lock concurrent access (e.g. with threading.Lock).

In production mode with a WSGI runner, Flask apps tend to run on multiple threads or processes. In the latter case, each process would initialize its own copy of the bot and then use that for its requests. This could have the side effect of fragmenting user variables (if a different process handles different requests), so you might need something external to synchronize user variables (e.g. a Redis cache that all procs can use). The Flask example I linked earlier accepts user variables from the client, if you want to offload that responsibility to the client side (e.g. have the client send in all the user variables it knows about, so no matter which Flask proc handles the request they will all have the same user vars).

Thanks!

I went ahead and implemented the initial method for our use case (using current_app for thread safety). In the coming days, we will be battle-testing against a large number of requests (we've hit as many as 100,000 in a day, so we'll be aiming big as far as test rate goes).

Appreciate your help, and once again appreciate this project!