Deploying a Python Text Analytics App with Flask, uWSGI and Nginx

By Adam McQuistan in Python  07/20/2019 Comment

Introduction

This is a continuation of the article Building a Text Analytics App in Python with Flask, Requests, BeautifulSoup, and TextBlob and will focus on how to deploy the previously built application to a cloud hosted Ubuntu LTS version 18 Virtual Private server. For this particular article I will be running the Flask application within the Web Server Gateway Interface (WSGI) compliant Python application server known as uWSGI. The Flask application / uWSGI server will sit behind an Nginx web server which serves static content such as CSS as well as reverse proxy non-static requests to uWSGI handled by Flask.

The code for this article is hosted on GitHub for you to follow along or experiment with.

Ubuntu Server Setup and Installing the Linux Dependency Packages

To start I need to SSH onto the server but, creating an SSH connection is not going to be going to be something I cover as it varies among the different cloud services. So, rather than giving preference to any one over the other I have provided below a few helpful links to get you onto some of the more popular ones.

• Amazon Web Services
• Digital Ocean
• Linode

A good practice is to always do a system update of packages as a first step. I issue this command as the root user.

$ sudo su # switch to root user
# apt-get update

Next I install the Python3 and Nginx packages.

# apt-get install python3-pip python3-dev python3-venv nginx -y

Next up I make a user named sentalizer to use for running the application under. After issuing the adduser command I enter a password which will be used later so I make sure to remember it then, I accept the defaults for the remaining prompts.

# adduser sentalizer

I also want to add the sentializer user to the sudo group for running elevated privellige commands. While I'm doing this I should also add the sentializer user to the www-data group because it is used to run a web based application.

usermod -aG sudo sentalizer
usermod -aG www-data sentalizer

After making sentalizer I then switch to that user, navigate to its home directory, create a Python3 virtual environment named venv in the home directory then, activate it.

# su sentalizer
$ cd
$ python3 -m venv venv
$ . venv/bin/activate

I am now ready to clone the project from GitHub.

Setting Up the Flask Application and TextBlob (NLTK) Data Files

Still SSHd onto the server as the sentalizer user with an activated Python virtual environment and in the directory of /home/sentalizer I clone the GitHub repo hosting the application made in the previous article. If you are following along please create a fork of the repo and clone from your newly forked copy.

(venv) $ git clone https://github.com/yourusername/flask-text-blob-sentiment-analyzer.git

Note that I am renaming the parent directory "flask-text-blob-sentiment-analyzer" to "sentalizerwebapp" so I don't have to type as much later. You can leave it the same if you would like.

(venv) $ mv flask-text-blob-sentiment-analyzer sentalizerwebapp

Next I change directories into the parent directory I just renamed to sentializerwebapp and install the dependencies (Flask, textblob, beautifulsoup4, requests) I previously placed in a file named requirements.txt like so.

(venv) $ cd sentalizerwebapp
(venv) $ pip install -r requirements.txt

Similar to the prior article I again need to download the NLTK corpus data files so they get stored in the sentalizer user's home directory making them accessible to the applicaiton.

(venv) $ python -m textblob.download_corpora

I can now move on to setting up uWSGI.

Configuring uWSGI Application Server

The uWSGI Python application server is just another python package so its easy installing with pip as shown below.

(venv) $ pip install uWSGI

Next up I add a settings file which will hold configurations for how the uWSGI application server should behave. I name this file uwsgi.ini and place the following in it.

[uwsgi]
module=sentalizer.uwsgi:application
master=true
processes=2

socket=sentalizer.sock
chmod-socket=660
logto=/var/log/uwsgi/uwsgi%n.log
die-on-term=true

The above tells uWSGI that it should look for a Python module named uwsgi.py within the sentalizer app and inside the module it should look for an object named application. The configs go to specify two processes (this should depend on the number of cores availble to your server), to use a socket connection file, plus where to log stuff at and to properly shut itself down.

Inside the sentalizer package (next to the __init__.py) I need to add the uwsgi.py module, import the create_app function and use it to create an application instance variable named application as shown below.

# uwsgi.py

from sentalizer import create_app

application = create_app()

Before I forget I should stop and create the parent logging directory at /var/log/uwsgi.

(venv) $ sudo mkdir /var/log/uwsgi
(venv) $ sudo chown sentalizer:www-data /var/log/uwsgi/

To properly manage the startup, execution, and shutdown of the uWSGI application server I utilize the systemd by creating a sentalizer.service file at /etc/systemd/system with the following definitions in it.

[Unit]
Description=uWSGI Python container server  
After=network.target

[Service]
User=sentalizer
Group=www-data
WorkingDirectory=/home/sentalizer/sentalizerwebapp
Environment="/home/sentalizer/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin"
ExecStart=/home/sentalizer/venv/bin/uwsgi --ini uwsgi.ini

[Install]
WantedBy=multi-user.target

At this point I can start the newly introduced service and check its status then, if all is well I enable it to start at system boot time.

(venv) $ sudo systemctl start sentalizer
(venv) $ sudo systemctl status sentalizer
● sentalizer.service - uWSGI Python container server
   Loaded: loaded (/etc/systemd/system/sentalizer.service; disabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-07-19 21:40:55 UTC; 7s ago
 Main PID: 19622 (uwsgi)
    Tasks: 3 (limit: 1152)
   CGroup: /system.slice/sentalizer.service
           ├─19622 /home/sentalizer/venv/bin/uwsgi --ini uwsgi.ini
           ├─19641 /home/sentalizer/venv/bin/uwsgi --ini uwsgi.ini
           └─19642 /home/sentalizer/venv/bin/uwsgi --ini uwsgi.ini

Jul 19 21:40:55 ip-172-31-23-91 systemd[1]: Started uWSGI Python container server.
Jul 19 21:40:55 ip-172-31-23-91 uwsgi[19622]: [uWSGI] getting INI configuration from uwsgi.ini

Now I enable the sentalizer service unit to startup automatically at boot time.

(venv) $ sudo systemctl enable sentalizer

Configuring Nginx Webserver

Next up is the setup of Nginx. To do this I place a config file in /etc/nginx/sites-available named sentalizer and place the following in it.

server {
    listen 80;
    server_name _;

   location / { try_files $uri @sentalizer; }
   location @sentalizer {
     include uwsgi_params;
      uwsgi_pass unix:/home/sentalizer/sentalizerwebapp/sentalizer.sock;
   }   
}

What this does is tells nginx to catch all requests on port 80 which is what the listen and server_name settings are for. Then I tell it to send all requests matching / and below to the uwsgi application and to talk to it over the socket file.

There is one thing that must be done in order to get this server_name catchall trick to work (ie, server_name _;) and that is to remove the file at /etc/nginx/sites-enabled/default.

(venv) $ sudo rm /etc/nginx/sites-enabled/default 

Now I need to create a symbolic link from /etc/nginx/sites-available/sentalizer to the /etc/nginx/sites-enabled so Nginx knows it exists.

(venv) $ sudo ln -s /etc/nginx/sites-available/sentalizer /etc/nginx/sites-enabled

A useful thing to do whenever changing the configs of Nginx is to run a config check to see if any mistakes were made using nginx -t command. A successful check looks as follows.

(venv) $ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Since all checked out I can restart nginx and make sure its enabled.

(venv) $ sudo systemctl restart nginx
(venv) $ sudo systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-07-19 21:58:02 UTC; 5s ago
     Docs: man:nginx(8)
  Process: 19844 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 19859 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
  Process: 19847 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
 Main PID: 19860 (nginx)
    Tasks: 2 (limit: 1152)
   CGroup: /system.slice/nginx.service
           ├─19860 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           └─19862 nginx: worker process

Jul 19 21:58:02 ip-172-31-23-91 systemd[1]: Stopped A high performance web server and a reverse proxy server.
Jul 19 21:58:02 ip-172-31-23-91 systemd[1]: Starting A high performance web server and a reverse proxy server...
Jul 19 21:58:02 ip-172-31-23-91 systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argument
Jul 19 21:58:02 ip-172-31-23-91 systemd[1]: Started A high performance web server and a reverse proxy server.

And now if I point my browser to the IP of the server I just installed on I should see the following.

Submitting a url for a web page such as https://thecodinginterface.com/blog/django-auth-part1/ will now show the following.

Resources for Further Learning on Flask and Python Web Architecture

• Miguel Grinberg's newly rewritten Flask Web Development: Developing Web Applications with Python is by far the superior Flask book on the market
• A great book on Nginx is Nginx HTTP Server - Fourth Edition: Harness the power of Nginx to make the most of your infrastructure and serve pages faster than ever before
• If your are looking for a reference for best practices on Python web app architecture Software Architecture with Python is an excellent choice

Conclusion

This article has been a follow up the the earlier Building a Text Analytics App in Python with Flask, Requests, BeautifulSoup, and TextBlob which shows how to deploy the previously built application using the performant tech architecture of Nginx and uWSGI. In a future article I plan to show how to make this application even more scalable by integrating a Redis / Celery message queue for asynchronous tasks.

As always, thanks for reading and don't be shy about commenting or critiquing below.