Python Web server with FLASK + WebSockets (SocketIO) on gunicorn

Guide to setup the project on PyCharm

Commandline way

mkdir myproject

make a file named base.txt having these lines:

flask
flask-socketio
gunicorn
gevent-socketio
gevent-websocket

make a file named gunicorn_config_dev.py inside myproject:

#http://stackoverflow.com/questions/13551050/gevent-socketio-flask-gunicorn

workers = 2
worker_class = 'socketio.sgunicorn.GeventSocketIOWorker'
bind = '0.0.0.0:8000'
pidfile = '/tmp/gunicorn.pid'
debug = True
loglevel = 'debug'
errorlog = '/tmp/gunicorn.log'
daemon = False

virtualenv --python=/usr/bin/python2.7 env
source env/bin/activate
pip install -r base.txt

Start server:
gunicorn -b 0.0.0.0:8000 -c gunicorn_config_dev.py -w3 app:app
or
python app.py # to diagnose problems

IDE: PyCharm, v3.4.1

Go to file and create a new Python Project or Flask based project

 

If we used the Project Type set to Flask, we get file hierarchy:

Otherwise for an Empty project, we need to create the directories:
static
templates

and the python file.

Install the dependencies

We create a new file in the root of our project named base.txt.







 That file is our pip requirements file.

 

 

 We paste the following content in the base.txt file:

flask
flask-socketio
gunicorn
gevent-socketio
gevent-websocket

 

 

 

 

 

Now pressing control + alt + S, or File --> Settings

will show the settings dialog.

From there we click on the "Python Integrated Tools"

We set the first field to "/home/gnu_d/Projects/myproject/base.txt", using the button "..." to choose the file location.
(don't type it manually, unless you got the full file path)

Now open a random py file:

you'll see a message alert saying you have dependencies to install.

Click on the "Install requirements" link

This tells a list of which packages will be installed.

We need to highlight all the packages and the click on OK.

After few seconds (or depending on your internet connection), the status bar will show  that the packages are installing, clicking on the .. process running, will show the Background Tasks dialog, to monitor the installation.

 

 When finished will display this:

unless some error during download occurs. 

Now we're ready to edit the application.

 

 

Index view

Create a new html file named index.html in templates directory.

Use the following HTML5 code.

Set the route

 

 The default PyCharm flask project will look something like that.

 

Now we need to change the return of the function hello_world to:

return render_template("index.html")

this will show the index html when visiting from:

http://localhost:5000/

However, it will fail to communicate with the WebSocket server.

That's because we have no WS server, yet :).

Now we need to run the project under gunicorn server, so we can have gevents.

Choose new Python.

Static files setup

Running the server now and trying a resource 

http://localhost:5000/js/main.js

will result in 404 error.

To fix this, we need this code:

@app.route('/<path:path>')
def static_file(path):
    return app.send_static_file(path)

 Now we can serve, css, js, images files.

Socket IO server setup

Now after app is set, we write this code:

app = Flask(__name__)
socketio = SocketIO(app)

That binds the socket io into the existing Flask app.

After that we set the socket routes.

Default Routes

@socketio.on('connect', namespace='/echo')
def test_connect():
    print('Client connected')


@socketio.on('disconnect', namespace='/echo')
def test_disconnect():
    print('Client disconnected')

We can also add our own

@socketio.on('search', namespace='/echo')
def action_search(query):
    emit('answer', {'message': "you searched for: " + query["msg"]})

Configure the GUNICORN server for development

 Create the gunicorn_config_dev.py in the root of the project (described in the commandline section)

Now click on OK to save it, choose the Gunicorn-dev as default run app in PyCharm (if it's not yet selected), now we can run or debug it with no problems.

If you get these errors:

Exception gevent.hub.LoopExit: LoopExit('This operation would block forever',) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored
sys.excepthook is missing
lost sys.stderr
Unhandled exception in thread started by
sys.excepthook is missing
lost sys.stderr

But you will :), go to File --> Settings again.

Click on the the Python debugger tab, and make sure "Gevent compatible debugging" is checked.

Debugging

This demonstrates how we can debug the app.

 Local

To be able to start without gunicorn, add this in the if __main__  ...

socketio.run(app, host='127.0.0.1', port=8000, resource="socket.io", policy_server=False)

instead of:

app.run()

Heroku

Make a file Procfile

web: gunicorn --worker-class socketio.sgunicorn.GeventSocketIOWorker --log-file=- myproject:app

Replace the myproject.py file __main__ part with:

if __name__ == '__main__':
    #app.run()
    import os
    PORT = os.environ['PORT']
    print("Gunicorn conf using port: " + PORT)
    socketio.run(app, host='127.0.0.1', port=PORT, resource="socket.io", policy_server=False)

 That coded fixes the following exception:

2015-09-14T11:48:01.239434+00:00 heroku[web.1]: Starting process with command `gunicorn --worker-class socketio.sgunicorn.GeventSocketIOWorker --log-file=- myproject:app`
2015-09-14T11:48:04.205137+00:00 app[web.1]: [2015-09-14 11:48:04 +0000] [3] [INFO] Starting gunicorn 19.3.0
2015-09-14T11:48:04.314968+00:00 app[web.1]: [2015-09-14 11:48:04 +0000] [10] [INFO] Booting worker with pid: 10
2015-09-14T11:48:04.205897+00:00 app[web.1]: [2015-09-14 11:48:04 +0000] [3] [INFO] Listening at: http://0.0.0.0:50732 (3)
2015-09-14T11:48:04.205986+00:00 app[web.1]: [2015-09-14 11:48:04 +0000] [3] [INFO] Using worker: socketio.sgunicorn.GeventSocketIOWorker
2015-09-14T11:48:04.222259+00:00 app[web.1]: [2015-09-14 11:48:04 +0000] [9] [INFO] Booting worker with pid: 9
2015-09-14T11:48:04.969636+00:00 heroku[web.1]: State changed from starting to up
2015-09-14T11:48:05.113424+00:00 app[web.1]:     worker.init_process()
2015-09-14T11:48:05.113427+00:00 app[web.1]:     super(GeventWorker, self).init_process()
2015-09-14T11:48:05.113416+00:00 app[web.1]: [2015-09-14 11:48:05 +0000] [9] [ERROR] Exception in worker process:
2015-09-14T11:48:05.113432+00:00 app[web.1]:     **ssl_args
2015-09-14T11:48:05.113428+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/base.py", line 124, in init_process
2015-09-14T11:48:05.113429+00:00 app[web.1]:     self.run()
2015-09-14T11:48:05.113431+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/socketio/sgunicorn.py", line 80, in run
2015-09-14T11:48:05.113421+00:00 app[web.1]: Traceback (most recent call last):
2015-09-14T11:48:05.113426+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
2015-09-14T11:48:05.113433+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/socketio/server.py", line 72, in __init__
2015-09-14T11:48:05.113434+00:00 app[web.1]:     address = args[0].cfg_addr[0]
2015-09-14T11:48:05.113436+00:00 app[web.1]: AttributeError: 'socket' object has no attribute 'cfg_addr'
2015-09-14T11:48:05.113437+00:00 app[web.1]: Traceback (most recent call last):
2015-09-14T11:48:05.113438+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
2015-09-14T11:48:05.113439+00:00 app[web.1]:     worker.init_process()
2015-09-14T11:48:05.113440+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
2015-09-14T11:48:05.113441+00:00 app[web.1]:     super(GeventWorker, self).init_process()
2015-09-14T11:48:05.113443+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/base.py", line 124, in init_process
2015-09-14T11:48:05.113423+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
2015-09-14T11:48:05.113444+00:00 app[web.1]:     self.run()
2015-09-14T11:48:05.113445+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/socketio/sgunicorn.py", line 80, in run
2015-09-14T11:48:05.113446+00:00 app[web.1]:     **ssl_args
2015-09-14T11:48:05.113447+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/socketio/server.py", line 72, in __init__
2015-09-14T11:48:05.113448+00:00 app[web.1]:     address = args[0].cfg_addr[0]
2015-09-14T11:48:05.113449+00:00 app[web.1]: AttributeError: 'socket' object has no attribute 'cfg_addr'
2015-09-14T11:48:05.128396+00:00 app[web.1]: [2015-09-14 11:48:05 +0000] [9] [INFO] Worker exiting (pid: 9)

Use the following commands in the root of the project:

git init
heroku create
git add .
git commit -m "Initial commit"
git push heroku master
heroku open
heroku logs

Not able to reproduce this in PyCharm :P, yet
(the git part is easy).