Web app with Flask & D3.js on Orange Pi

Create a Flask web page

In this tutorial I will show you how to build a simple web app using Flask for the server side scripting and D3.js for the data visualization.

Flask is a simple yet powerful Python micro web framework, that can be set up in minutes. D3.js is a JavaScript library that is using HTML, SVG, and CSS to create stunning data driven visualizations, animations and dynamic interactions.

The web app we are going to build will read data from a MySQL database and then render the rows in an HTML table. In the second part of the tutorial we will visualize the data on a map of the world. We can color the countries by different indicators such as outbound tourist flights or GDP per capita.

This tutorial will work both on the Raspberry Pi as well as the Orange Pi, as long as you install all the required packages. To begin with you will need to install MySQL and create a database. Then, get some data to store on your database (you can learn how to do that from my previous post). I will use the same outbound tourists data provided by the World Bank.

On the Python side, you will need the SQLAlchemy and mysqldb packages. Again, I already covered connecting Python to MySQL here.

So it’s time to install Flask:

sudo pip install Flask

The official documentation is good and I also found this tutorial pretty helpful.

So now we need to create a new folder for our app in our home directory, let’s call it “WorldTurists”. In this folder we will store the code for our app, and any additional resources. Inside this folder, create two more folders, called “static” and “templates”. The folders have to be named exactly like that, it’s just part of how Flask works. The “static” folder will contain things such as images, CSS and JavaScript resources. The “templates” folder will contain HTML templates, that we can use to render dynamically in our app.

The Python code for the app should be in the “WorldTurists” folder. Let’s create a new file and call it “WorldTouristsApp.py” for example. Open it and write inside:

from flask import Flask, render_template

app=Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')
 
if __name__=="__main__":
    app.run(debug=True)

Here the most interesting lines for us are the ones containing “@app.route” and “render_template”. The rest is code required by the Flask framework to run a web application. With the “route” decorator we simply map URLs to functions in our app. The “render_template” function allows us to use Python to dynamically create page content, by using variables and flow control code. I will get back to this later on.

Let’s create a simple web page, with Bootstrap support, in the “templates” folder:

<!DOCTYPE html>
<html lang="en">
<head>
  <title>Countries by outbound tourists</title>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <!-- Bootstrap CSS -->
  <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
  <!-- jQuery -->
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
  <!-- Bootstrap JavaScript -->
  <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
</head>
<body>

<div class="container">
  <h2>List of countries by outbound tourists in 2014</h2>
  <p>Source: World Bank</p>
  <!--Placeholder for table -->
</div>

</body>
</html>

Save it as “index.html”. Let’s just run this example and see what happens:

python WorldTouristsApp.py

You should get back something like:

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!

If you installed Raspbian or Armbian, you can navigate to the “http://127.0.0.1:5000/” address in the web browser. You should see this simple web page:

simple-web-page

Now let’s get some actual data and create our dynamic HTML table. First we need to modify our app to read data from SQL in a DataFrame  and use it in the render function:

from flask import Flask, render_template
import pandas
from sqlalchemy import create_engine

app=Flask(__name__)

@app.route('/')
def index():
#create the connection string
con=create_engine('mysql+mysqldb://wbuser:wbpwd@192.168.1.117:3306/wb',
echo=False)
#get data
datar=pandas.read_sql('SELECT * FROM wbdt', con)
#keep only data for 2014
datar14=datar[datar.yr==2014]
#assign the dataframe to a variable "table"
return render_template('table.html',table=datar14)

if __name__=="__main__":
app.run(debug=True)

So after we imported the data from MySQL, we store it in a Python DataFrame. Next we do some filtering on the DataFrame. Lastly we assign the filtered DataFrame to a variable “table” that will be used in the HTML template. The Jinja2 template engine will evaluate any Python code written in the template and will generate HTML code for our web app. The Python code we need to add to our template is:

{{table.to_html(index=False,classes="table table-striped")|safe}}

So here we are using the pandas DataFrame’s built in method of converting data to a HTML table. using this methods arguments, we leave out the index and we style the table nicely, using Bootstrap table classes. Additionally, using the “safe” filter we tell Jinja2 that we are passing HTML.

Running your app again, you should see the formatted table:

dataframe-html-table

Visualize country data with D3.js

We now have a dynamic web app, feeding from a MySQL database. Let’s take it a step further and add a country-level map visualization using D3.js. In particular we will use the Datamaps library. To do that we need to include this library and it’s dependencies in our page:

<script src="//cdnjs.cloudflare.com/ajax/libs/d3/3.5.3/d3.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/topojson/1.6.9/topojson.min.js"></script>
<script src="//datamaps.github.io/scripts/0.4.4/datamaps.world.min.js"></script>

Then we need to add a container to hold our map. I added it just above the table. Lastly we need to write the JavaScript to color the countries by the number of outbound flights. Here is the HTML code, which adapts a Datamaps example code:

<!DOCTYPE html>
<html lang="en">
<head>
<title>Countries by outbound tourists</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Bootstrap CSS -->
<link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<!-- jQuery -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
<!-- Bootstrap JavaScript -->
<script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
<!-- d3 and Datamaps JavaScript -->
<script src="//cdnjs.cloudflare.com/ajax/libs/d3/3.5.3/d3.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/topojson/1.6.9/topojson.min.js"></script>
<script src="//datamaps.github.io/scripts/0.4.4/datamaps.world.min.js"></script>
</head>
<body>

<div class="container">
<h2>Countries by outbound tourists in 2014</h2>
<!-- This container will have our map -->
<div id="container" style="position: relative; width: 800px; height: 480px;"></div>

<h2>List of countries by outbound tourists in 2014</h2>
<p>Source: World Bank</p>
{{table.to_html(index=False,classes="table table-striped") |safe}}
</div>

</body>

And the JavaScript section:

//convert the Python DataFrame to JSON
var data={{table.to_json(orient ='records') |safe}}
//create an empty object for Datamap
//Datamap expects data in format:
// { "USA": { "fillColor": "#42a844", numberOfWhatever: 75},
// "FRA": { "fillColor": "#8dc386", numberOfWhatever: 43 } }
var dataToMap = {};

//pick some values for the color scale range
var maxValue = 25000000;
var minValue = d3.min(data, function(d) { return +d.tourout;} );

//create color palette function
var paletteScale = d3.scale.linear()
.domain([minValue,maxValue])
.range(["#EFEFFF","#02386F"]); // blue color

//our data does not have the 3 letter ISO code
//so we try to get it by matching the country name
var ctr = Datamap.prototype.worldTopo.objects.world.geometries;
data.forEach(function(d){ //
var iso = "";
for (var i = 0, j = ctr.length; i < j; i++) {
if (ctr[i].properties.name.startsWith(d.countries)){
iso=ctr[i].id};
}
//fill dataset in appropriate format
var value = d.tourout;
dataToMap[iso] = { tourout: value, fillColor: paletteScale(value) };
});
//render map
var map = new Datamap({element: document.getElementById('container'),fills: { defaultFill: '#F5F5F5' },
data: dataToMap});

When you refresh the web page it should look like this:

dataframe-datamaps-d3js

You now have a web app running on your Orange Pi. You can make this app available on your home network by changing this line in the Flask app:

app.run(debug=True)

Change it to point to your IP address, for example:

app.run('192.168.1.117')

Now you can access you web page from any device connected to your network, by typing in your Orange Pi’s IP address and port in a web browser!

Touchscreen display on Raspberry Pi
OpenELEC on the Orange Pi

Leave a Reply

Your email address will not be published / Required fields are marked *