How to Create a Google App Engine Data Logging Server

Dr. Ramesh Yerraballi, Dr. Jonathan Valvano, and Jason He

February 15, 2017

0) You will need to have a Google account. However, many of us have multiple Google accounts, which creates a potential confusion. It is important to decide to which account you wish to attach this web server. Throughout the creation of your web server, please make you are logged into that one google account. You will need to verify you are logged into the correct google account at each step of the process.

  1. Build your project with Python through App Engine
  1. Go to https://console.cloud.google.com
  2. Hover over Project and click Create project
  3. Choose a unique name, e.g., ee445l-youreid
    (Note that although the     Project Name can, the Project ID cannot have capital letters or underscores. The URL of your project will be projectid.appspot.com.)
  4. Click the hamburger button
  5. Choose App Engine
  6. And under Select a language, choose Python
  7. Click Next
  8. At this point, you can go ahead and click Cancel Tutorial

  1. Download Python 2.7 for local development
  1. Go to https://www.python.org/downloads/
  2. Click the most recent 2.7.X release; for me, it was Python 2.7.13
    (Make sure you do NOT get Python 3.X)
  3. Download the appropriate installer and follow the installer prompts
  1. Install the App Engine SDK
  1. Go to https://cloud.google.com/appengine/docs/python/download#appengine_sdk
  2. Find the heading “Optional: Download and install the App Engine SDK for Python”
  3. Choose your OS (Linux, Mac OS X, or Windows) and follow the instructions listed

  1. Set up the Google App Engine local environment
  1. Download the server files, and unzip these files into a local easy to find location
    http://users.ece.utexas.edu/~valvano/EE445L/serverstarter.zip
  2. Open app.yaml in a text editor, e.g., Notepad or Notepad++
  3. In line 1, modify it so it says “application: “ then the name of your project, e.g., ee445l-youreid
  4. Save the app.yaml file
  5. Optional step: open the main.py file and edit line 63 to display your name on the web server, placing it between the <h2> and </h2>
    self.response.write('<h2>Embedded Systems - University of Texas at Austin</h2>')
    main_py.jpg
    Save the     main.py file
  6. Rename the folder containing the files so it matches your project name
  7. Open Google App Engine Launcher from your computer
  8. Click File → Add Existing Application → Browse
  9. Navigate to the location of the folder with the files and click on it
  10. You can leave the ports at the default settings and click Add
  11. Click the new project you just created and then the big green Run button
  12. Click Browse and your browser should open and direct you to http://localhost:8080/

  1. Test the project on the local host
  1. Manually query the database (rather than using the LaunchPad + CC3100 to send a JSON request) by copying and pasting this example query into the local host browser and hitting enter
            
    e.g.,     http://localhost:8080/query?city=Austin,%20Texas&id=json&greet=Ta-da%21

    You can modify “Austin,%20Texas”, “json”, and “Ta-da%21” to hold whatever city, name, or message you want.

If it is working properly, you should see the following:

And the main page should look like this:

(The IP address part (“accessed from”) will not be populated because you are running locally.)

  1. To check what is in the database, go to: http://localhost:8000/datastore?kind=Greeting
  2. You can clear data by selecting the box next to specific entries and clicking Delete

  1. Deploy your data logging server
  1. If you’re satisfied with how your server looks, just click Deploy on Google App Engine Launcher!
  2. The first time you deploy, you’ll be asked to authenticate your credentials

    Our experience is we needed to execute deploy multiple times before it authenticated properly, and sometimes it takes time to complete the deploy.
  3. A browser window will open and prompt you to login to your Google Account
    Once you login, if deployment is successful, the shell will look like this:

    You can close the shell window
  4. To test, query from your browser again, but this time to the URL instead of the local host

e.g., http://ee445l-youreid.appspot.com/query?city=Austin%2C Texas&id=json&greet=Ta-da%21


You can modify “Austin,%20Texas”, “json”, and “Ta-da%21” to hold whatever city, name, or message you want.

You should see your server on the web at http://ee445l-youreid.appspot.com

  1. To check what is in the database, go back to: https://console.cloud.google.com
  2. Click the hamburger button and select Datastore
  3. From the Entities window, you can now use Create Entity to manually add to the database and Delete to manually remove from the database
  4. When you’re done testing, open main.py
  5. Stop browser-querying by uncommenting lines 79-81
    (Remove the “#” in front of each line)
  6. Save main.py
  7. Click Deploy again in Google App Engine Launcher
    (You will need to redeploy every time you edit any of the server files. If you just save, only the local host will receive the update. Deploying will update the actual website.)

  1. Test server from the IoT embedded object
    To test sending JSON requests to the server from your Launchpad + CC3100, simply define a JSON request formatted as a string in C. For example:

    #define SERVER  "ee445l-youreid.appspot.com"
    #define REQUEST "GET /query?city=Austin%2C%20Texas&id=Jason%27s%20LaunchPad&greet=hello HTTP/1.1\r\nUser-Agent: Keil\r\nHost: ee445l-youreid.appspot.com\r\n\r\n"

    You can look at the     ValvanoWare CC3100-related example projects to see how you copy the request into the send buffer, which is sent to the host. Make sure you update the Host Name, in addition to the JSON request.

  1. How main.py works
    This project was based on Google’s example guestbook project. Modifications were made so the server would process JSON requests, and store them in the database.

    The first 22 lines are libraries that Dr. Yerraballi included. You can find more information on what each of the libraries provides by searching the name of the library. The currently included libraries should suffice for simple modifications, such as adding new fields to the database.

    The next set of lines, through line 57, set up the database, referred to as a Datastore. Lines 31-36 construct a Datastore key. In our example, we use “logbook_name” as our key. Then lines 39-47 define the entity, “Greeting,” we use in our logbook. The entity defines what the different attributes of our Datastore are. In this case, we want the latest entry to be posted first, so we index by “date.” The “auto_now_add” property sets that for us. For the other columns, we put “indexed=False” so the Datastore is not indexed by the other parts of the entry.

    Lines 60-74 and 76-95 specify how the main page and /query page behave. The MainPage class first displays the HTML for the title of the page (“Embedded Systems - University of Texas at Austin”). Then, it fetches from the logbook Datastore all Greeting entities. Finally, it loops through all the Greetings and posts the HTML for each corresponding entry. Similar to printf in C, you can print variables in your HTML statements by using format specifiers.

    For example, in     main.py, lines 72-73 currently says:

    e.g., self.response.write('<b>%s@%s</b>: <i>%s</i> (accessed from %s)' % (greeting.author, greeting.city, greeting.greet, greeting.ipaddr))

    The variables, “greeting.author,” “greeting.city,” etc. get passed into the respective %s specifiers, and are then shown on the website.

    Finally, the Auto class, lines 76-95, handles JSON queries from both the browser and the LaunchPad + CC3100. It creates a new greeting, finds the IP address by using an imported library, and processes the JSON stream. The urllib2 library reads the JSON stream, and the ID, city, and greet values are assigned to the author, city, and greet attributes of the greeting. Finally, lines 92-95 posts HTML that confirms what the Datastore received.

    At the end, lines 97-100 tie the web application together and connect URLs with scripts (such as ‘/query’ with Auto).

    If you would like to add an attribute to the Datastore, follow these steps:
  1. Add the attribute to the Datastore initialization:
    e.g., age = ndb.StringProperty(indexed=False)
  2. Display the attribute on the main page:
    e.g., self.response.write(‘%s’ % (greeting.age))
  3. Add the attribute to the JSON request processing:
    e.g., greeting.age = cgi.escape(self.request.get(‘age’))
  4. Display the attribute on the query response page:
    e.g. self.response.write(‘%s:%s’ & (“age”, greeting.age))

You can also modify the script so it responds to the data in certain ways. For example, you can set your server so it only posts queries with a specific ID or a specific greet to the main page. To do this, in the MainPage class, add a line between 71 and 72:

e.g., if int(greeting.age) >= 18:

Then, when the main page is looping through to post HTML for each query, it will only post Datastore entries where the age attribute is greater than or equal to 18.

If you have any other modifications you’d like to make to the server that you can’t figure out, feel free to email the TAs, and we can try to help. Google’s App Engine tutorials have plenty of information, though, so I would encourage you to peruse those resources.