#webapp2

[deadrhyme]

Multiple GAE Apps can bind well

On Ubuntu 14.04

You ran an application successfully from terminal with

dev_appserver.py  <project_name>

By default: App started running with port 8080 Admin console for that app runs on 8000

Nah Man! I cannot run another application by running dev_appserver.py  <another_project_name>

It must be giving some creepy bind error, even though we know our server can run multiple apps simultaneously

So what do we do ?

We will run another app again with defined ports for app and respective admin console as dev_appserver.py --port 9080 --admin_port 9000  <another_project_name>

Hundo! Way to go.

Image Ref : google.com

[timey-wimey-code]

Adding templates and static files to our Google App Engine and webapp2

References:

GAE Documentation - Static Files

GAE Documentation - Templates

Zurb Foundation Framework

As a developer, I understand the documentations need to start with the server code first, but for demonstration purposes, I like to make things pretty first. I will start with static files and templates up onto our application.

Static Files like CSS and Javascript

We are going to use Zurb’s foundation css framework. So go download it from their website, and go to where we left our last project at “~/gae” directory. We are going to add several static files/locations like:

robots.txt

css

js

images

Now, we’ll create the files to be added:

cd ~/gae/helloworld touch robots.txt mkdir images js css

I would copy the files from my Foundation zipfile, everything in the css files in the css directory and the js files in the js directory.

Edit ~/gae/helloworld/app.yaml and change it to the following:

application: your-app-id version: 1 runtime: python27 api_version: 1 threadsafe: true handlers: - url: /(robots\.txt) static_files: \1 upload: (robots\.txt) - url: /images static_dir: images - url: /js static_dir: js - url: /css static_dir: css - url: /.* script: helloworld.application libraries: - name: webapp2 version: latest - name: jinja2 version: latest

So you should notice to see the handler changes. You should notice a few new things:

the static_files and static_dir -- static files cannot be in the same directory as application code. This directive tells the url handler that these paths are static files or are directories contain static files.

upload -- The upload directive contains the regular expression that matches the file paths for all files that are referenced by the handler.

libraries -- The libraries directive are where you can include third party modules to your application. I have explicitly added webapp2, the framework that I’m trying out, and jinja2, which is a template engine and what we’re covering next.

Jinja2 and Templates

Using templates allow us to keep our frontend (html) and backend code separate... as well as keeping our backend cleaner. Here we are going use Jinja2 templates and it's pretty simple to use.

Edit your python file, ~/gae/helloworld/helloword.py

import os import urllib import jinja2 import webapp2 JINJA_ENVIRONMENT = jinja2.Environment( loader=jinja2.FileSystemLoader(os.path.dirname(__file__)), extensions=['jinja2.ext.autoescape'], autoescape=True) class MainPage(webapp2.RequestHandler): def get(self): template_values = { 'greetings': 'Hello World', } template = JINJA_ENVIRONMENT.get_template('templates/index.html') self.response.write(template.render(template_values)) application = webapp2.WSGIApplication([ ('/', MainPage), ], debug=True)

Basically the above code allows us to use the jinja2 library and load all the html into a separate template file.

In the above code, I define the template using "templates = JINJA_ENVIRONMENT.get_template('templates/index.html')" and pass in the variables stored in template_values into the template using "self.response.write(template.render(template_values))".

Then, to pass on those values, we need to create the template. Add a new template directory and the index.html template by doing the following:

cd ~/gae/helloworld mkdir templates vim templates/index.html

In the index.html file, add the following:

<!doctype html> {% autoescape true %} <html class="no-js" lang="en"> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Foundation | Welcome</title> <link rel="stylesheet" href="css/foundation.css" /> <script src="js/vendor/modernizr.js"></script> </head> <body> <div class="row"> <div class="large-12 columns"> <h1>{{greetings}}</h1> </div> </div> <script src="js/vendor/jquery.js"></script> <script src="js/foundation.min.js"></script> <script> $(document).foundation(); </script> </body> </html> {% endautoescape %}

The template above has all our javascript and css files included and if you reload your server, you will also see the greeting variable output “Hello World”.

[timey-wimey-code]

Framework: webapp2

A simple web application framework, webapp2, is included with the AppEngine.  It’s not as complex as other Python frameworks but it does get the job done for simple websites.  For more complex sites, the AppEngine will support other WSGI frameworks that are written in pure Python -- Flask, Django, CherryPy, Pylons, web.py and web2py frameworks.

It’s a lot easier to use a framework to handle a lot of the lower level web application code like parsing the request headers, what the appropriate responses are to the browsers, make database requests, etc.  I’ll continue using webapp2 for the forseeble future.

[jgeewax]

Friendly URL routing in App Engine using webapp2

I used to do crazy things like hard-code URLs in my templates:

<a href="/accounts/create/">Create your account</a> or <a href="/accounts/view/?id={{ the_id }}">View account {{ the_id }}</a>

Ever since webapp2 came around, I've been really into the URL routing helpers, which make it much easier to keep long lists of URLs organized. I started using a new project layout that makes routing really simple -- maybe even fun.

Before I get to the interesting part I need to digress slightly. A while back, I posted a question on Stack Overflow asking whether it was better to break your app apart into separate routes in app.yaml or inside aWSGIApplication. Put simply: should you create one WSGIApplication object and route everything there? Or many separate ones, each with their own routing in app.yaml?

The short answer was: It doesn't really matter.

The longer answer was: if you're particularly sensitive about start-up time or memory, you might break things apart.

I decided to use one big app. Having a single app makes unit testing with ext.testbed a bit easier.

In addition to making testing easier, I've been using the Python 2.7 runtime which lends itself nicely to a single WSGIApplication: I put my code in main.py and then route /.* to main.app and don't think about it anymore. But how are you supposed to keep track of all the URLs, Handlers, methods, and routes in your app as it grows larger and larger?

webapp2 to the rescue.

Take a look at routes.py in this gist:

from webapp2_extras.routes import RedirectRoute from handlers.account import AccountHandler __all__ = ['application_routes'] application_routes = [] _route_info = [ ('account.list', 'GET', '/accounts/', AccountHandler, 'list'), ('account.create', None, '/accounts/create/', AccountHandler, 'create'), ('account.view', 'GET', '/accounts/<id:\d+>/', AccountHandler, 'view'), ('account.delete', None, '/accounts/<id:\d+>/delete/', AccountHandler, 'delete'), ('account.update', None, '/accounts/<id:\d+>/update/', AccountHandler, 'update'), ] for name, methods, pattern, handler_cls, handler_method in _route_info: # Allow a single string, but this has to be changed to a list. # None here means any method if isinstance(methods, basestring): methods = [methods] # Create the route route = RedirectRoute(name=name, template=pattern, methods=methods, handler=handler_cls, handler_method=handler_method) # Add the route to the public list application_routes.append(route)

There are a few things to notice here:

You could type out all the boiler-plate every time. I don't think routes have enough distinguishing factors to merit that. I removed the boiler plate and put together a list of tuples.

Each tuple defines a route: (name, methods, URL pattern, HandlerClass, handler_method).

Routes are named as thing.action (thing is singular -- not things).

The method can be 'GET', 'POST', None, or any other method (or list of methods). None simply means "I don't care".

URL patterns allow fancy matching (<id:\d+>).

Rather that having many handlers (CreateUserHandler, DeleteUserHandler), I use one handler per "thing" and have the methods correspond to the actions (UserHandler.create, UserHandler.delete). If necessary, I can use self.request.method to distinguish between intent (ie, "show me the account creation form" versus "create my account and rediect me somewhere").

The handler method name (AccountHandler.list) and the "action" part of the route name ('account.list') are identical. This makes linking easy -- you know the route's name because it lines up with your code. 

I break from PEP8 about lining data up vertically. This is data you'd put in a table. I don't like jagged tables.

Now to tie this all together, I just put some configuration for webapp2 into config.py:

__all__ = ['webapp2_config'] webapp2_config = { 'webapp2_extras.sessions': { 'secret_key': 'Put some magical secret here.', }, 'webapp2_extras.jinja2': { 'environment_args': { 'autoescape': True, }, }, }

and hook it all up in main.py:

import webapp2 from config import webapp2_config from routes import application_routes app = webapp2.WSGIApplication(routes=application_routes, config=webapp2_config)

After that, your handlers might look like this:

import webapp2 from webapp2_extras import auth from webapp2_extras import jinja2 from webapp2_extras import sessions class BaseHandler(webapp2.RequestHandler): @webapp2.cached_property def jinja2(self): return jinja2.get_jinja2(app=self.app) @webapp2.cached_property def auth_config(self): return {'login_url': self.uri_for('login'), 'logout_url': self.uri_for('logout')} @webapp2.cached_property def auth(self): return auth.get_auth() @webapp2.cached_property def session_store(self): return sessions.get_store(request=self.request) def dispatch(self): """Override dispatch to persist session data.""" try: super(BaseHandler, self).dispatch() finally: self.session_store.save_sessions(self.response) def render_template(self, template, context=None): context = context or {} extra_context = { 'request': self.request, 'uri_for': self.uri_for, } # Only override extra context stuff if it's not set by the template: for key, value in extra_context.items(): if key not in context: context[key] = value rendered = self.jinja2.render_template(template, **context) self.response.write(rendered)

from handlers import base class AccountHandler(base.BaseHandler): def create(self): if self.request.method == 'POST': # Create the account pass # Redirect them to the dashboard page return self.redirect(self.uri_for('account.view')) else: return self.render_template('account_create.html')

And your templates can now create links that look like this:

<a href="{{ uri_for('account.create', my_param='something') }}">Create your account</a> or <a href="{{ uri_for('account.view', id=the_id) }}">View account {{ the_id }}</a>

Extra bonus: If you delete a URL and the template tries to look it up, you'll get an exception -- and so will your unit tests. I guess this could be a scary thing if you don't have tests, but hopefully it is helpful.

[gosurob-blog]

User Accounts with Webapp2 + Google App Engine

We recently finished the first rev of Legit's developer site. We built it on google app engine using the webapp2 framework. Overall the experience was great, albeit with a few hiccups.

One of the hiccups was creating a backend to handle developer accounts (signup/login/logout etc). With other frameworks this is part of the "batteries included." Django for example has a very robust authentication module.

As far as I can tell a completely pre-built, well maintained solution doesn't exist for webapp2 + gae (let me know if it does and I missed it!). Fortunately, it's not that hard to create.

I found a few excellent resources in the webapp2 mailing list, docs, and source. But I couldn't find a page/blog post/stone tablet cleanly laying everything out. So, I decided to be the change I wanted to see.

Pre reqs

Aside from gae + webapp2, we need:

Python 2.7 runtime for GAE (not 2.5).

NDB It's better than the default google datastore access library and it's written by GvR. Nuff said.

WTForms Killer forms library.

To use 3rd party libraries that aren't bundled with gae you'll need to copy them to your gae app directory. I usually check the library out elsewhere, create a symlink to it inside my gae app, and add the library to my gae app's .gitignore file:

> cd /where/I/keep/thirparty/libraries/ > hg clone https://bitbucket.org/simplecodes/wtforms > cd /my/google/app/ > ln -s /where/I/keep/thirdparty/libraries/wtforms/wtforms wtforms > echo "wtforms" >> .gitignore

Remember to ln to the python module within the library's source directory, not to the source directory.

Follow this pattern for NDB and wtforms.

Note that NDB is now bundled with google app engine so technically you don't need to install it like this. You can include it with your app. But, I had trouble getting that way to work correctly. Plus, NDB is in such active development you probably want to check out the latest version from source anyway.

I'm also going to use jinja2 to render templates. You don't have to use jinja2, but trust me, use jinja2. There's a technique in the code to make rendering jinja2 templates a breeze. Install jinja2 using the technique above (as with ndb, in theory you can just use the jinja2 bundled with gae, but I like being able to control the jinja2 version myself).

The Plan

What we're going to whip up is pretty simple:

Custom User model

RequestHandler subclass with access to current user

Signup/Login/Logout handlers w/ accompanying forms

login_required decorator to put pages behind a login wall

There are some templates you'll need to build, but those are for the front-end team amirite?? (yes I am aware that the "front-end team" is just you wearing your tightest pair of jeans).

User Model

Webapp2 comes with a built-in user model. See the docs / source at: webapp2_extras.appengine.auth.models.User. It's basically ready to go - we're just going to add an email field. This isn't strictly necessary since it's an expando model, but I like having the email field be explicit:

import ndb import webapp2_extras.appengine.auth.models as auth_models class AwesomeUser(auth_models.User): email = ndb.StringProperty()

Feel free to name your model whatever you'd like, you obviously don't need to use AwesomeUser.

User Aware RequestHandler

Here is where 90% of the magic (and 99% of the complexity) happens. As you (hopefully) know from working with webapp2, each request made to your app is handled by a subclass of RequestHandler. We're going to create a subclass of RequestHandler that all the app's handlers will subclass in turn. This new base class will be aware of the currently logged in user (if there is one). We'll build it up in pieces.

Sessions

Sessions let us keep track of users across requests by storing information in a browser cookie.

This is slightly complicated as webapp2 splits things into the SessionStore model, which provides/creates sessions, and the Session, a dict-like object that stores session data. This is actually quite cool, as it let's you have multiple sessions going in a single request, all of which are generated by a single SessionStore. But for simplicity we'll use a single session from a single session store.

Creating our UserAwareHandler class with session support:

from webapp2_extras import sessions, auth # we'll use auth later on class UserAwareHandler(webapp2.RequestHandler): @webapp2.cached_property def session_store(self): return sessions.get_store(request=self.request) @webapp2.cached_property def session(self): return self.session_store.get_session(backend="datastore")

Hopefully pretty straight forward.

The webapp2.cached_property decorator does two things:

Caches the value after the first time the property is accessed (rather than recalculate it each time). Our session store and session aren't going to change in the course of the request, so it will speed things up to cache them.

Converts the method into a property like using the builtin @property decorator so you can access it like self.session['some_var'] rather than self.session()['some_var'].

I've hardcoded the session backend to datastore. The 3 builtin types of session backends are listed in the docs for get_session. I didn't pick securecookie because I don't want the user to monkeying with the session by editing with their cookies, and I didn't use memcache because I didn't want to tie sessions to memcache. Datastore is the slowest so if you need the performance, change it.

Last we need to persist changes made to the session object at the end of each request so. We add this to the dispatch method, which gets called whenever the request handler handles a request:

def dispatch(self): try: super(UserAwareHandler, self).dispatch() finally: # Save the session after each request self.session_store.save_sessions(self.response)

Auth & User

Now we need to add the auth backend and users to the request handler. Adding to our existing UserAwareHandler class:

@webapp2.cached_property def auth(self): return auth.get_auth(request=self.request) @webapp2.cached_property def user(self): user = self.auth.get_user_by_session() return user @webapp2.cached_property def user_model(self): user_model, timestamp = self.auth.store.user_model.get_by_auth_token( self.user['user_id'], self.user['token']) if self.user else (None, None) return user_model

This is very similar to the session code above. We've added an auth property that handles actually authenticating the user, and user and user_model properties for accessing the currently logged in user.

We have both user and user_model because get_user_by_session() doesn't actually return a full user object, just minimal identifying user data stored in the session. The user_model property goes to the datastore and gets the model object for the logged in user.

Also note self.auth.store.user_model is used to refer to our user model. Webapp2 is very clever and let's you configure the model you're using for users in one place (it defaults to webapp2_extras.appengine.auth.models.User). You can then reference this model with self.auth.store.user_model. If you ever want to change your user model, you just have to change your code in the one place where you do the configuration (it should be clear by now that Rodrigo Moraes and the rest of the webapp2 guys are true nerd ballers).

We need to configure the app to use our AwesomeUser model as auth.store.user_model, and we also need to configure our sessions to with a secret key, so we'll do both at the same time.

This is done by passing a config dictionary when creating the WSGI app (which app.yaml points to):

config = {} config['webapp2_extras.sessions'] = { 'secret_key': 'zomg-this-key-is-so-secret', } config['webapp2_extras.auth'] = { 'user_model': AwesomeUser, } app = webapp2.WSGIApplication([ # ... all of our applications URL routes ], config=config)

Bang! Now we're cooking with gas.

Remember, subclass ALL of your app's request handlers from UserAwareHandler. This let's you access the currently logged in user for every request and in every template.

Jinja2 Templates

Last trick. This isn't necessary for doing auth, but it has been so useful I couldn't resist putting it in. I mentioned that I recommend using jinja2 for templating. To make this seamless, we're going to add a simple render_response method to our request handler so we can render templates in one line. Add the top level function:

def jinja2_factory(app): "True ninja method for attaching additional globals/filters to jinja" j = jinja2.Jinja2(app) j.environment.globals.update({ 'uri_for': webapp2.uri_for, }) return j

And to the end of the UserAwareHandler add two more methods:

@webapp2.cached_property def jinja2(self): return jinja2.get_jinja2(factory=jinja2_factory, app=self.app) def render_response(self, _template, **context): ctx = {'user': self.user_model} ctx.update(context) rv = self.jinja2.render_template(_template, **ctx) self.response.write(rv)

Now when you want to render a jinja template while handling a request, it's as easy as:

class MyTemplateHandler(UserAwareHandler): def get(self): self.render_response("path/to/your/template", context_var1="wow", context_var2="that was easy")

And thanks to our custom jinja2_factory you can use the uri_for function in your templates and it just works. By adding self.user_model to every context, can also refer to the user variable in any template and have access to the currently logged in user. Heaven.

Signup / Login / Logout Handlers

Armed with UserAwareHandler it's easy to create a signup form and handler. First we a wtform form to handle validation and parsing the request data. They're rather self explanatory so I won't say much more about forms, but you can get completely up to speed with the wtforms crash course.

I assume you have the templates "auth/signup.html" and "auth/login.html" ready to go. Templates are a whole separate discussion, but wtforms and jinja2 make easy. The templates need to display the form fields and provide a submit button that POSTs to the same URL that's serving the form. See the wtforms crash course.

First the signup form:

from wtforms import Form, TextField, PasswordField, validators class SignupForm(Form): email = TextField('Email', [validators.Required(), validators.Email()]) password = PasswordField('Password', [validators.Required(), validators.EqualTo('confirm_password', message="Passwords must match.")]) password_confirm = PasswordField('Confirm Password', [validators.Required()])

and the signup handler:

class SignupHandler(UserAwareHandler): "Serves up a signup form, creates new users" def get(self): self.render_response("auth/signup.html", form=SignupForm()) def post(self): form = SignupForm(self.request.POST) error = None if form.validate(): success, info = self.auth.store.user_model.create_user( "auth:" + form.email.data, unique_properties=['email'], email= form.password.data, password_raw= form.password.data) if success: self.auth.get_user_by_password("auth:"+form.email.data, form.password.data) return self.redirect_to("home") else: error = "That email is already in use." if 'email'\ in user else "Something has gone horrible wrong." self.render_response("auth/signup.html", form=form, error=error)

The logic of the handler can be broken down as:

For all GET requests render the template with a blank form.

For POST requests validate the form.

If it passes validation, try to create a new user.

If creation succeeds, log the user in and redirect them to the homepage.

If user creation fails, set the error message and render the template again.

If it fails validation, render the template again with the existing form (which contains all the errors).

The tricky part is the create_user method. The first argument is an auth_id, which needs to be unique to the user. This is different from the user_id, a single, unique, numeric ID automatically assigned upon creation.

In contrast to the user_id, a user can have multiple auth_ids. For instance, the same user could log in with multiple usernames or emails, or through facebook or twitter, and the user could have a separate auth_id for each authentication method. I've just used "auth:email@address" as the auth_id, but feel free to use whatever string you like.

create_user is also doing some magic to ensure that each user's email address is unique. The google app datastore does not support the idea of "unique" properties. Webapp2 gets around this by creating a separate collection of Unique objects. When creating a user, it first tries to create new unique objects for each unique property. It uses transactions to see if the unique properties already exist and only creates them if they do not.

This is all quite clever. If you want to see how they do it, check out the source. Or, just know that by passing a list of strings to unique_properties, create_user will ensure that those properties are not in use by other users. If they are, success will come back false and info will be a dictionary of the problematic fields. If there were no issues info will be a dictionary of user data and success will be true.

On to our last two handlers, Login & Logout, which are even simpler. First, the login form:

class LoginForm(Form): email = TextField('Email', [validators.Required(), validators.Email()]) password = PasswordField('Password', [validators.Required()])

The login handler:

class LoginHandler(UserAwareHandler): def get(self): self.render_response("auth/login.html", form=LoginForm()) def post(self): form = LoginForm(self.request.POST) error = None if form.validate(): try: self.auth.get_user_by_password( "auth:"+form.email.data, form.password.data) return self.redirect_to('home') except (auth.InvalidAuthIdError, auth.InvalidPasswordError): error = "Invalid Email / Password" self.render_response("auth/login.html", form=form, error=error)

and the logout handler:

class LogoutHandler(UserAwareHandler): """Destroy the user session and return them to the login screen.""" @login_required def get(self): self.auth.unset_session() self.redirect_to('login')

If the login form validates and we find a user matching the email/password combo, we log them in and redirect the user to the homepage. It would be better to redirect the user to whatever page they were looking at, but I'll leave that as an exercise for the reader (or future post).

If the form doesn't validate or the email/password doesn't check out, we re-render the template with the filled in form and any errors.

The last piece is the @login_required decorator used in the LogoutHandler. We apply it here as it doesn't make sense to logout a user that isn't logged in. However, first we need to define the decorator.

login_required Decorator

The objective is to create decorator that we can apply to request methods within our handlers. When the method is invoked: - If the user is logged in, handle the request as per usual. - If they are not logged in, kick them to the login screen.

Here's the decorator:

def login_required(handler): "Requires that a user be logged in to access the resource" def check_login(self, *args, **kwargs): if not self.user: return self.redirect_to('login') else: return handler(self, *args, **kwargs) return check_login

This is straight forward thanks to UserAwareHandler. We check the self.user property. If it exists, we have a user. If it does not, redirect to the login page.

To put a page behind the login wall, simply apply the decorator to the handler method as shown in the LogoutHandler.

Missing Pieces

Even though this post has turned out to be longer than the average Java class (LOL JAVA), there are still a few pieces missing that are necessary before going live. From smallest to the most important:

Redirecting users to the page they were trying to view after logging them in

Templates for signup/login

A "forgot password" workflow

CSRF protection

CSRF is by far the most important. Don't leave your forms vulnerable! There is a CSRF module for wtforms that's quite helpful. While you're at it, you should also take advantage of app engine's SSL features to secure your URLs.

Building these features shouldn't be too hard if you grok everything going on in this post. I'll also see about cranking out some subsequent blog posts of some of these topics, in particular the forgot password workflow.

Questions & Comments

I hope this was useful to people just getting started on app engine. GAE's still a little rough in places, but just keep repeating to yourself "I'll never have to configure apache again." Drop any comments below or hit me up on the twitters.