cleanest way to glue generated Flask app code (Swagger-Codegen) to backend implementation

Question

I have:

1. a library that does [Stuff]
2. a swagger API definition, which is roughly #1 with minor differences to map cleanly to a REST service
3. a flask app generated #2 using Swagger-Codegen - eg results in python controller functions roughly one-to-one with #1.

My intent is that the flask app (all generated code) should only handle mapping that actual REST api and parameter parsing to match the API spec coded in swagger. After any parameter parsing (again, generated code) it should call directly over to my (non-generated) backend.

My question is, how best to hook these up withOUT hand-editing the generated python/flask code? (Feedback on my design, or details of a formal design pattern that accomplishes this would be great too; I'm new to this space).

Fresh from the generator, I end up with python functions like:

def create_task(myTaskDefinition):
    """
    comment as specified in swagger.json
    :param myTaskDefinition: json blah blah blah
    :type myTaskDefinition: dict | bytes
    :rtype: ApiResponse
    """
    if connexion.request.is_json:
        myTaskDefinition = MyTaskTypeFromSwagger.from_dict(connexion.request.get_json())
    return 'do some magic!' # swagger codegen inserts this string :)

On the backend I have my actual logic:

def create_task_backend(myTaskDefinition):
    # hand-coded, checked into git: do all the things
    return APIResponse(...)

What is the right way to get create_task() to call create_task_backend() ?

Of course if I make breaking changes to my swagger spec I will have to hand-update the non-generated code regardless; however there are many reasons I may want to re-generate my API (say, add/refine the MyTaskTypeFromSwagger class, or skip checking into git the generated code at all) and if I have to hand-edit the generated API code, then all those edits are blown away with each re-generation.

Of course I could script this with a ~simple grammar in eg. pyparsing; but while this is my first time with this issue, it seems likely it's been widely solved already!

Answer 1

The following approach worked for me:

• created three directories:

  • src - for my code,
  • src-gen for the swagger generated code,
  • codegen in which I have put a script that generate the server along with a few tricks.

• I copied all the templates (available in the swagger build) to codegen/templates and edited the controller.mustache to refer to src/server_impl , so it can use my own code. The editing uses the template language so it is generic. Still it is not perfect (I would change a few naming conventions) but it does the job. So, first add to controller.mustache :

from {{packageName}}.server_impl.controllers_impl import {{classname}}_impl

then add instead of return 'do some magic!' the following:

return {{classname}}_impl.{{operationId}}({{#allParams}}{{paramName}}{{^required}}=None{{/required}}{{#hasMore}}, {{/hasMore}}{{/allParams}})

• Script:
  • The src has a server_impl directory.
  • It creates a symobolic link so that server_impl can be imported as a python module

cd ../src-gen/swagger_server/
ln -s ../../src/server_impl/
cd ../../codegen
java -jar swagger-codegen-cli.jar generate  \
-i /path_to_your_swagger definition.yaml \
-l python-flask \
-o ../src-gen \
-t ./templates
cd ../src-gen/
python3 -m swagger_server

Answer 2

I was tempted to use swagger-codegen before and ran into the same conundrum. Everything is fine until you update the spec. Although you can use custom templates, this just seemed like a lot of overhead and maintenance, when all I want is a design first API.

I ended up using connexion instead, which uses the swagger specification to automatically handle routing, marshaling, validation, etc. Connexion is built on flask, so you would not need to worry about switching frameworks or anything, you will just get the benefit of portions of your application being automatically handled from swagger instead of having to maintain auto-generated code.

Answer 3

For now I am working around this by doing the build in these steps

1. run the codegen
2. sed-script the generated code to fix trivial stuff like namespaces
3. hand-edit the files, so that instead of returning 'do some magic' (thats the string all the generated controller endpoints return) they simply call a corresponding function in my 'backend'
4. use git format-patch to make a patch of the preceeding changes, so that when i re-generated code the build can automatically apply the changes.

Thus, i can add new endpoints and I only have to hand-code the calls to my backend ~once. Instead of using patch files, i could do this directly by writing a py-parsing grammar for the generated code and using the parsed generated code to create the calls to my backend ... that would take longer so I did this all as a quick hack.

This is far from optimal, i'm not going to mark this as accepted as I'm hoping someone will offer a real solution.

Answer 4

The workflow I came to.

The idea is to generate the code, then extract swagger_server package to the project directory. But separately, keep controllers your are coding in the separate directory or (as I do) in the project root and merge them with generated ones after each generations using git merge-files . Then you need to inject your fresh controllers code into swagger_server/controllers , ie before starting server.

project
+-- swagger_server
|   +-- controllers
|       +-- controller.py <- this is generated
+-- controller.py <- this is you are typing your code in
+-- controller.py.common <- common ancestor, see below
+-- server.py <- your server code, if any

So the workflow is the following:

1. Generate code, copy swagger_server to your project directory, completely overwrite existing
2. Backup controller.py and controller.py.common from project root
3. git merge-file controller.py controller.py.common swagger_server/controllers/controller.py
4. Make swagger_server/controllers/controller.py new common ancestor so copy it to controller.py.common , overwrite existing

Feel free to automate all of this with shell script, ie

#!/bin/bash
# Swagger generate server and client stub based on specification, them merge it into the project.
# Use carefully! Commit always before using this script!
# The following structure is assumed:
# .
# +-- my_client
# |   +-- swagger_client
# +-- my_server
# |   +-- swagger_server
# +-- merge.sh <- this script

read -p "Have you commited the project??? " -n 1 -r
if [[ ! $REPLY =~ ^[Yy]$ ]]; then echo 'Commit first!'; exit 1; fi

rm -rf swagger-python-client
rm -rf swagger-python-server

java -jar swagger-codegen-cli.jar generate -i swagger.yaml -l python -o swagger-python-client 
java -jar swagger-codegen-cli.jar generate -i swagger.yaml -l python-flask -o swagger-python-server

# Client - it's easy, just replace swagger_client package
rm -rf my_client/swagger_client
cp -rf swagger-python-client/swagger_client/ my_client

# Server - replace swagger_server package and merge with controllers
rm -rf my_server/.backup
mkdir -p my_server/.backup
cp -rf my_server/swagger_server my_server/.backup


rm -rf my_server/swagger_server
cp -rf swagger-python-server/swagger_server my_server


cd my_server/swagger_server/controllers/
files=$( ls * )
cd ../../..

for f in $files; do

    # skip __init__.py
    if [ -z "$flag" ]; then flag=1; continue; fi
    echo "======== $f"

    # initialization
    cp -n my_server/swagger_server/controllers/$f my_server/$f.common
    cp -n my_server/swagger_server/controllers/$f my_server/$f


    # real merge
    cp -f my_server/$f my_server/.backup/
    cp -f my_server/$f.common my_server/.backup/
    git merge-file my_server/$f my_server/$f.common my_server/swagger_server/controllers/$f
    cp -f my_server/swagger_server/controllers/$f otmini-repo/$f.common

done

rm -rf swagger-python-client
rm -rf swagger-python-server

Answer 5

Use connexion as @MrName suggested.

I first started using this together with codegen.

openapi-generator generate -i ../myapi.yaml -g python-flask -o .

This generates a directory with the openapi server.

  |- openapi_server\
      |--controllers\
           |--mytag._controller.py\
      |--openapi\
           |--my-api.yaml\

If you add tags to your paths in the api spec, then a separate tagname-controller.py is created for each tag. For each operationId a function is generated.

However, once this is set up, connexion can handle updates to the api spec. If I add a new path to openapi/my-api.yaml, with an operationId=new_func, then I can add new_func() to the existing controller. I don't lose the existing server logic (but I would still back it up before just in case). I haven't tried radical changes to existing paths yet.