Hydrating Jinja with JSON as a Middle Step

Posted by mwguy on Fri 11 November 2022
Here I discussed a simple pattern for choosing different configurations with jinja
this is a little more depth into that pattern. By default jinja can only utilize strings to "hydrate into".
But sometimes you want things that are a little more in depth. For example, on of the things you may wish to configure
if a pattern that looks like this:
  • Make call to API.
  • Iterate through results (using something like pyjq ).
  • Make a call to a different API using specified arguments from the iterated object.

To make this a reality you could have a configuration (in yaml here as an exmaple) that looks like so:

---
calla: https://api.example.org/
calla_iterator: ".data[]"
callb: https://api.example.org/b
callb_args:
  query_arg_one: a
  query_arg_two: {{ thingfromdata }}

The idea is that in your configuration you can build a chain of events that tie together. This can be very useful when attempting to combine various rest api endpoints into a single object, but in a fashion where you don't have to write custom code every time. But when you use the "callb_args" in this example you may not wish to iterate through each item to make the call. Instead you can send it all!

# Loads the Dictionary to a string

for each_item in calla_iterator:

    # Load callB to a String
    json.loads(config[callb_args], default=str)

    # Make it a Jinja Template
    jtemplate = jinja2.Environment(loader=jinja2.BaseLoader).from_string(template)

    # Hydrate it
    rstring = jtemplate.render(**each_item)

    # Unstring it
    callb_args = json.dumps(rstring)

    # use it
    r = request.get(config["callb"], query=callb_args)
    ...

I made a gist that goes over the concept in a little more depth, but I think that there's a lot of ability to create code that has "more complex" configurations with this pattern.

tags: jinja, json, Python, yaml