rainier ababao reads and writes

Crazy Python one-liners

Motivation

In this post, we will explore a string manipulation task with the constraint that the solution is to be in as few lines as possible. Why? Just for fun. I would never commit an unclear, maybe inefficient one-liner into production code just to show off, and you shouldn’t either. However, I think this and similar constraint puzzles (in the category of what is called “code golf”) are really fun mental exercises in abstraction, problem decomposition, and mastering a language’s esoteric features.

The Slack for one of the student organizations I’m in has a channel called #politics. The channel’s pinned post is an ever-growing URL to a visualization of each channel member’s political views on the two axes of the political spectrum created by this website: “Libertarian-Authoritarian” and “Left-Right”.

The following is a sample URL that’s pinned in our channel (names replaced with the Greek alphabet):

https://www.politicalcompass.org/crowdchart?alpha=-3.38%2C-5.38&beta=-2.75%2C-4.62&gamma=-4.63%2C-4.15&delta=-4.75%2C-4.97&epsilon=-9.0%2C-8.31&zeta=-7.38%2C-4.36&eta=-3.5%2C-3.6&theta=-4.38%2C-2.92&iota=-7.13%2C-6.0&kappa=1.63%2C-2.36&lambda=-5.75%2C-6.51&mu=-3.25%2C-3.64&nu=1.75%2C-2.15&xi=-3.75%2C-5.08&name=omicron&ec=-4.88&soc=-2.05

And a sample image:

image

(Not surprising, considering that I go to the University of Texas). It would be nice to get a JSON encoding of this information so we might be able to integrate it into dataviz tools for practice.

Problem

Given an URL, such as:

https://www.politicalcompass.org/charts/crowdchart?alpha=-3.38%2C-5.38&beta=-2.75%2C-4.62&gamma=-4.63%2C-4.15&name=omicron&ec=-4.88&soc=-2.05

write a function to deserialize it into a JSON-like encoding, such as:

[
    {
        "alpha": {
            "economic": "-3.38",
            "social": "-5.38"
        }
    },
    {
        "beta": {
            "economic": "-2.75",
            "social": "-4.62"
        }
    },
    {
        "gamma": {
            "economic": "-4.63",
            "social": "-4.15"
        }
    },
    {
        "omicron": {
            "economic": "-4.88",
            "social": "-2.05"
        }
    }
]

using as few lines as possible.

Decomposition

We are mainly interested in the names of people and their econo-/socio- political preference scores, which are delimited by various characters. In a string s = 'https://www.politicalcompass.org/charts/crowdchart?alpha=-3.38%2C-5.38&beta=-2.75%2C-4.62&gamma=-4.63%2C-4.15&name=omicron&ec=-4.88&soc=-2.05', we can split it by ? to return a list object:

In [1]: s = 'https://www.politicalcompass.org/charts/crowdchart?alpha=-3.38%2C-5.38&beta=-2.75%2C-4.62&gamma=-4.63%2C-4.15&name=omicron&ec=-4.88&soc=-2.05'

In [2]: s.split('?')
Out[2]:
['https://www.politicalcompass.org/charts/crowdchart',
 'alpha=-3.38%2C-5.38&beta=-2.75%2C-4.62&gamma=-4.63%2C-4.15&name=omicron&ec=-4.88&soc=-2.05']

Now we can get the string we’re interested in using the expression s.split('?')[1]. In length-constrained coding, we want to minimize the number of new variable assignments, and this language’s AST lets us do this. Since s.split('?')[1] returns our string of interest, we can call .split('&') on that expression to obtain a list of strings:

In [3]: s.split('?')[1].split('&')
Out[3]:
['alpha=-3.38%2C-5.38',
 'beta=-2.75%2C-4.62',
 'gamma=-4.63%2C-4.15',
 'name=omicron',
 'ec=-4.88',
 'soc=-2.05']

We are getting closer to the form we would like, but there is some cleaning up to do with special cases at the end. As soon as a person (let’s say their name is omicron) adds their own name and scores to the list, the new HTTP GET parameters remain encoded in the URL (after the third-to-the-last &). (The endpoint handler can handle URLs without the last three GET parameters, but we don’t want the client to have to conform their parameters to the previous parameters by hand). Let’s make an expression to select the last 3 elements in the list to transform it into one element that conforms to the rest of the list:

In [4]: s.split('?')[1].split('&')[-3:]
Out[4]: ['name=omicron', 'ec=-4.88', 'soc=-2.05']

But we’re still only interested in the string parts after the =, and want to make a sequence of those to combine later. If we only focus on one string name=omicron, we can see that we want to call split('=') and grab only second element of its return, i.e., 'name=omicron'.split('=')[1]. But what if we wanted to apply this function to a list of strings?

Enter the lambda syntax. With lambda, we can convert a function into a one-liner.

lambda x: x.split('=')[1]

We can even assign it to a variable and call it like this:

In [5]: my_lambda = lambda x: x.split('=')[1]

In [6]: my_lambda('foo=bar')
Out[6]: 'bar'

Also note that it looks like a mathematical function, such as f(x). But like I said earlier, we’ll keep assigning expressions to variables to the minimum. It’s important to understand that the lambda definition always contains an expression (after the :) that is returned upon evaluation.

So back to the last question - how do we apply this function to a list of strings that have = in them?

Enter the map syntax. map(func, seq) applies function func to every element in sequence seq, and it always returns a sequence (such as a list or tuple) such that each element is the return of the function applied to that element. It’s probably easier to show you:

In [7]: map(my_lambda, ['foo=bar', 'baz=qux', 'ham=spam', 'eggs=bacon'])
Out[7]: ['bar', 'qux', 'spam', 'bacon']

Let’s use map and lambda to compose a list of the strings we were interested in, building on the list expression we composed earlier in [4]:

In [8]: map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:])
Out[8]: ['omicron', '-4.88', '-2.05']

Now use this list as arguments and the string '{}={}%2C{}' as the caller to the function str.format(*args) to get a string that conforms to the other strings (e.g., alpha=-3.38%2C-5.38) in the original list:

In [9]: '{}={}%2C{}'.format(*map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:]))
Out[9]: 'omicron=-4.88%2C-2.05'

(* is the “splat” operator, which converts a Python list into positional arguments in a function call.)

Before we append this string to the original list, let’s get all but the last 3 elements from the original list in [3]:

In [10]: s.split('?')[1].split('&')[:-3]
Out[10]: ['alpha=-3.38%2C-5.38', 'beta=-2.75%2C-4.62', 'gamma=-4.63%2C-4.15']

We can wrap the expression from [9] that returns the new string in brackets to convert it into a single-item list, and append it to the original list using the + operator:

In [11]: s.split('?')[1].split('&')[:-3] + ['{}={}%2C{}'.format(*map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:]))]
Out[11]:
['alpha=-3.38%2C-5.38',
 'beta=-2.75%2C-4.62',
 'gamma=-4.63%2C-4.15',
 'omicron=-4.88%2C-2.05']

We need to break up each string by = and %2C now. Hello, map and lambda again:

In [12]: map(lambda x: [x.split('=')[0]] + x.split('=')[1].split('%2C'), s.split('?')[1].split('&')[:-3] + ['{}={}%2C{}'.format(*map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:]))])
Out[12]:
[['alpha', '-3.38', '-5.38'],
 ['beta', '-2.75', '-4.62'],
 ['gamma', '-4.63', '-4.15'],
 ['omicron', '-4.88', '-2.05']]

And the final map/lambda combo:

In [13]: convert_to_dictlist = lambda s: map(lambda x: {x[0] : {'economic' : x[1], 'social' : x[2]}}, map(lambda x: [x.split('=')[0]] + x.split('=')[1].split('%2C'), s.split('?')[1].split('&')[:-3] + ['{}={}%2C{}'.format(*map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:]))]))

Let’s try it:

In [14]: convert_to_dictlist(s)
Out[14]:
[{'alpha': {'economic': '-3.38', 'social': '-5.38'}},
 {'beta': {'economic': '-2.75', 'social': '-4.62'}},
 {'gamma': {'economic': '-4.63', 'social': '-4.15'}},
 {'omicron': {'economic': '-4.88', 'social': '-2.05'}}]

It worked, and we defined the function in one line.

We can use print json.dumps() to print it out in a nicer looking format.

In [15]: import json

In [16]: print json.dumps(convert_to_dictlist(s), indent=4)
[
    {
        "alpha": {
            "economic": "-3.38",
            "social": "-5.38"
        }
    },
    {
        "beta": {
            "economic": "-2.75",
            "social": "-4.62"
        }
    },
    {
        "gamma": {
            "economic": "-4.63",
            "social": "-4.15"
        }
    },
    {
        "omicron": {
            "economic": "-4.88",
            "social": "-2.05"
        }
    }
]

Nice!!

The answer

Disclaimer

I didn’t get this one-liner immediately off the top of my head and on the first try. I use a fantastic REPL/interactive shell called IPython (where the In/Out lines come from) that makes it easy to quickly iterate (no pun intended) on composing Python code. There, I decomposed the problem, made dozens of intermediate variable assignments to heavily abstract, and then substituted the variables with the original statements and expressions they were assigned to.

The one-liner

lambda s: map(lambda x: {x[0] : {'economic' : x[1], 'social' : x[2]}}, map(lambda x: [x.split('=')[0]] + x.split('=')[1].split('%2C'), s.split('?')[1].split('&')[:-3] + ['{}={}%2C{}'.format(*map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:]))]))

Expanded for “clarity”

lambda s:
  map(
    lambda x:
      { x[0]: { 'economic' : x[1], 'social' : x[2] } }
      map(
        lambda x:
          [x.split('=')[0]] + x.split('=')[1].split('%2C'),
            s.split('?')[1].split('&')[:-3] + ['{}={}%2C{}'.format(*
            map(
              lambda x:
              x.split('=')[1],
                s.split('?')[1].split('&')[-3:]
            )
            )
          ]
      )
  }

Resources

Some links about map and lambda (kinda Python specific) since my explanation didn’t do it justice:

A pretty approachable introduction to lambda expressions in Python that demonstrates their usefulness in production code, scoping rules, and addresses common points of confusion: Yet Another Lambda Tutorial

Quora: Is it better to use list comprehensions or map with a lambda function when manipulating elements in a list?

Does your favorite programming language have syntax for map, filter, reduce, and/or lambda expressions?