I was reading a blog article of a fellow Python developer. He wants to update all dictionaries inside a list. In old versions of Python there isn’t immediately a trivial way to solve this problem. The solution he made is pretty interesting as it contains a combination of list comprehensions, dictionaries and lambda functions. I tried to solve this issue by applying recent additions to the Python language to have a cleaner result.
Be sure to read the article at estebansastre.com
Merging dictionaries
As noted in the referred article, there isn’t a standard way in python to return an updated dictionary. This is because unlike most other data types, dictionaries are mutable. When applying functional programming concepts at them, the result might not always be as expected.
See the Python code below.
a = {'a key': 'a value'}
b = {'b key': 'b value'}
print(a.update(b)) # returns None
When reading the code one would expect to print {'a key': 'a value', 'b key': 'b value'}.
Why does it behave this way? Well, a dictionary is mutable. So the update() method updates the value in in memory.
It returns None because this is a part of the Python internals.
The correct solution for this problem would be something like this:
a = {'a key': 'a value'}
b = {'b key': 'b value'}
a.update(b) # None, the dictionary a is updated.
print(a) # {'a key': 'a value', 'b key': 'b value'}
Python 3.5 introduced unpacking generalizations as defined in PEP 448. When applied to the previous example, the result is something like this:
a = {'a key': 'a value'}
b = {'b key': 'b value'}
print(dict(**a, **b)) # {'a key': 'a value', 'b key': 'b value'}
See how easy that was!
Applying unpacking generalizations
I figured out that the unpacking generalizations technique can be applied to the algorithm to update a list of dictionaries using list list comprehension. Let’s rewrite the first algorithm:
[dict(**item, **ext) for item in original]
And apply this algorithm to the same examples:
original = [
{'title': 'Oh boy'},
{'title': 'Oh girl'}]
ext = {'meta': {'author': 'Myself'}}
print([dict(**item, **ext) for item in original])
# [{'title': 'Oh boy',
# 'meta': {'author': 'Myself'}},
# {'title': 'Oh girl',
# 'meta': {'author': 'Myself'}}]
What’s the catch? This example will probably only work in Python 3.5 or higher. I’ve tested this code in Python 3.6. Yet another good reason to move away from legacy Python 2 to a recent Python 3 release.
Full code
The full Python 3.6 (or higher) code that can be run locally and tweaked accordingly.
import unittest
import copy
from typing import List
ListOfDicts = List[dict]
class TestUpdatingListOfDicts(unittest.TestCase):
original = [
{'title': 'Oh boy'},
{'title': 'Oh girl'}]
ext1 = {'meta': {'author': 'Myself'}}
ext2 = dict()
result1 = [{
'title': 'Oh boy',
'meta': {'author': 'Myself'}},
{'title': 'Oh girl',
'meta': {'author': 'Myself'}}]
result2 = copy.deepcopy(original)
def esteban(self, original: ListOfDicts, ext: dict) -> ListOfDicts:
""""
:author Esteban Sastre
"""
return [(lambda x, y=item.copy(): (y.update(x), y))(ext)[1] for item in original]
def yennick(self, original: ListOfDicts, ext: dict) -> ListOfDicts:
"""
:author Yennick Schepers
"""
return [dict(**item, **ext) for item in original]
def test_result1_esteban(self):
result = self.esteban(original=self.original, ext=self.ext1)
self.assertEqual(self.result1, result)
def test_result1_yennick(self):
result = self.yennick(original=self.original, ext=self.ext1)
self.assertEqual(self.result1, result)
def test_result2_esteban(self):
result = self.esteban(original=self.original, ext=self.ext2)
self.assertEqual(self.result2, result)
def test_result2_yennick(self):
result = self.yennick(original=self.original, ext=self.ext2)
self.assertEqual(self.result2, result)
if __name__ == '__main__':
unittest.main()