Removing Duplicates from a List of Dictionaries

In certain scenarios, it may be necessary to remove duplicate entries from a list of dictionaries. Duplicates occur when multiple dictionaries contain the same set of keys and values.

To achieve this, one approach involves creating a temporary dictionary where the keys are the unique identifiers of each dictionary. This filters out duplicates because dictionaries can only have unique keys. The values of the temporary dictionary represent the original dictionaries.

In Python, this can be accomplished using a dictionary comprehension:

temp = {v["id"]: v for v in L}
unique_dicts = list(temp.values())

Here's an example:

L = [
    {"id": 1, "name": "john", "age": 34},
    {"id": 1, "name": "john", "age": 34},
    {"id": 2, "name": "hanna", "age": 30},
]

temp = {v["id"]: v for v in L}
unique_dicts = list(temp.values())

print(unique_dicts)

This will output:

[{'id': 1, 'name': 'john', 'age': 34}, {'id': 2, 'name': 'hanna', 'age': 30}]

As you can see, the duplicate dictionary has been removed, resulting in a list of unique dictionaries.