In this post, we are going to understand different 7 ways to find duplicates in Python List with code examples. We will return a new list of duplicates items from the Python list. These ways can be used both string and int types of list items
1. Using Set with list comprehension
In this example, we are using list comprehension to check the elements that exist more than once in the original list, later converting it to a SET to get the unique count of elements and printing the newly created list.
# Program to find duplicates in the list and return list of duplicate items in list
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
duplicate_item_list = set([item for item in original_list if original_list.count(item) > 1])
print('duplicate items in list :',duplicate_item_list)
Output
duplicate items in list : {'C#', 13, 15}
2. Using collections module
In this example, we are using the collection module, checking which item represents more than once and creating a new list with duplicate items in the list, and printing the list.
# Program to find duplicates in the list and return list of duplicate items in list
import collections
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
duplicate_list_item = [item for item, count in collections.Counter(original_list).items() if count > 1]
print('new list of duplicate items :',duplicate_list_item)
Output
new list of duplicate items : [13, 15, 'C#']
3. Using for loop and dictionary
In this example, we are using the dictionary for loop to check if the item is not in the dictionary. We are then adding it into the dictionary else incrementing 1 each time item is present more than once and happening to Create a new list with duplicate items in the list, and printing the list.
# Program to find duplicates in the list and return list of duplicate items in Python
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
dict = {}
duplicate_item_list = []
for item in original_list:
if item not in dict:
dict[item] = 1
else:
if dict[item] == 1:
duplicate_item_list.append(item)
dict[item] += 1
print('duplicate items in list :',duplicate_item_list)
Output
duplicate items in list :[13, 15, 'C#']
4. Using iteration_utilities to find duplicate in list
In this example we are using the iteration_utilities modules methods to find duplicates in the list and creating a new list.
# Program to find duplicates in the list and return list of duplicate items in Python
from iteration_utilities import duplicates,unique_everseenfrom
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
dupes = list (unique_everseen(duplicates(original_list)))
print(dupes)
Output
duplicate items in list : [13, 15, 'C#']
5. Using Pandas
Using the Pandas module we can easily find the duplicate in the list and create a new list.
# Program to find duplicates in the list and return list of duplicate items in Python
import pandas as pd
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
pd_vec = pd.Series(original_list).value_counts()
list_dupes = pd_vec[pd_vec > 1].index.tolist()
print('duplicate items in list :',list_dupes)
Output
duplicate items in list : [13, 15, 'C#']
6. Counter to find duplicate in Python list
In this example, we are using the counter module to find the duplicate and create a new list of duplicate items.
# Program to find duplicates in the list and return list of duplicate items in Python
from collections import Counter
original_list = ['C++',13,15,13,15,17,'C#','Go','C#','C#','C#']
dupvalue = Counter(original_list) - Counter(set(original_list))
duplicate_list_items = list(dups.keys())
print('duplicate items in list:',duplicate_list_items)
Output
duplicate items in list : ['C#', 13, 15]
7. Using group by to find duplicate in Python list
In this example, we are using the itertools module groupby() method to find duplicates in the list and creating a new list of duplicate items in the list.
# Program to find duplicates in the list and return list of duplicate items in Python
from itertools import groupby
original_list = [13,15,13,15,17,17]
duplicate_item_list = []
for a, b in groupby(sorted(original_list)):
if len(list(b)) > 1:
duplicate_item_list.append(a)
print('duplicate item in a list :',duplicate_item_list)
Output
duplicate item in a list : [13, 15, 17]
Conclusion
In this post, we have explored 7 different ways to find duplicates and return a list of duplicate items in Python with code examples. You can use any of them as per your requirement. We hope that you find this post useful.