Itertools module in python for data analytics

image courtesy: Dataquest.io

Itertools let you create combine multiple list into one or generate permutations, combinations of a set and to find subset in just a method call of itertools.

You can implement some fun and interesting tactics with it like creating infinite counting series of number, creating a cycle and make number repeat itself for times.

The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination.

Let’s generate a sequence of infinite positive number starting from 5 using itertools method count():

It will generally return an iterative variable that you can use in your looping, but in code commented out is the output how it will be actually

import itertools

for i in itertools.count(5):

 if i == 100:

   break;

 print(i)

# 5 6 7 8 9 10 11 12 …

# passing additional argument to method call you can set step size

for i in itertools.count(5,0.5):

 if i == 100:

   break;

 print(i)

# 5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0 …

Always make sure to give breaking condition in your loop unless you don’t want a loop to continue forever

What if we have to repeat in a loop of ABCD only? Just a simple method call and pass the sequence of alphabet or digit or symbols. cycle() is method that make the task simple let’s see:

import itertools

count = 0;

for i in itertools.cycle('ABCD'):

 if count > 50:

   break;

 print(i); count = count + 1;

# A B C D A B C D A B C ….

Suppose from a long set you have to generate all possible subset that can be form from taking two elements at a time. Using itertools.combinations() you can get all possible subsets. Argument could be numpy array or a list or a tuple or just a string.

import itertools

#list we pass to combinations() as a set

ls = [5,6,8,7,9,10,12]

for i in itertools.combinations(ls,2):

 print(i)

Let’s look at another interesting method that will be always come in handy if you are doing data cleaning. Here we have list of natural numbers from which we want only positive numbers and all other negative numbers to be dropped.

import itertools

age = [18,19,17,20,0,-20,24,-1]

filtered_age = []

cond = itertools.filterfalse(lambda x : x<=0,age)

for a in cond:

 filtered_age.append(a)

print(filtered_age)

It will output a list that only contains positive age, though age cannot be 0 or negative unless we are time travelling back ☺

+

Iterators terminating on the shortest input sequence:

Iterator	Arguments	Results	Example
accumulate()	p [,func]	p0, p0+p1, p0+p1+p2, …	accumulate([1,2,3,4,5]) --> 13 6 10 15
chain()	p, q, …	p0, p1, … plast, q0, q1, …	chain('ABC', 'DEF') --> A B CD E F
chain.from_iterable()	iterable	p0, p1, … plast, q0, q1, …	chain.from_iterable(['ABC','DEF']) --> A B C D E F
compress()	data, selectors	(d[0] if s[0]), (d[1] if s[1]), …	compress('ABCDEF',[1,0,1,0,1,1]) --> A C E F
dropwhile()	pred, seq	seq[n], seq[n+1], starting when pred fails	dropwhile(lambda x: x<5,[1,4,6,4,1]) --> 6 4 1
filterfalse()	pred, seq	elements of seq where pred(elem) is false	filterfalse(lambda x: x%2,range(10)) --> 0 2 4 6 8
groupby()	iterable[, key]	sub-iterators grouped by value of key(v)
islice()	seq, [start,] stop [, step]	elements from seq[start:stop:step]	islice('ABCDEFG', 2, None) -->C D E F G
starmap()	func, seq	func(*seq[0]), func(*seq[1]), …	starmap(pow, [(2,5), (3,2),(10,3)]) --> 32 9 1000
takewhile()	pred, seq	seq[0], seq[1], until pred fails	takewhile(lambda x: x<5,[1,4,6,4,1]) --> 1 4
tee()	it, n	it1, it2, … itn splits one iterator into n
zip_longest()	p, q, …	(p[0], q[0]), (p[1], q[1]), …	zip_longest('ABCD', 'xy',fillvalue='-') --> Ax By C- D-

Infinite iterators:

Iterator	Arguments	Results	Example
count()	start, [step]	start, start+step, start+2*step, …	count(10) --> 10 11 12 13 14...
cycle()	p	p0, p1, … plast, p0, p1, …	cycle('ABCD') --> A B C D AB C D ...
repeat()	elem [,n]	elem, elem, elem, … endlessly or up to n times	repeat(10, 3) --> 10 10 10

Combinatoric iterators:

Iterator	Arguments	Results
product()	p, q, … [repeat=1]	cartesian product, equivalent to a nested for-loop
permutations()	p[, r]	r-length tuples, all possible orderings, no repeated elements
combinations()	p, r	r-length tuples, in sorted order, no repeated elements
combinations_with_replacement()	p, r	r-length tuples, in sorted order, with repeated elements
product('ABCD', repeat=2)		AA AB AC AD BA BB BC BD CA CB  CC CD DADB DC DD
permutations('ABCD', 2)		AB AC AD BA BC BD CA CB CD  DA DB DC
combinations('ABCD', 2)		AB AC AD BC BD CD
combinations_with_replacement('ABCD',2)		AA AB AC AD BB BC BD CC CD DD

Source method list: official python documentation