Python provides the datetime module to work with real dates and times. Python enables us to schedule our Python script to run at a particular time. The date is not a data type, but we can work with the date objects by importing the modules named datetime, time, and calendar.
In this section of the tutorial, we will discuss how to work with the date and time objects in Python.
The datetime classes are classified into six main classes.
Date – It is a simple date. It consists of the year, month, and day as attributes.
Time – It is a perfect time, assuming every day has precisely 24*60*60 seconds. It has hour, minute, second, microsecond, and tzinfo as attributes.
datetime – It is a grouping of date and time, along with the attributes year, month, day, hour, minute, second, microsecond, and tzinfo.
timedelta – It represents the difference between two dates, times, or datetime instances to microsecond resolution.
tzinfo – It provides time zone information objects.
Timezone – It is included in the new version of Python. It is the class that implements the tzinfo abstract base class.
Tick
In Python, the time instants are counted since 12 AM, 1st January 1970. The function time() of the module time returns the total number of ticks spent since 12 AM, 1st January 1970. A tick can be seen as the smallest unit to measure time.
Python Tick Example
Let us take an example to demonstrate the tick in Python.
import time;
#prints the number of ticks spent since 12 AM, 1st January 1970
print(time.time())
Output:
1585928913.6519969
How to get the current time?
The localtime() functions of the time module are used to get the current time tuple. Consider the following example to get the current time using the localtime() function.
import time;
#returns a time tuple
print(time.localtime(time.time()))
The time is treated as the tuple of 9 numbers. Let’s look at the members of the time tuple.
Index
Attribute
Values
0
Year
4 digit (for example 2018)
1
Month
1 to 12
2
Day
1 to 31
3
Hour
0 to 23
4
Minute
0 to 59
5
Second
0 to 60
6
Day of weak
0 to 6
7
Day of year
1 to 366
8
Daylight savings
-1, 0, 1 , or -1
Getting formatted time
The time can be formatted by using the asctime() function of the time module. It returns the formatted time for the time tuple being passed.
Python Example to Getting Formatted Time
Let us take an example to demonstrate how to get formatted time in Python.
import time
#returns the formatted time
print(time.asctime(time.localtime(time.time())))
Output:
Wed Sep 10 13:52:19 2025
Python sleep time
The sleep() method of the time module is used to stop the execution of the script for a given amount of time. The output will be delayed for the number of seconds provided as a float.
Python sleep() time Method Example
Let us take an example to demonstrate the sleep() time method in Python.
import time
for i in range(0,5):
print(i)
#Each element will be printed after 1 second
time.sleep(1)
Output:
0
1
2
3
4
The datetime Module
The datetime module enables us to create custom date objects and perform various operations on dates, such as comparison. In order to work with dates as date objects, we have to import the datetime module into the Python source code.
Python datetime Module Example
Consider the following example to get the datetime object representation for the current time.
import datetime
#returns the current datetime object
print(datetime.datetime.now())
Output:
2025-09-10 13:54:25.140259
Creating date objects
We can create the date objects by passing the desired date in the datetime constructor for which the date objects are to be created.
Python Example to Create Date Objects
Here, we are going to take an example that demonstrate gow to create date objects in Python.
import datetime
#returns the datetime object for the specified date
print(datetime.datetime(2025,9,10))
Output:
2025-09-10 00:00:00
Python datetime Module Example with Custom Date and Time
We can also specify the time along with the date to create the datetime object. Consider the following example.
import datetime
#returns the datetime object for the specified time
print(datetime.datetime(2025,9,10,1,58,20))
Output:
2025-09-10 01:58:20
Explanation:
In the above code, we have passed in the datetime() function year, month, day, hour, minute, and millisecond attributes in a sequential manner.
Comparison of two dates
We can compare two dates by using the comparison operators like >, >=, <, and <=.
Python Example for Comparing Two Dates
Let us take an example to demonstrate how to compare two dates in Python.
from datetime import datetime as dt
#Compares the time. If the time is in between 8 AM and 4 PM, then it prints working hours; otherwise, it prints fun hours
if dt(dt.now().year,dt.now().month,dt.now().day,8)<dt.now()<dt(dt.now().year,dt.now().month,dt.now().day,16):
print("Working hours....")
else:
print("fun hours")
Output:
Working hours....
Calendar module
Python provides a calendar object that contains various methods to work with calendars.
Calender Module Example in Python
Consider the following example to print the calendar for the last month of 2018.
import calendar;
cal = calendar.month(2025,9)
#printing the calendar of December 2018
print(cal)
Output:
Printing the calendar for the whole year
The prcal() method of the calendar module is used to print the calendar of the entire year. The year for which the calendar is to be printed must be passed into this method.
import calendar
#printing the calendar of the year 2025
s = calendar.prcal(2025)
Simple Mail Transfer Protocol (SMTP) is used as a protocol to handle the email transfer using Python. It is used to route emails between email servers. It is an application layer protocol which allows to users to send mail to another. The receiver retrieves email using the protocols POP(Post Office Protocol) and IMAP(Internet Message Access Protocol).
When the server listens for the TCP connection from a client, it initiates a connection on port 587.
Python provides a smtplib module, which defines an the SMTP client session object used to send emails to an internet machine. For this purpose, we have to import the smtplib module using the import statement.
$ import smtplib
The SMTP object is used for the email transfer. The following syntax is used to create the smtplib object.
host: It is the hostname of the machine which is running your SMTP server. Here, we can specify the IP address of the server like (https://google.com) or localhost. It is an optional parameter.
port: It is the port number on which the host machine is listening to the SMTP connections. It is 25 by default.
local_hostname: If the SMTP server is running on your local machine, we can mention the hostname of the local machine.
The sendmail() method of the SMTP object is used to send the mail to the desired machine. The syntax is given below.
smtpObj.sendmail(sender, receiver, message)
Python Example for Sending Email using SMTP
Let us take an example to demonstrate how to senb Email using SMTP in Python.
#!/usr/bin/python3
import smtplib
sender_mail = '[email protected]'
receivers_mail = ['[email protected]']
message = """From: From Person %s
To: To Person %s
Subject: Sending SMTP e-mail
This is a test e-mail message.
"""%(sender_mail,receivers_mail)
try:
smtpObj = smtplib.SMTP('localhost')
smtpObj.sendmail(sender_mail, receivers_mail, message)
print("Successfully sent email")
except Exception:
print("Error: unable to send email")
Sending Email from Gmail
There are cases where the emails are sent using the Gmail SMTP server. In this case, we can pass Gmail as the SMTP server instead of using the localhost with the port 587.
If we want to send email from the gmail, we have to use the following syntax.
$ smtpObj = smtplib.SMTP("gmail.com", 587)
Here, we need to login to the Gmail account using Gmail user name and password. For this purpose, the smtplib provide the login() method, which accepts the username and password of the sender.
This may make your Gmail ask you for access to less secure apps if you’re using Gmail. You will need to turn this ON temporarily for this to work.
Python Example to Send Email from Gmail
Let’s take an example to demonstrate how to send Email from Gmail in Python.
#!/usr/bin/python3
import smtplib
sender_mail = '[email protected]'
receivers_mail = ['[email protected]']
message = """From: From Person %s
To: To Person %s
Subject: Sending SMTP e-mail
This is a test e-mail message.
"""%(sender_mail,receivers_mail)
try:
password = input('Enter the password');
smtpObj = smtplib.SMTP('gmail.com',587)
smtpobj.login(sender_mail,password)
smtpObj.sendmail(sender_mail, receivers_mail, message)
print("Successfully sent email")
except Exception:
print("Error: unable to send email")
Sending HTML in Email
We can format the HTML in the message by specifying the MIME version, content-type, and character set to send the HTML.’
Python Example to Send HTML in Email
Let us take an example to demonstrate how to send HTML in Email in Python.
#!/usr/bin/python3
import smtplib
sender_mail = '[email protected]'
receivers_mail = ['[email protected]']
message = """From: From Person %s
To: To Person %s
MIME-Version:1.0
Content-type:text/html
Subject: Sending SMTP e-mail
<h3>Python SMTP</h3>
<strong>This is a test e-mail message.</strong>
"""%(sender_mail,receivers_mail)
try:
smtpObj = smtplib.SMTP('localhost')
smtpObj.sendmail(sender_mail, receivers_mail, message)
print("Successfully sent email")
except Exception:
print("Error: unable to send email")
A Regular expression, abbreviated as RegEx, is a sequence of unique characters that forms a search pattern. We can use RegEx in order to check if a string contains the specified search pattern.
By matching a given pattern for a text, it can find the presence and absence of that text and also break the pattern into small parts.
Python Regex Module
Regular expressions in Python are handled via the “re” built-in package. The import statement can be used to import this module.
Python Regex Module Syntax
It has the following syntax:
# importing the re module
import re
How to Use RegEx in Python?
The following example searches for the word “platform” in a given string and prints its starting and ending index:
Example
# simple example to show the use of regular expression
# importing the re module
import re
# given string
str_1 = 'Tpoint Tech: An amazing platform to learn coding'
# searching for specified pattern in given string
matched_str = re.search(r'platform', str_1) # using the search() method
# printing the starting and ending index
print('Beginning Index:', matched_str.start())
print('Ending Index:', matched_str.end())
Output:
Beginning Index: 24
Ending Index: 32
Explanation:
In this example, we have used the search() function of the re module to locate the specified pattern in the given string. Here, the search pattern consisting of the character r in (r’platform’) stands for raw. We have then used the start() and end() functions to return the starting and ending index of the matched pattern in the string.
RegEx Functions in Python
In Python, the re module comprises of various functions that allow users to find, match, and manipulate the strings on the basis of the specified patterns.
Some of these functions are mentioned in the following table:
RegEx Function
Description
re.search()
It is used to locate the first occurrence of a character.
re.findall()
It is used to find and return all matching that occurs in a list.
re.compile()
It is used to compile the regular expressions into pattern objects.
re.split()
It is used to split the string on the basis of the occurrences of a specific character or pattern.
re.sub()
It is used to replace all occurrences of a character or pattern with a specified replacement string.
re.escape()
It is used to escape special characters.
Let us understand the working of these functions with the help of examples.
re.search()
The re.search() function is used to return the first occurrence of the pattern. This function searches the entire string and returns a match object if a match is found. Otherwise, it returns None in case the match is not found.
Python re.search() Function Example
Let us see an example to search a specified pattern in the given string.
Example
# importing the re module
import re
# given string
str_1 = 'I have been working as a Web Developer since 2023.'
regex_pattern = r"([a-zA-Z]+) (\d+)"
# searching for specified pattern in given string
matched_str = re.search(regex_pattern, str_1) # using the search() method
# checking the returned object
if matched_str:
# printing the matched pattern details
print('Match Found:', matched_str.group())
print('Beginning Index:', matched_str.start())
print('Ending Index:', matched_str.end())
else:
print('Match Not Found')
Output:
Match Found: since 2023
Beginning Index: 39
Ending Index: 49
Explanation:
In the above example, we have used the search() function of the re module to find the occurrence of a word followed by a number. Since, such pattern exists in the given string as ‘since 2023’, it was returned. We have then used the group() function to print the matched pattern along with the starting and ending index() with the help of start() and end() functions, respectively.
re.findall()
The re.findall() function is used to return a list of the non-overlapping matches in the given string. Unlike, the search() function which only returns the first match, the findall() function returns all matches as a list.
Python re.findall() Function Example
Let us now take a look at a simple example:
Example
# importing the re module
import re
# given string
str_1 = """My house no. is 4567 and
my office no. is 8910."""
regex_pattern = r"([a-zA-Z]+) (\d+)"
# searching all occurrences in the given string
matched_str_list = re.findall(regex_pattern, str_1)
# checking the returned object
if matched_str_list:
print(matched_str_list)
else:
print("No match found")
Output:
[('is', '4567'), ('is', '8910')]
Explanation:
In this example, we used the findall() function from the re module to find all the occurrences of the specified pattern in the given string and stored the returned object in a variable. We then printed the list of matched patterns.
re.compile()
The re.compile() function is used to compile a regular expression pattern into a reusable regex object, allowing us to utilize its methods (such as search(), findall(), etc.), multiple times without rewriting the pattern.
Python re.compile() Function Example
Here is an example showing the use of compile() function:
Example
# importing the re module
import re
# given string
str_1 = "Welcome to Tpoint Tech."
# using compile() function
regex_pattern = re.compile('[a-e]')
# searching all occurrences in the given string
matched_str_list = re.findall(regex_pattern, str_1)
if matched_str_list:
print(matched_str_list)
else:
print("No match found")
Output:
['e', 'c', 'e', 'e', 'c']
Explanation:
In this example, we have used the re.compile() function to compile a regular expression pattern into a reusable regex object. We have then called the findall() function to search all occurrences in the given string.
re.split()
The re.split() function is used to split a string wherever the regex pattern matches, similar to the str.split() function; however, with more powerful pattern support.
Python re.split() Function Example
Let us take a look at the following example to demonstrate the re.split() function in Python.
Example
# importing the re module
import re
# given string
str_1 = "mango banana,apple;orange,cherry"
# using the split() function
regex_pattern = r'[;,\s]'
# splitting on semicolon, comma, or space
matched_str_list = re.split(regex_pattern, str_1)
if matched_str_list:
print(matched_str_list)
else:
print("No match found")
Output:
['mango', 'banana', 'apple', 'orange', 'cherry']
Explanation:
Here, we have used the split() function from re module to split the given string wherever the specified regex pattern matches.
re.sub()
The re.sub() function is a function of re module used to replace all occurrences of a regex pattern in a string with a replacement string. This function is like a “find and replace” using regex patterns.
Python re.sub() Function Example
We will now look at the following example to demonstrate the re.sub() function in Python.
Example
# importing the re module
import re
# given string
original_str = "Roses are red, Violets are blue."
print("Original String:", original_str)
# pattern and replacement
pattern = "red"
replacement = "white"
# using the sub() function
new_str = re.sub(pattern, replacement, original_str)
print("New String:", new_str)
Output:
Original String: Roses are red, Violets are blue.
New String: Roses are white, Violets are blue.
Explanation:
Here, we have used the sub() function from re module to find the specified pattern in the given string and replace it with the specified replacement.
re.subn()
The subn() function is another function of re module, that works just like sub() function. However, it returns a tuple consisting of the new string along with the number of changes done in the given string.
Python re.subn() Function Example
Let us see the following example to understand the working of the re.subn() function.
Example
# importing the re module
import re
# given string
original_str = "This building has 4 floors. There are 3 flats on each floor."
# pattern and replacement
pattern = r'\d+'
replacement = 'many'
# using the subn() function
new_str, num_subs = re.subn(pattern, replacement, original_str)
# printing the results
print("Original string:", original_str)
print("New string:", new_str)
print("Number of substitutions:", num_subs)
Output:
Original string: This building has 4 floors. There are 3 flats on each floor.
New string: This building has many floors. There are many flats on each floor.
Number of substitutions: 2
Explanation:
In this example, we used the re.subn() function to replace all the occurrence of the specified pattern in the given string with the replacement. We then stored the new string and number of substitutions and printed them.
re.escape()
The escape() function is a function of re module, that is used to escape all special characters in a string so that it can be used safely as a literal in a regular expression.
Python re.escape() Function Example
Here is an example showing the use of the re.escape() function in Python.
Example
# importing the re module
import re
# using the re.escape() function
print(re.escape("Welcome to Python App"))
print(re.escape("We've \t learned various [a-9] concepts of& Python ^!"))
Here, we have used the re.escape() function to escape the special characters from the given strings.
Meta-characters in Python Regex
In regex, meta-characters are said to be the special characters that control the way patterns are matched. These characters are not treated as literal unless they are escaped with a backslash “\”.
The following is a table consisting of various meta-characters in Python regex:
Meta-Character
Description
\
It is used to drop the special meaning of character following it
[]
It represents a character class
^
It is used to match the beginning
$
It is used to match the end
.
It is used match any character except newline
|
It means OR (Matches with any of the characters separated by it.
?
It matches zero or one occurrence
*
It is used to show any number of occurrences (including 0 occurrences)
+
It is used to display one or more occurrences
{}
It indicates the number of occurrences of a preceding regex to match.
()
It encloses a group of Regex
Meta-characters Example in Python Regex
Let us see an example of meta-characters in Python regex.
Example
# importing the re module
import re
# The . meta-character matches any character (except newline)
text = "cat bat sat mat"
pattern = r"at" # Matches 'at' in any word
matches = re.findall(pattern, text)
print(matches)
# The ^ meta-character matches the start of the string
text = "The quick brown fox"
pattern = r"^The" # Matches 'The' only if it is at the beginning
matches = re.findall(pattern, text)
print(matches)
# The $ meta-character matches the end of the string
text = "The quick brown fox"
pattern = r"fox$" # Matches 'fox' only if it is at the end
matches = re.findall(pattern, text)
print(matches)
# The * meta-character matches zero or more occurrences of the preceding character
text = "ab abb abbb"
pattern = r"ab*" # Matches 'a' followed by zero or more 'b's
matches = re.findall(pattern, text)
print(matches)
# The + meta-character matches one or more occurrences of the preceding character
text = "ab abb abbb"
pattern = r"ab+" # Matches 'a' followed by one or more 'b's
matches = re.findall(pattern, text)
print(matches)
In this example, we can see the use of several meta-characters, such as ., ^, $, *, and +. We have used these meta-characters in different regex pattern and used the findall() function to find all the matches in the given strings.
Special Sequences in Python Regex
Special Sequences are shorthand notations used to represent common character classes or positions. These sequences begin with a backslash “\” followed by a letter or symbol.
The following is the list of commonly used special sequences in regex:
Special Sequence
Description
\d
Digit (0 – 9)
\D
Non-digit
\w
Word character (letters, digits, underscore)
\W
Non-word character
\s
Whitespace (space, tab, newline)
\S
Non-whitespace
\b
Word boundary
\B
Not a word boundary
\A
Start of string
\Z
End of string
Special Sequences Example in Python Regex
We will now see the following example of special sequences in Python regex.
Example
# importing the re module
import re
# \d Matches a digit
txt = "Welcome to Tpoint 123 Tech"
x = re.findall("\d", txt)
print(x)
# \S Matches a non-whitespace character
txt = "tpoint tech"
x = re.findall("\S", txt)
print(x)
# \w Matches a word character (alphanumeric + underscore)
txt = "tpoint world_123"
x = re.findall("\w", txt)
print(x)
# \b Matches the boundary between a word and a non-word character
txt = "tpoint tech"
x = re.findall(r"\btech\b", txt)
print(x)
In this example, we can see the use of different special sequences like \d, \S, \w, and \b in regex.
Match Objects in Python Regex
When we use functions like re.search() or re.match(), they return a match object in case a match is found. This object consists of the details regarding the match, including its content and position.
Match Objects Example in Python Regex
Let us see an example to demonstrate the match objects in Python regex.
Example
# importing the re module
import re
# given string
str_1 = "He is a boy and he plays cricket everyday"
# regex pattern
regex_pattern = r"he"
# Finding the first occurrence and return a match object
match_obj = re.search(regex_pattern, str_1, re.IGNORECASE)
if match_obj:
print(f"Search match object: {match_obj}")
print(f"Match start index: {match_obj.start()}")
print(f"Match end index: {match_obj.end()}")
print(f"Matched string: {match_obj.group(0)}")
Output:
Search match object: <re.Match object; span=(0, 2), match='He'>
Match start index: 0
Match end index: 2
Matched string: He
Explanation:
In this example, we can see the use of different methods of match object in regex.
The following is a list of commonly used methods of match object in Python regex:
Method
Description
.start()
It returns the start index of the match
.end()
It returns the end index of the match
.span()
It returns a tuple of (start, end)
.group()
It returns the matched string
.groups()
It returns all capture groups as a tuple
.group(n)
It returns the nth capture group
Conclusion
In this tutorial, we learned about the regular expression in Python programming language. We understood how regex work in Python and discovered its various functions. We also learn about other important concepts like meta-characters, special sequences, and match objects.
In Python, multiprocessing is the ability of the system to run one or more processes in parallel. In simple words, multiprocessing uses the two or more CPU within the single computer system. This method is also capable to allocate the tasks between more than one process.
Processing units share the main memory and peripherals to process programs simultaneously. Multiprocessing Application breaks into smaller parts and runs independently. Each process is allocated to the processor by the operating system.
Python provides the built-in package called multiprocessing which supports swapping processes. Before working with the multiprocessing, we must aware with the process object.
Why Multiprocessing?
Multiprocessing is essential to perform the multiple tasks within the Computer system. Suppose a computer without multiprocessing or single processor. We assign various processes to that system at the same time.
It will then have to interrupt the previous task and move to another to keep all processes going. It is as simple as a chef is working alone in the kitchen. He has to do several tasks to cook food such as cutting, cleaning, cooking, kneading dough, baking, etc.
Therefore, multiprocessing is essential to perform several task at the same time without interruption. It also makes easy to track all the tasks. That is why the concept of multiprocessing is to arise.
Multiprocessing can be represented as a computer with more than one central processor.
A Multi-core processor refers to single computing component with two or more independent units.
In the multiprocessing, the CPU can assign multiple tasks at one each task has its own processor.
Multiprocessing In Python
Python provides the multiprocessing module to perform multiple tasks within the single system. It offers a user-friendly and intuitive API to work with the multiprocessing.
Python Multiprocessing Example
Let’s understand the simple example of multiple processing in Python.
from multiprocessing import Process
def disp():
print ('Hello !! Welcome to Python Tutorial')
if __name__ == '__main__':
p = Process(target=disp)
p.start()
p.join()
Output:
'Hello !! Welcome to Python Tutorial'
Explanation:
In the above code, we have imported the Process class then create the Process object within the disp() function. Then we started the process using the start() method and completed the process with the join() method. We can also pass the arguments in the declared function using the args keywords.
Python Multprocessing Example with Arguments
Let’s understand the following example of the multiprocessing with arguments.
# Python multiprocessing example
# importing the multiprocessing module
import multiprocessing
def cube(n):
# This function will print the cube of the given number
print("The Cube is: {}".format(n * n * n))
def square(n):
# This function will print the square of the given number
print("The Square is: {}".format(n * n))
if __name__ == "__main__":
# creating two processes
process1 = multiprocessing.Process(target= square, args=(5, ))
process2 = multiprocessing.Process(target= cube, args=(5, ))
# Here we start the process 1
process1.start()
# Here we start process 2
process2.start()
# The join() method is used to wait for process 1 to complete
process1.join()
# It is used to wait for process 1 to complete
process2.join()
# Print if both processes are completed
print("Both processes are finished")
Output:
The Cube is: 125
The Square is: 25
Both processes are finished
Explanation –
In the above example, We created the two functions – the cube() function calculates the given number’s cube, and the square() function calculates the square of the given number.
Next, we defined the process object of the Process class that has two arguments. The first argument is a target that represents the function to be executed, and the second argument is args that represents the argument to be passed within the function.
We have used the start() method to start the process.
process1.start()
process2.start()
As we can see in the output, it waits to completion of process one and then process 2. The last statement is executed after both processes are finished.
Python Multiprocessing Classes
Python multiprocessing module provides many classes which are commonly used for building parallel program. We will discuss its main classes – Process, Queue and Lock. We have already discussed the Process class in the previous example. Now, we will discuss the Queue and Lock classes.
Python Multiprocessing Classes Example
Let’s see the simple example of a get number of CPUs currently in the system.
import multiprocessing
print("The number of CPU currently working in system : ", multiprocessing.cpu_count())
Output:
('The number of CPU currently woking in system : ', 32)
The above number of CPUs can vary for your pc. For us, the number of cores is 32.
Python Multiprocessing Using Queue Class
We know that Queue is important part of the data structure. Python multiprocessing is precisely the same as the data structure queue, which based on the “First-In-First-Out” concept. Queue generally stores the Python object and plays an essential role in sharing data between processes.
Queues are passed as a parameter in the Process’ target function to allow the process to consume data. The Queue provides the put() function to insert the data and get() function to get data from the queues.
Python Multiprocessing Example using Queue Class
Let us take an example to demonstrate the multiprocessing using Queue class in Python.
# Importing Queue Class
from multiprocessing import Queue
fruits = ['Apple', 'Orange', 'Guava', 'Papaya', 'Banana']
count = 1
# creating a queue object
queue = Queue()
print('pushing items to the queue:')
for fr in fruits:
print('item no: ', count, ' ', fr)
queue.put(fr)
count += 1
print('\npopping items from the queue:')
count = 0
while not queue.empty():
print('item no: ', count, ' ', queue.get())
count += 1
In the above code, we have imported the Queue class and initialized the list named fruits. Next, we assigned a count to 1. The count variable will count the total number of elements. Then, we created the queue object by calling the Queue() method. This object will used to perform operations in the Queue. In for loop, we inserted the elements one by one in the queue using the put() function and increased the count by 1 with each iteration of loop.
Python Multiprocessing Lock Class
The multiprocessing Lock class is used to acquire a lock on the process so that we can hold the other process to execute a similar code until the lock has been released. The Lock class performs mainly two tasks. The first is to acquire a lock using the acquire() function and the second is to release the lock using the release() function.
Python Multiprocessing Lock Class Example
Suppose we have multiple tasks. So, we create two queues: the first queue will maintain the tasks, and the other will store the complete task log. The next step is to instantiate the processes to complete the task. As discussed previously, the Queue class is already synchronized, so we don’t need to acquire a lock using the Lock class.
In the following example, we will merge all the multiprocessing classes together. Let’s see the below example.
from multiprocessing import Lock, Process, Queue, current_process
import time
import queue
def jobTodo(tasks_to_perform, complete_tasks):
while True:
try:
# The try block to catch task from the queue.
# The get_nowait() function is used to
# raise queue.Empty exception if the queue is empty.
task = tasks_to_perform.get_nowait()
except queue.Empty:
break
else:
# if no exception has been raised, the else block will execute
# add the task completion
print(task)
complete_tasks.put(task + ' is done by ' + current_process().name)
time.sleep(.5)
return True
def main():
total_task = 8
total_number_of_processes = 3
tasks_to_perform = Queue()
complete_tasks = Queue()
number_of_processes = []
for i in range(total_task):
tasks_to_perform.put("Task no " + str(i))
# defining number of processes
for w in range(total_number_of_processes):
p = Process(target=jobTodo, args=(tasks_to_perform, complete_tasks))
number_of_processes.append(p)
p.start()
# completing process
for p in number_of_processes:
p.join()
# print the output
while not complete_tasks.empty():
print(complete_tasks.get())
return True
if __name__ == '__main__':
main()
Output:
Task no 2
Task no 5
Task no 0
Task no 3
Task no 6
Task no 1
Task no 4
Task no 7
Task no 0 is done by Process-1
Task no 1 is done by Process-3
Task no 2 is done by Process-2
Task no 3 is done by Process-1
Task no 4 is done by Process-3
Task no 5 is done by Process-2
Task no 6 is done by Process-1
Task no 7 is done by Process-3
Python Multiprocessing Pool
Python multiprocessing pool is essential for parallel execution of a function across multiple input values. It is also used to distribute the input data across processes (data parallelism).
Python Multiprocessing Pool Example
Consider the following example of a multiprocessing Pool.
from multiprocessing import Pool
import time
w = (["V", 5], ["X", 2], ["Y", 1], ["Z", 3])
def work_log(data_for_work):
print(" Process name is %s waiting time is %s seconds" % (data_for_work[0], data_for_work[1]))
time.sleep(int(data_for_work[1]))
print(" Process %s Executed." % data_for_work[0])
def handler():
p = Pool(2)
p.map(work_log, w)
if __name__ == '__main__':
handler()
Output:
Process name is V waiting time is 5 seconds
Process V Executed.
Process name is X waiting time is 2 seconds
Process X Executed.
Process name is Y waiting time is 1 seconds
Process Y Executed.
Process name is Z waiting time is 3 seconds
Process Z Executed.
Another Python Example for Multiprocessing Pool
Let’s understand another example of the multiprocessing Pool.
from multiprocessing import Pool
def fun(x):
return x*x
if __name__ == '__main__':
with Pool(5) as p:
print(p.map(fun, [1, 2, 3]))
Output:
[1, 8, 27]
Proxy Objects
The proxy objects are referred to as shared objects which reside in a different process. This object is also called as a proxy. Multiple proxy objects might have a similar referent. A proxy object consists of various methods which are used to invoked corresponding methods of its referent.
Python Proxy Objects Example
Let us take an example to demonstrate the proxy objects in Python.
from multiprocessing import Manager
manager = Manager()
l = manager.list([i*i for i in range(10)])
print(l)
print(repr(l))
print(l[4])
print(l[2:5])
The proxy objects are picklable so we can pass them between processes. These objects are also used for level of control over the synchronization.
Commonly Used Functions of Multiprocessing
So far, we have discussed the basic concepts of multiprocessing using Python. Multiprocessing is a broad topic itself and essential for performing various tasks within a single system. We are defining a few essential functions that are commonly used to achieve multiprocessing.
Method
Description
pipe()
The pipe() function returns a pair of connection objects.
run()
The run() method is used to represent the process activities.
start()
The start()method is used to start the process.
join([timeout])
The join() method is used to block the process until the process whose join() method is called terminates. The timeout is optional argument.
is_alive()
It returns if process is alive.
terminate()
As the name suggests, it is used to terminate the process. Always remember – the terminate() method is used in Linux, for Windows, we use TerminateProcess() method.
kill()
This method is similar to the terminate() but using the SIGKILL signal on Unix.
close()
This method is used to close the Process object and releases all resources associated with it.
qsize()
It returns the approximate size of the queue.
empty()
If queue is empty, it returns True.
full()
It returns True, if queue is full.
get_await()
This method is equivalent get(False).
get()
This method is used to get elements from the queue. It removes and returns an element from queue.
put()
This method is used to insert an element into the queue.
cpu_count()
It returns the number of working CPU within the system.
current_process()
It returns the Process object corresponding to the current process.
parent_process()
It returns the parent Process object corresponding to the current process.
task_done()
This function is used indicate that an enqueued task is completed.
Python generator is a special type of function that returns an iterator object. Instead of returning a value, it uses yield to give a sequence of values over time, which pauses after each yield statement while keeping its state between iterations.
Generators are memory efficient, which means making items one at a time, with the help of yield keyword.
Python Generators Example
Let us take a simple example to demonstrate the Generator in Python:
Example
def generate_numbers(limit):
for num in range(1, limit + 1):
yield num
sequence = generate_numbers(5)
for value in sequence:
print(value)
Output:
1
2
3
4
5
Explanation:
In the above code, a generate_numbers function is defined, which accepts a limit as argument, that executes a for loop in the function range from 1 to limit + 1, and produces the number. The function is called with a limit value of 5 and prints the yielded value.
How to Define a Generator in Python?
It is easy to create a generator, simply define a function that has at least one yield statement in the function. When the function is called, instead of returning a single value, you will get back a generator object, which is an iterator.
Syntax:
Here is the standard syntax to define the generator in Python.
def function(parameters):
yield statement
Python Generator Example
Let’s take an example to demonstrate how to use generators in Python.
Example
def number_generator():
numbers = [1, 2, 3]
for num in numbers:
yield num
# Running the generator to display its output
for item in number_generator():
print(item)
Output:
1
2
3
Explanation:
The above code in the example defines a generator function ‘number_generator’ which generates the values from a list [1, 2, 3] one at a time with the help of a for loop. Each call to the generator function provides the next value in sequence and can be more memory-efficient, which involves iterating over the data in a lazy manner.
Difference between Yield and Return
The yield statement is used in a generator function to get a sequence of values over time. When the statement yield produces, it pauses the running of the function, returns the values, and saves the function statement, so when it is resumed later, it will continue from where it left off. This is especially useful for working efficiently with large or complicated data streams.
On the other hand, the return statement will give the result immediately and give you back a single final value. The function will end when the return statement is called, and the local state will not be preserved, which is helpful when the function is only needed to return one result.
Example
def calculate_total():
values = [1, 2, 3]
result = sum(values)
return result
output = calculate_total()
print(output)
Output:
6
Explanation:
The function calculate_total() makes a list of numbers based on Python’s built-in sum() function to add the numbers. The function results in the total number and is displayed. This method is simple, easy to understand, and can be quickly updated to use more values.
Python Generator Expression
Generator expressions enable generators to be more compact. They are similar to lists but use parentheses rather than square brackets, and are more memory efficient.
Python Generator Expression Example
Let us take an example to demonstrate the generator expression in Python.
Example
squares_generator = (num ** 2 for num in range(1, 6))
for square in squares_generator:
print(square)
Output:
1
4
9
16
25
Explanation:
The code uses a generator expression to generate the squares of a given number. The generator yields a single square at any time, which is then displayed at the time of the loop. This is efficient regarding memory as it does not have the whole list in memory.
Application of Generators in Python
Several applications of generators in Python are as follows:
Using a generator to produce a stream of Fibonacci numbers is convenient because to get the next number, you can just call next(x) without worrying about the end of the sequence to interrupt us.
For a more common use case of generators, we can also use them to process large data files, like log files, where they can offer a memory-efficient way of handling just the piece we want.
Generators can also work as iterators, but are generally faster since you don’t have to define the functions for __next__ and __iter__ methods.
Conclusion
In Python, generators are an excellent method for working on large datasets and for generating complex sequences of results. The yield statement enables a function to generate one result while maintaining its state between iterations. This is very useful when working on large files or generating infinite sequences like the Fibonacci numbers. Generators are beneficial when compared with a function that simply returns, as a generator can generate more data for less memory, generating more values at a time in a given time interval.
Decorators are one of the most helpful and powerful tools of Python. These are used to modify the behavior of the function. Decorators provide the flexibility to wrap another function to expand the working of wrapped function, without permanently modifying it.
In Decorators, functions are passed as an argument into another function and then called inside the wrapper function.
It is also called meta programming where a part of the program attempts to change another part of program at compile time.
Before understanding the Decorator, we need to know some important concepts of Python.
What are the functions in Python?
Python has the most interesting feature that everything is treated as an object even classes or any variable we define in Python is also assumed as an object. Functions are first-class objects in the Python because they can reference to, passed to a variable and returned from other functions as well. The example is given below:
Python Function Example
Let’s consider an example to demonstrate the function in Python.
def func1(msg): # here, we are creating a function and passing the parameter
print(msg)
func1("Hii, welcome to function ") # Here, we are printing the data of function 1
func2 = func1 # Here, we are copying the function 1 data to function 2
func2("Hii, welcome to function ") # Here, we are printing the data of function 2
Output:
Hii, welcome to function
Hii, welcome to function
Explanation: In the above program, when we run the code it give the same output for both functions. The func2referred to function func1 and act as function. We need to understand the following concept of the function:
The function can be referenced and passed to a variable and returned from other functions as well.
The functions can be declared inside another function and passed as an argument to another function.
Inner Function
Python provides the facility to define the function inside another function. These types of functions are called inner functions. Consider the following example:
Python Inner Function Example
Let’s consider an example to demonstrate the inner function in Python.
def func(): # here, we are creating a function and passing the parameter
print("We are in first function") # Here, we are printing the data of function
def func1(): # here, we are creating a function and passing the parameter
print("This is first child function") # Here, we are printing the data of function 1
def func2(): # here, we are creating a function and passing the parameter
print("This is second child function") # Here, we are printing the data of # function 2
func1()
func2()
func()
Output:
We are in first function
This is first child function
This is second child function
Explanation: In the above program, it doesn’t matter how the child functions are declared. The execution of the child function makes effect on the output. These child functions are locally bounded with the func() so they cannot be called separately.
Higher Order Function
A function that accepts other function as an argument is also called higher order function. Consider the following example:
Example:
def add(x): # here, we are creating a function add and passing the parameter
return x+1 # here, we are returning the passed value by adding 1
def sub(x): # here, we are creating a function sub and passing the parameter
return x-1 # here, we are returning the passed value by subtracting 1
def operator(func, x): # here, we are creating a function and passing the parameter
temp = func(x)
return temp
print(operator(sub,10)) # here, we are printing the operation subtraction with 10
print(operator(add,20)) # here, we are printing the operation addition with 20
Output:
9
21
Explanation: In the above program, we have passed the sub() function and add() function as argument in operator() function.
A function can return another function. Consider the below example:
Example:
def hello(): # here, we are creating a function named hello
def hi(): # here, we are creating a function named hi
print("Hello") # here, we are printing the output of the function
return hi # here, we are returning the output of the function
new = hello()
new()
Output:
Hello
Explanation: In the above program, the hi() function is nested inside the hello() function. It will return each time we call hi().
Decorating functions with parameters
Let’s have an example to understand the parameterized decorator function:
Example:
def divide(x,y): # here, we are creating a function and passing the parameter
print(x/y) # Here, we are printing the result of the expression
def outer_div(func): # here, we are creating a function and passing the parameter
def inner(x,y): # here, we are creating a function and passing the parameter
if(x<y):
x,y = y,x
return func(x,y)
# here, we are returning a function with some passed parameters
return inner
divide1 = outer_div(divide)
divide1(2,4)
Output:
2.0
Syntactic Decorator
In the above program, we have decorated out_div() that is little bit bulky. Instead of using above method, Python allows to use decorator in easy way with @symbol. Sometimes it is called “pie” syntax.
Syntactoc Decorator Example:
Let’s consider an example to demonstrate the syntactic decorator in Python.
def outer_div(func): # here, we are creating a function and passing the parameter
def inner(x,y): # here, we are creating a function and passing the parameter
if(x<y):
x,y = y,x
return func(x,y) # here, we are returning the function with the parameters
return inner
# Here, the below is the syntax of generator
@outer_div
def divide(x,y): # here, we are creating a function and passing the parameter
print(x/y)
Output:
2.0
Reusing Decorator
We can reuse the decorator as well by recalling that decorator function. Let’s make the decorator to its own module that can be used in many other functions. Creating a file called mod_decorator.py with the following code:
def do_twice(func): # here, we are creating a function and passing the parameter
def wrapper_do_twice():
# here, we are creating a function and passing the parameter
func()
func()
return wrapper_do_twice
We can import mod_decorator.py in another file.
from decorator import do_twice
@do_twice
def say_hello():
print("Hello There")
say_hello()
We can import mod_decorator.py in other file.
from decorator import do_twice
@do_twice
def say_hello():
print("Hello There")
say_hello()
Output:
Hello There
Hello There
Python Decorator with Argument
We want to pass some arguments in function. Let’s do it in following code:
from decorator import do_twice
@do_twice
def display(name):
print(f"Hello {name}")
display()
As we can see that, the function didn’t accept the argument. Running this code raises an error. We can fix this error by using *args and **kwargsin the inner wrapper function. Modifying the decorator.pyas follows:
Now wrapper_function() can accept any number of argument and pass them on the function.
from decorator import do_twice
@do_twice
def display(name):
print(f"Hello {name}")
display("John")
Output:
Hello John
Hello John
Returning Values from Decorated Functions
We can control the return type of the decorated function. The example is given below:
from decorator import do_twice
@do_twice
def return_greeting(name):
print("We are created greeting")
return f"Hi {name}"
hi_adam = return_greeting("Adam")
Output:
We are created greeting
We are created greeting
Fancy Decorators
Let’s understand the fancy decorators by the following topic:
Class Decorators
Python provides two ways to decorate a class. Firstly, we can decorate the method inside a class; there are built-in decorators like @classmethod, @staticmethod and @property in Python. The @classmethod and @staticmethod define methods inside class that is not connected to any other instance of a class. The @property is generally used to modify the getters and setters of a class attributes. Let’s understand it by the following example:
Example: 1-
@property decorator – By using it, we can use the class function as an attribute. Consider the following code:
class Student: # here, we are creating a class with the name Student
def __init__(self,name,grade):
self.name = name
self.grade = grade
@property
def display(self):
return self.name + " got grade " + self.grade
stu = Student("John","B")
print("Name of the student: ", stu.name)
print("Grade of the student: ", stu.grade)
print(stu.display)
Output:
Name of the student: John
Grade of the student: B
John got grade B
Example: 2-
@staticmethod decorator– The @staticmethod is used to define a static method in the class. It is called by using the class name as well as instance of the class. Consider the following code:
class Person: # here, we are creating a class with the name Student
@staticmethod
def hello(): # here, we are defining a function hello
print("Hello Peter")
per = Person()
per.hello()
Person.hello()
Output:
Hello Peter
Hello Peter
Singleton Class
A singleton class only has one instance. There are many singletons in Python including True, None, etc.
Nesting Decorators
We can use multiple decorators by using them on top of each other. Let’s consider the following example:
In the above code, we have used the nested decorator by stacking them onto one another.
Decorator with Arguments
It is always useful to pass arguments in a decorator. The decorator can be executed several times according to the given value of the argument. Let us consider the following example:
Example:
Import functools # here, we are importing the functools into our program
def repeat(num): # here, we are defining a function repeat and passing parameter
# Here, we are creating and returning a wrapper function
def decorator_repeat(func):
@functools.wraps(func)
def wrapper(*args,**kwargs):
for _ in range(num): # here, we are initializing a for loop and iterating till num
value = func(*args,**kwargs)
return value # here, we are returning the value
return wrapper # here, we are returning the wrapper class
return decorator_repeat
#Here we are passing num as an argument which repeats the print function
@repeat(num=5)
def function1(name):
print(f"{name}")
In the above example, @repeatrefers to a function object that can be called in another function. The @repeat(num = 5)will return a function which acts as a decorator.
The above code may look complex but it is the most commonly used decorator pattern where we have used one additional def that handles the arguments to the decorator.
Note: Decorator with argument is not frequently used in programming, but it provides flexibility. We can use it with or without argument.
Stateful Decorators
Stateful decorators are used to keep track of the decorator state. Let us consider the example where we are creating a decorator that counts how many times the function has been called.
Example:
Import functools # here, we are importing the functools into our program
def count_function(func):
# here, we are defining a function and passing the parameter func
@functools.wraps(func)
def wrapper_count_calls(*args, **kwargs):
wrapper_count_calls.num_calls += 1
print(f"Call{wrapper_count_calls.num_calls} of {func.__name__!r}")
return func(*args, **kwargs)
wrapper_count_calls.num_calls = 0
return wrapper_count_calls # here, we are returning the wrapper call counts
@count_function
def say_hello(): # here, we are defining a function and passing the parameter
print("Say Hello")
say_hello()
say_hello()
Output:
Call 1 of 'say_hello'
Say Hello
Call 2 of 'say_hello'
Say Hello
In the above program, the state represented the number of calls of the function stored in .num_callson the wrapper function. When we call say_hello()it will display the number of the call of the function.
Classes as Decorators
The classes are the best way to maintain state. In this section, we will learn how to use a class as a decorator. Here we will create a class that contains __init__() and take func as an argument. The class needs to be callable so that it can stand in for the decorated function.
To making a class callable, we implement the special __call__() method.
Code
import functools # here, we are importing the functools into our program
class Count_Calls: # here, we are creating a class for getting the call count
def __init__(self, func):
functools.update_wrapper(self, func)
self.func = func
self.num_calls = 0
def __call__(self, *args, **kwargs):
self.num_calls += 1
print(f"Call{self.num_calls} of {self.func.__name__!r}")
return self.func(*args, **kwargs)
@Count_Calls
def say_hello(): # here, we are defining a function and passing the parameter
print("Say Hello")
say_hello()
say_hello()
say_hello()
Output:
Call 1 of 'say_hello'
Say Hello
Call 2 of 'say_hello'
Say Hello
Call 3 of 'say_hello'
Say Hello
The __init__() method stores a reference to the function and can do any other required initialization.
