Making a shift to Python 3?

Python 2.7 has already reached to sunset on 1st Jan 2020 and most of the companies are still paddling hard to shift their Python 2.7 codebase to Python 3. VMware is also one of the companies that are shifting its gears towards Python 3. Let me tell you, this shift is not easy and contains regressive testing of existing workloads on the new version of Python.

This article is not about to discuss the various conceptual differences between Python 2.7 and Python 3 but to lightning the most common code changes that we may need to make in the existing codebase to make this porting easy and faster. This article can mostly be considered as a small list of items to note the syntactical differences across the two versions while performing the same operation.

Print Statements — Printing statements are very common in computer languages. It not only helps the user to interact with the application but also helps developers to debug the code. Below is the minute difference while printing the same statements across both the versions.

Python 2.7

print "hello world"

Python 3

print("hello world")

Integer Division — Below is the minute difference while dividing two values and expecting an integer across both the versions.

Python 2.7

res = 100 / 10
res == 10

Python 3

res = 100 // 10
res == 10

# To make the existing code works, use extra / while dividing the values

Exception Handling — Exceptions are the method to protect running applications from unwanted and sudden falls. Python is a runtime language which makes a little hard to predict the exact impact of user inputs per se to the application in advance. Below are the code syntaxes to handle the exception objects thrown by the application from both the versions.

Python 2.7

# ex.message would point the exception message passed as an argument to Exception object while raising it


try:
    raise Exception("Zero division error")
except Exception as ex:
    log.error(ex.message)

try:
    raise Exception("Zero division error")
except Exception as ex:
    if "Zero" in ex.message:
        log.error(ex.message)

Python 3

# ex would be a Exception object here and it would be casted to `str` implictly before performing any string level operations such as printing , logging etc and no `message` property would be avaiable on this object any more.

try:
    raise Exception("Zero division error")
except Exception as ex:
    log.error(ex)


# Where as if we would want to use ex as an iterable then we need to cast it explictly

try:
    raise Exception("Zero division error")
except Exception as ex:
    if "Zero" in str(ex):
        log.error(ex)

# To make the existing code works, stop using the message property of ex object and cast the ex explictly while using as iterable

Utility Functions — Python provides many utility functions to operate over container data types such as list, tuple, set, and zip. Below are the differences to use these functions and their return values.

Python 2.7 — The return values of the below util methods would be of type list which means Python 2.7 loads all the data in memory first and then returns an iterable to access the data.

l = range(10)
type(l) == list

l = filter(lambda x: x is True, [True, False, True])
type(l) == list

l = map(lambda x: x * 2, [1, 2, 3])
type(l) == list

l = [1, 2]
m = [3, 4]
z = zip(l, m)
type(zip) == list

Python 3 — The return values of the below util methods would be of type iterator which means Python 3 loads the data in memory on the need basis and returns only the loaded element when asked for.

Python 2.7 had xrange which used to return an iterator but Python 3 has replaced xrange with range.

l = range(10)
type(l) == range

l = filter(lambda x: x is True, [True, False, True])
type(l) == filter

l = map(lambda x: x * 2, [1, 2, 3])
type(l) == map


l = [1, 2]
m = [3, 4]
z = zip(l, m)
type(z) == zip
# To make the existing code works, wrap around all the return values with list() explictly.

Dictorinories — Dictionary is one of the data structures in Python which is equivalent to hash map in other languages. It stores data in terms of key-value pairs but to access these key-value pairs got a minute difference across both the versions.

Python 2.7

d = {1: 2, 2: 3}
type(d.keys()) == list

type(d.values()) == list

Python 3

d = {1: 2, 2: 3}
type(d.keys()) == dict_keys

type(d.values()) == dict_values

# To make the existing code works, wrap around all the return values with list() explictly.

Python 3 Bytes vs Str — Python 3 has introduced two types of streams,

bytes — machine-readable format

str — human-readable format

Generally in applications, we may get data from a Web URL or we may want to execute a command on a remote machine using paramiko module. Now in both cases, we get a bytes stream back from the other end as a response, and to use it further we need to decode it to a human-readable format.

To convert bytes into str, we have to use decode() method and encode() is to be used to convert str to bytes again.

b = b'hello world'
type(b) == bytes

h = b.decode()
h == 'hello world'
type(h) == str

b = h.encode()
b == b'hello world'
type(b) == bytes


# This change is a loud mouth and only for Python 3. It cant be converted from Python 2 to Python 3 silently like above others.

Import Modules from the same directory — Python 3 has changed the way to import modules from the same directory. First, we have to create an __init__.py file in a directory before importing any module from it. It is a way to tell Python that this directory is special and we would like to import modules from it in our code.

For example, in a user directory, we have two files, login.py, and home.py. we want to import login inside the home. Below are the ways in both the versions,

Python 2.7

from login import Login

Python 3

from .login import Login

That is it. These are my findings so far while porting the code from Python 2 to Python 3. Python community has released 2to3 utility to help in porting the code and it is helpful too. But this tool does not convert the code 100%. It still needs manual intervention to complete the porting.

Hope this article may help you to save time while porting the code and feel free to add your findings in comments as well so that others can leverage your findings too.