Summary of Function Concepts

Calling Functions

Python has many useful built-in functions that can be called directly. To call a function, you need to know its name and parameters. You can check the official Python documentation or use the help function, such as help(abs).

If you pass the wrong number of arguments when calling a function, a TypeError will be raised, and Python will clearly tell you how many arguments the function requires. If the number of arguments is correct but the argument type is not accepted by the function, a TypeError will also be raised.

Some functions can accept any number of arguments and return the largest one, such as the max function:

max(1, 3)  # 3
max(1, -3, 2)  # 2

Data Type Conversion

Python’s built-in functions also include data type conversion functions, such as int, str, etc.

A function name is actually a reference to a function object, so you can assign the function name to a variable, which is equivalent to giving the function an “alias”:

m = max  # Variable m points to the max function
m(1, 3, 5)  # You can call the max function through variable m

Defining Functions

In Python, functions are defined using the def statement, followed by the function name, parentheses, parameters, a colon, and then the indented function body. The return statement is used to return a value. Example: absolute value function my_abs:

def my_abs(x):
        if x >= 0:
                return x
        else:
                return -x

When the function body executes a return statement, it ends and returns the result. If there is no return statement, the function returns None by default, which is equivalent to return None or just return.

Empty Functions

If you want to define an empty function that does nothing, you can use the pass statement in the function body. For example:

if height < 180:
        pass

Parameter Checking

As mentioned above, if you call a function with the wrong number of arguments, Python will raise a TypeError. However, if the argument type is incorrect, Python will not automatically check it like built-in functions do. This may cause problems during execution. You can use Python’s built-in isinstance function to check parameter types:

def my_abs(x):
        if not isinstance(x, (int, float)):
                raise TypeError("bad operand type")  # If the wrong type is passed, the function raises an error
        if x >= 0:
                return x
        else:
                return -x

Returning Multiple Values

You can use return A, B to return two values, or more. In fact, Python always returns a single value, but when returning multiple values, they are automatically packed into a tuple.

Function Parameters

When defining a function, the parameter names and positions are determined, and the function interface is defined. Although Python function definitions are simple, they are very flexible.

Positional Parameters

Example: a function to calculate the power of a number:

def power(x):
        return x ** x

For the power function, the parameter x is a positional parameter. When calling power, you must pass exactly one argument. But the above function is not practical if you want to calculate cubes, fourth powers, etc. You would have to write many functions. You can modify the power function as follows:

def power(x, n):
        s = 1
        while n > 0:
                n -= 1
                s *= x
        return s

This modified power function can calculate any power, and now it has two positional parameters, x and n. When calling, you need to assign values to these two parameters in order.

Default Parameters

In daily development, calculating squares is more common than other powers. You can use default parameters to set the default value of n to 2:

def power(x, n = 2):
        s = 1
        while n > 0:
                n -= 1
                s *= x
        return s

So calling power(5) is equivalent to power(5, 2). For cases where n ≠ 2, you need to explicitly pass the value of n. Default parameters simplify function calls, but there are some pitfalls:

• Default parameters must come after required parameters;
• If there are multiple default parameters, you can provide some of them in order, or provide them out of order by specifying parameter names;

(e.g., add_student('Jensen', 'Male', city = 'Hefei') means city uses the passed value, other default parameters use their defaults)

• Default parameters must point to immutable objects;

(For example:

def add_end(L = []):
        L.append('END')
        return L

Calling add_end() twice returns ['END', 'END'] because the default parameter is a variable pointing to [], and its value changes with each call.)

Variable Parameters

Python also allows variable parameters, i.e., the number of arguments passed in is variable. Just add a * before the parameter:

def calc(seed, *nums):
        sum = 0
        for n in nums:
                sum += n * n
        return seed, sum
        
calc(1, 2, 3, 4)  # 1 corresponds to seed, the rest are variable parameters nums

Inside the calc function, nums receives a tuple. You can pass any number of arguments, including just the required parameter seed. If you already have a list or tuple, you can add a * before it to unpack its elements as variable parameters:

alist = [2, 5, 6, 7]
calc(1, *alist)  # Unpack the list as variable parameters

Keyword Parameters

Keyword parameters allow you to pass zero or more named arguments, which are automatically packed into a dict inside the function. Like variable parameters, you can assemble a dict and use ** to unpack it as keyword parameters:

def person(name, age, **kw):
        print("name: ", name, ", age: ", age, ", otr: ", kw)

info = {'city': 'HFE', 'major': 'Computer Science'}
        
person('Jensen', 22)  # Only required parameters
person('Jensen', 22, city = 'Hefei', uni = 'HFUT')  # Two named parameters
person('Jensen', 22, **info)  # Unpack pre-assembled info dict

Named Keyword Parameters

To restrict the names of keyword parameters, e.g., only accept city and job, add a separator * in the parameter list. Parameters after * are named keyword parameters:

def person(name, age, *, city, uni):  # Parameters after * are named keyword parameters
        print("name: ", name, ", age: ", age, ", city: ", city, ", uni: ", uni)

If the function already has a variable parameter, named keyword parameters do not need the * separator:

def person(name, age, *args, city, uni):  # Already has variable parameters
        print("name: ", name, ", age: ", age, ", city: ", city, ", uni: ", uni)
        print("\nargs: ", args)

Named keyword parameters must be passed with parameter names, otherwise Python will treat them as positional parameters and raise an error.

Named keyword parameters can also have default values and may be omitted when calling.

Parameter Combination

In Python, all the above parameter types can be combined, but the order must be: required parameters, default parameters, variable parameters, named keyword parameters, and keyword parameters. However, using too many combinations is not recommended as it reduces readability:

def final(a, b, c = 0, *args, d, **kw):
        print("a: ", a, ", b: ", b , ", d: ", d, ", args: ", args, ", d: ", d, ", kw: ", kw)
        
final(1, 2, 3, 4, d = 5, name = 'final function')

Recursive Functions

A recursive function is a function that calls itself. Example: factorial function:

def fact(n):
        if n == 1:  # Only when n==1 needs special handling
                return 1
        return n * fact(n - 1)

fact(1), fact(10)

Be careful to avoid stack overflow when using recursion. In computers, function calls are implemented through a stack data structure. Each function call adds a stack frame, and each return removes one. The stack size is limited, so too many recursive calls will cause a stack overflow. For example, running fact(10000) will raise RecursionError: maximum recursion depth exceeded in comparison.

Tail recursion optimization can solve stack overflow. Tail recursion means the function returns itself, and the return statement does not contain an expression. This way, no matter how many times the function calls itself, only one stack frame is used. The above fact function is not tail recursive because its return statement uses a multiplication expression:

def fact(n):
        return fact_iter(n, 1)

def fact_iter(num, product):
        if num == 1:
                return product
        return fact_iter(num - 1, num * product)

fact(10)  # Running fact(10000) still raises an error

Unfortunately, most programming languages, including Python, do not optimize tail recursion, so even the improved fact function will cause stack overflow.

Advanced Python Features

Slicing

To get the first three elements of a list or tuple, use Python’s slicing feature:

names = ['Jensen', 'Eric', 'Jerry', 'Anderson', 'Taylor']
names[0:3]  # Output: ['Jensen', 'Eric', 'Jerry']

names[0:3] means take elements from index 0 up to, but not including, index 3. If the start index is 0, it can be omitted, e.g., names[:3]. (Tip: names[:] copies the entire list.)

Similarly, Python supports negative slicing, e.g., names[-1] for the last element, names[-3:] for the last three elements.

Slicing is very useful. For example, create a list from 0 to 99:

L = list(range(100))

You can easily get sublists using slicing:

L[:10]  # First ten elements: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[-10:]  # Last ten elements: [90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
L[25:35]  # Elements 25-34: [25, 26, 27, 28, 29, 30, 31, 32, 33, 34]
L[:10:2]  # Every other element in the first ten: [0, 2, 4, 6, 8]
L[::10]  # Every tenth element: [0, 10, 20, 30, 40, 50, 60, 70, 80, 90]

Strings can also be treated as lists, so slicing works on them as well.

(The above also applies to tuples. Tuples and lists are similar, but tuples are immutable after initialization.)

Iteration

Given a list or tuple, you can iterate over them using a loop. In Python, use for ... in to iterate, which also works on other iterable objects (like dict):

d = {'a': 1, 'b': 2, 'c': 3}
for key in d:
        print(key)  # Outputs: a, b, c (default iterates over keys)

To iterate over values, use for value in d.values(). To iterate over both keys and values, use for k, v in d.items(). Strings can also be iterated over.

You can use collections.abc.Iterable to check if an object is iterable:

from collections.abc import Iterable
isinstance('abc', Iterable)  # True
isinstance(123, Iterable)  # False
isinstance([1, 2, 3], Iterable)  # True

The built-in enumerate function turns a list into index-element pairs, allowing you to iterate over both index and value:

L = [100, 'Jensen', 99, 'Eric', 98]
for idx, value in enumerate(L):
        print(idx, value)

You can also iterate over multiple variables at once:

for x, y in ((1, 2), (2, 3), (3, 4)):
        print(x, y)

List Comprehensions

List comprehensions are a simple yet powerful way to create lists in Python. Example: generate list([1, 2, 3, 4, 5, 6, 7, 8, 9]) with list(range(1, 10)).

To generate [1*1, 2*2, ..., 100*100], use [x * x for x in range(1, 101)].

You can add an if condition to filter, e.g., [x * x for x in range(1, 101) if x % 2 == 0].

You can use nested loops for permutations:

[m + n for m in 'ABC' for n in 'XYZ']

You can also list all files and directories in the current directory:

import os
[dir for dir in os.listdir('.')]

List comprehensions can use multiple variables:

d = {'a': 1, 'b':2 , 'c': 3}
[k + '=' + v for k, v in d.items()]  # Output: ['a=1', 'b=2', 'c=3']

if…else

In a list comprehension, an if ... else before the for must have an else clause, while an if after the for is a filter and cannot have else.

For example:

[x if x % 2 == 0 else 0 for x in range(1, 11)]

Generators

If you create a huge list but only need a few elements, most of the space is wasted. Python provides generators, which compute elements on the fly.

To create a generator, replace [] with () in a list comprehension: g = (x * x for x in range(1, 10)). Use next to get elements one by one.

Generators are iterable, so you can use a for loop:

g = (x * x for x in range(1, 10))
for i in g:
        print(i)

If the algorithm is complex, use a function with yield to create a generator, e.g., Fibonacci sequence:

def fib(max):
        n, a, b = 0, 0, 1
        while n < max:
                yield b
                a, b = b, a + b
                n += 1
        return 'done'

When calling a generator, each next resumes from the last yield. Multiple generator calls create independent objects.

To get the return value from a generator, catch the StopIteration exception:

g = fib(10)
while True:
        try:
                x = next(g)
                print('g: ', x)
        except StopIteration as e:
                print('Generator return value: ', e.value)
                break

Iterators

Objects that can be used in a for loop include:

• Collection types: list, tuple, dict, set, str, etc.
• Generators

These are called iterable objects (Iterable). Use isinstance([], Iterable) to check.

Objects that can be called by next are iterators (Iterator). Use isinstance([], Iterator) to check.

Generators are both iterable and iterators. list, dict, str, etc., are iterable but not iterators; use iter to convert them.

Iterator computes values lazily, only when needed. Python’s for loop is essentially implemented by repeatedly calling next.

Functional Programming

Higher-Order Functions

Variables Can Point to Functions

You can assign a function to a variable, e.g., f = abs. Calling f() is the same as calling abs().

Function Names Are Variables

A function name is just a variable pointing to a function. For example, abs = 100 would make abs point to 100, so you can’t call the absolute value function anymore. Don’t do this in real code.

Passing Functions as Arguments

Since variables can point to functions, and function parameters can accept variables, a function can accept another function as a parameter. This is called a higher-order function:

def add(x, y ,f):
        return f(x) + f(y)

When calling add(-6, 5, abs), x, y, and f receive -6, 5, and abs respectively.

map & reduce

The map function takes a function and an iterable, applies the function to each element, and returns a new iterator. Example: apply a square function to a list:

def f(x):
        return x * x

m = map(f, [1, 2, 3, 4 ,5, 6, 7, 8])
list(m)

map can be used for any function, e.g., convert all elements to strings: list(map(str, [1, 2, 3, 4, 5, 6, 7, 'A', 'B'])).

The reduce function also takes a function and an iterable. The function must take two arguments, and reduce accumulates the result: reduce(f, [x, y, z, k]) = f(f(f(x, y), z), k).

Example: sum a list:

from functools import reduce
def add(x, y):
        return x + y

reduce(add, [2, 5, 6, 7, 5])

Example: convert a string to an integer:

from functools import reduce

DIGITS = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9}

def str2int(s):
        def fn(x, y):
                return x * 10 + y
        def char2num(s):
                return DIGITS[s]
        return reduce(fn, map(char2num, s))

str2int('2021111052')

This can be further simplified with lambda:

from functools import reduce

DIGITS = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9}

def str2int(s):
        return reduce(lambda x, y: x * 10 + y, map(lambda x: DIGITS[x], s))

str2int('13579')

filter

The built-in filter function filters elements in an iterable. It takes a function and an iterable, applies the function to each element, and keeps those where the function returns True. Example: remove odd numbers, keep even numbers:

def odd(n):
        return n % 2 == 0
list(filter(odd, [2, 5, 6, 7, 5, 0, 1, 3, 9]))

Example: remove empty strings:

def not_empty(s):
        return s and s.strip()

list(filter(not_empty, ['A', ' ', 'C', None]))

sorted

The built-in sorted function sorts an iterable and returns a list.

sorted can take a key function for custom sorting, e.g., sort by absolute value: sorted([2, 5, 6 ,7, 9, 0, -3, -11], key = abs).

Example: sort strings:

sorted(['Jensen', 'Bob', 'eric', 'yiming'])

To sort alphabetically, use key = str.lower:

sorted(['Jensen', 'Bob', 'eric', 'yiming'], key = str.lower)

To sort in reverse, use reverse = True.

d = {'A': 'America', 'C': 'China', 'R': 'Russia'}
sorted(d.values())

Returning Functions

Functions as Return Values

Higher-order functions can also return functions. For example, a sum function:

def calc_sum(*nums):
        ax = 0
        for n in nums:
                ax += n
        return ax

If you want to delay the calculation, return a function instead:

def lazy_sum(*nums):
        def sum():
                ax = 0
                for n in nums:
                        ax += n
                return ax
        return sum

f = lazy_sum(1, 3, 4, 5, 7 ,9)
f()

Each call to lazy_sum returns a new, independent function. The inner function can reference parameters and variables from the outer function. This is called a “closure”.

Closures

When a function returns another function that references variables from the outer function, it’s called a closure. Note that the returned function is not executed immediately. Be careful not to reference loop variables or variables that may change:

def count():
        fs = []
        for i in range(1, 4):
                def f():
                        return i * i
                fs.append(f)
        return fs

f1, f2, f3 = count()
f1(), f2(), f3()

All three return 9 because the variable i is 3 when the functions are called. To fix this, bind the current value:

def count():
        def f(j):
                return lambda: j * j
        fs = []
        for i in range(1, 4):
                fs.append(f(i))
        return fs

f1, f2, f3 = count()
f1(), f2(), f3()

nonlocal

If the inner function only reads the outer variable, it’s fine:

def inc():
        x = 0
        def fn():
                return x + 1
        return fn

f = inc()
print(f())
print(f())

But if you assign to the outer variable, Python treats it as a local variable unless you declare nonlocal:

def inc():
        x = 0
        def fn():
                nonlocal x
                x += 1
                return x
        return fn

f = inc()
print(f())
print(f())

Anonymous Functions (Lambda)

Anonymous functions, or lambda expressions, are convenient for passing functions without defining them explicitly. For example, lambda x: x * x is equivalent to:

def f(x):
        return x * x

lambda can only have one expression and does not need return.

Anonymous functions are function objects and can be assigned to variables or returned.

Decorators

Decorators enhance functions, e.g., automatically print logs before and after calling a function, without modifying the function itself. Example:

def log(func):
        def wrapper(*args, **kw):
                print('call %s():' % func.__name__)
                return func(*args, **kw)
        return wrapper

@log
def date():
        print('2021-12-11')
        
date()

If the decorator itself needs parameters, use a closure:

def log(text):
        def decorator(func):
                def wrapper(*args, **kw):
                        print('%s %s(): ' % (text, func.__name__))
                        return func(*args, **kw)
                return wrapper
        return decorator

@log('execute')
def date():
        print('2021-12-11')
        
date()

To preserve the original function’s attributes, use functools.wraps:

import functools

def log(func):
        @functools.wraps(func)
        def wrapper(*args, **kw):
                print('call %s(): ' % func.__name__)
                return func(*args, **kw)
        return wrapper

@log
def date():
        print('2021-12-11')
        
date.__name__

Partial Functions

A partial function fixes some parameters of a function and returns a new function, making calls simpler. Use functools.partial.

For example, to frequently convert hexadecimal strings:

import functools

int16 = functools.partial(int, base=16)
int16('25675')

Partial functions can also be used with variable parameters:

import functools
 
max2 = functools.partial(max, 10, -11)
max2(19)

Object-Oriented Programming

Classes and Instances

The most important concepts in OOP are classes and instances. For example, define a Student class:

class Student(object):
        pass

After defining a class, you can create instances:

class Student(object):
        pass

stu = Student()
stu, Student

You can freely bind attributes to an instance, e.g., stu.name = 'Jensen'. Usually, use the __init__ constructor to bind required attributes:

class Student(object):
        
        def __init__(self, name, score):
                self.name = name
                self.score = score
                
stu = Student('Jensen', 100)
stu.name, stu.score

Data Encapsulation

Encapsulation is important in OOP. You can define methods inside the class to access data:

class Student(object):
        
        def __init__(self, name, score):
                self.name = name
                self.score = score
                
        def print_score(self):
                print('%s: %s' % (self.name, self.score))
                
stu = Student('Jensen', 20)
stu.print_score()

Access Restriction

Although data is encapsulated, external code can still modify instance variables. To make attributes private, prefix them with double underscores __:

class Student(object):
        
        def __init__(self, name, score):
                self.__name = name
                self.__score = score
                
        def print_score(self):
                print('%s: %s' % (self.__name, self.__score))
                
stu = Student('Jensen', 20)
stu.print_score()

To allow controlled modification, add setter methods.

Note: variables like __XXX__ are special, not private. Variables like _name are meant to be private by convention.

Private variables like __name are actually changed to _Student__name internally.

Inheritance and Polymorphism

A class can inherit from another class. The new class is called a subclass, and the inherited class is the base or parent class.

class Animal(object):
        def run(self):
                print('Animal is running...')
                
class Dog(Animal):
        pass

class Cat(Animal):
        pass

Subclasses can override parent methods:

class Animal(object):
        def run(self):
                print('Animal is running...')
                
class Dog(Animal):
        def run(self):
                print('Dog is running...')
                
class Cat(Animal):
        def run(self):
                print('Cat is running...')
                

dog, cat = Dog(), Cat()
dog.run()
cat.run()

Polymorphism

A subclass instance is both its own type and its parent’s type:

a, b, c = list(), Animal(), Dog()

print(isinstance(a, list))
print(isinstance(b, Animal))
print(isinstance(c, Dog))
print(isinstance(c, Animal))
print(isinstance(b, Dog))

You can pass any subclass instance to a function expecting the parent class:

class Animal(object):
        def run(self):
                print('Animal is running...')
                
class Dog(Animal):
        def run(self):
                print('Dog is running...')
                
class Cat(Animal):
        def run(self):
                print('Cat is running...')
                
def run_twice(animal):
        animal.run()
        animal.run()
        
run_twice(Animal())
run_twice(Dog())
run_twice(Cat())

Static vs Dynamic Languages

In static languages, you must pass the correct type. In Python, as long as the object has the required method, it works.

Getting Object Information

Use type to check object types:

print(type(123))  
print(type('Jensen'))
print(type(None))
a = abs
print(type(abs))
print(type(a))

You can compare types:

type(123) == type(456), type('jensen') == str, type(123) == str

To check if an object is a function, use the types module.

Using isinstance()

For inheritance, use isinstance:

isinstance([1, 2, 3], (list, tuple))
isinstance('jensen', (str, int))

Using dir()

To get all attributes and methods of an object:

dir('Jensen')

Special methods like __len__ can be used to customize behavior:

class TestLen(object):
        def __len__(self):
                return 120

test = TestLen()
len(test)

You can use getattr, setattr, hasattr to inspect and modify object state.

Instance Attributes vs Class Attributes

Instances can have attributes bound via instance variables or self. Class attributes are defined in the class and shared by all instances.

class Student(object):
        name = 'Students'

If an instance has an attribute with the same name as a class attribute, the instance attribute overrides the class attribute.

Advanced Object-Oriented Programming

You can dynamically bind attributes and methods to instances and classes.

Using slots

To restrict instance attributes, define __slots__:

class Student(object):
        __slots__ = ('name', 'age')
        
s = Student()
s.name = 'Eric'
s.age = '22'

__slots__ only affects the current class, not subclasses unless they also define __slots__.

Using @property

To add validation to attributes, use the @property decorator:

class Student(object):
        @property
        def score(self):
                return self._score
        
        @score.setter
        def score(self, value):
                if not isinstance(value, int):
                        raise ValueError('Score must be an integer!')
                if value < 0 or value > 100:
                        raise ValueError('Score must between 0 ~ 100!')
                self._score = value

s = Student()
s.score = 90
s.score

You can define read-only properties by only defining a getter.

Multiple Inheritance

Python supports multiple inheritance (MixIn):

class Ostrich(Bird, Runnable):
        pass

Custom Classes

You can define special methods like __str__, __iter__, __getitem__, __getattr__, and __call__ to customize class behavior.

Using Enum

Use the built-in Enum class to define enumerations:

from enum import Enum

Month = Enum('Month', ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'))

Or define your own:

from enum import Enum, unique

@unique
class Weekday(Enum):
        Sun = 0
        Mon = 1
        Tue = 2
        Wed = 3
        Thu = 4
        Fri = 5
        Sat = 6

Using Metaclasses

type()

You can dynamically create classes using type:

def func(self, name = 'world'):
        print('Hello, %s.' % name)
        
Hello = type('Hello', (object,), dict(hello = func))
h = Hello()
h.hello()

metaclass

Metaclasses allow you to control class creation. For example, add an add method to a custom list:

class ListMetaclass(type):
        def __new__(cls, name, bases, attrs):
                attrs['add'] = lambda self, value: self.append(value)
                return type.__new__(cls, name, bases, attrs)
        
class MyList(list, metaclass = ListMetaclass):
        pass

L = MyList()
L.add(1)
L

ORM frameworks use metaclasses to map classes to database tables.

Error Handling, Debugging, and Testing

Error Handling

Python uses try...except...finally... for error handling.

try

Example: division by zero:

try:
        print('try...')
        r = 10 / 0
        print('result: ', r)
except ZeroDivisionError as e:
        print('except: ', e)
finally:
        print('finally...')
print('END')

You can have multiple except blocks for different errors.

Errors are classes, and except catches the specified type and its subclasses.

You can catch errors at a higher level to simplify code:

def foo(s):
        return 10 / int(s)

def bar(s):
        return foo(s) * 2

def main():
        try:
                bar('0')
        except Exception as e:
                print('Error: ', e)
        finally:
                print('finally...')

main()

Call Stack

If an error is not caught, Python prints a traceback and exits.

Logging Errors

Use the logging module to record errors:

import logging

def foo(s):
        return 10 / int(s)

def bar(s):
        return foo(s) * 2

def main():
        try:
                bar('0')
        except Exception as e:
                logging.exception(e)
                
main()
print('END')  

Raising Errors

You can raise errors with raise:

class FooError(ValueError):
        pass

def foo(s):
        n = int(s)
        if n == 0:
                raise FooError('invalid value: %s' % s)
        return 10 / n

foo('0')

You can also re-raise errors or convert them to another type.