Threading Example

Scripting style

Start with working code that is clear, simple, and runs top to bottom. This is easy to develop and test incrementally.

counter = 0

print('Starting up')
for i in range(10):
    counter += 1
    print('The count is %d' % counter)
    print('---------------')
print('Finishing up')

That gives us the obvious output:

Starting up
The count is 1
---------------
The count is 2
---------------
The count is 3
---------------
The count is 4
---------------
The count is 5
---------------
The count is 6
---------------
The count is 7
---------------
The count is 8
---------------
The count is 9
---------------
The count is 10
---------------
Finishing up

Note

Get your app tested and debugged in a singled threaded mode first before you start threading. Threading NEVER makes debugging easier.

Function style

A next step in development is to factor re-usable code into functions.

counter = 0

def worker():
    'My job is to increment the counter and print the current count'
    global counter

    counter += 1
    print('The count is %d' % counter)
    print('---------------')

print('Starting up')
for i in range(10):
    worker()
print('Finishing up')

Multi-threading is easy!

It is just a matter of launching a few worker threads.

import threading

counter = 0

def worker():
    'My job is to increment the counter and print the current count'
    global counter

    counter += 1
    print('The count is %d' % counter)
    print('---------------')

print('Starting up')
for i in range(10):
    threading.Thread(target=worker).start()
print('Finishing up')

Testing proves the code is correct!

A simple test run compares perfectly to the original output:

$ python3.6 threading_multi1.py
Starting up
The count is 1
---------------
The count is 2
---------------
The count is 3
---------------
The count is 4
---------------
The count is 5
---------------
The count is 6
---------------
The count is 7
---------------
The count is 8
---------------
The count is 9
---------------
The count is 10
---------------
Finishing up

Can you spot the race conditions?

Most people spot the “counter increment” race condition, but most don’t immediately see the “print function” race condition.

Note

Testing cannot prove the absence of errors. It is still useful, don’t rely on it. Many interest racing conditions don’t reveal themselves in test environments.

Why didn’t testing reveal the flaws?

What can we do to improve the effectiveness of testing?

Fuzzing

Fuzzing is a technique for amplifying race conditions.

import threading, time, random

##########################################################################################
# Fuzzing is a technique for amplifying race condition errors to make them more visible

FUZZ = True

def fuzz():
    if FUZZ:
        time.sleep(random.random())

###########################################################################################

counter = 0

def worker():
    'My job is to increment the counter and print the current count'
    global counter

    fuzz()
    oldcnt = counter
    fuzz()
    counter = oldcnt + 1
    fuzz()
    print('The count is %d' % counter, end='')
    fuzz()
    print()
    fuzz()
    print('---------------', end='')
    fuzz()
    print()
    fuzz()

print('Starting up')
fuzz()
for i in range(10):
    threading.Thread(target=worker).start()
    fuzz()
print('Finishing up')
fuzz()

This technique is limited to relatively small blocks of code and is imperfect in that is can’t prove the absence of errors.

Still, fuzzed tests do reveal the presence of errors:

Starting up
The count is 1The count is 2The count is 2The count is 2


---------------The count is 3
---------------The count is 4
---------------
---------------The count is 4


The count is 5------------------------------
Finishing up


The count is 5
------------------------------



The count is 6---------------
---------------

David Baron at Mozilla’s San Francisco Office

_images/thistall.jpg

More Careful Threading with Queues

Interestingly, the rules for threading are just for computing and programming. The physical world is full of concurrency as well. Many of these techniques has physical analogs that are useful for managing people and projects.

Note

RR 1000

ALL shared resources SHALL be run in EXACTLY ONE thread. ALL communication with that thread SHALL be done using an atomic message queue: typically the Queue module, email, message queues like RabbitMQ or ZeroMQ, interesting you can communicate via a database as well.

Resources that need this technique: global variables, user input, output devices, files, sockets, etc.

Some resources that already have locks inside (thread-safe): logging module, decimal module (thread local variables), databases (reader locks and writer locks), email (this is an atomic message queue).

Note

RR 1001

One category of sequencing problems is to make sure that step A and step B happen sequentially. The solution is to put both in the same thread where all actions proceed sequentially.

Note

RR 1002

To implement a “barrier” that waits for parallel threads to complete, just join() all of the threads.

Note

RR 1003

You can’t wait on daemon threads to complete (they are infinite loops). Instead, you join() on the queue itself. It waits until all the requested tasks are marked as being done.

Note

RR 1004

Sometimes you need a global variable to communicate between functions. Global variables work great for this purpose in a single threaded program. In multi-threaded code, it mutable global state is a disaster. The better solution is to use a threading.local() that is global WITHIN a thread but not without.

Note

RR 1005

Never try to kill a thread from something external to that thread. You never know if that thread is holding a lock. Python doesn’t provide a direct mechanism for kill threads externally; however, you can do it using ctypes, but that is a recipe for a deadlock.

Applying all the rules

import threading, time, random, queue

##########################################################################################
# Fuzzing is a technique for amplifying race condition errors to make them more visible

FUZZ = True

def fuzz():
    if FUZZ:
        time.sleep(random.random())

###########################################################################################

counter = 0

counter_queue = queue.Queue()

def counter_manager():
    'I have EXCLUSIVE rights to update the counter variable'
    global counter

    while True:
        increment = counter_queue.get()
        fuzz()
        oldcnt = counter
        fuzz()
        counter = oldcnt + increment
        fuzz()
        print_queue.put([
            'The count is %d' % counter,
            '---------------'])
        fuzz()
        counter_queue.task_done()

t = threading.Thread(target=counter_manager)
t.daemon = True
t.start()
del t

###########################################################################################

print_queue = queue.Queue()

def print_manager():
    'I have EXCLUSIVE rights to call the "print" keyword'
    while True:
        job = print_queue.get()
        fuzz()
        for line in job:
            print(line, end='')
            fuzz()
            print()
            fuzz()
        print_queue.task_done()
        fuzz()

t = threading.Thread(target=print_manager)
t.daemon = True
t.start()
del t

###########################################################################################

def worker():
    'My job is to increment the counter and print the current count'
    counter_queue.put(1)
    fuzz()

print_queue.put(['Starting up'])
fuzz()

worker_threads = []
for i in range(10):
    t = threading.Thread(target=worker)
    worker_threads.append(t)
    t.start()
    fuzz()
for t in worker_threads:
    fuzz()
    t.join()

counter_queue.join()
fuzz()
print_queue.put(['Finishing up'])
fuzz()
print_queue.join()
fuzz()

Cleaned-up code without fuzzing

import threading, queue

###########################################################################################

counter = 0

counter_queue = queue.Queue()

def counter_manager():
    'I have EXCLUSIVE rights to update the counter variable'
    global counter

    while True:
        increment = counter_queue.get()
        counter += increment
        print_queue.put([
            'The count is %d' % counter,
            '---------------'])
        counter_queue.task_done()

t = threading.Thread(target=counter_manager)
t.daemon = True
t.start()
del t

###########################################################################################

print_queue = queue.Queue()

def print_manager():
    'I have EXCLUSIVE rights to call the "print" keyword'
    while True:
        job = print_queue.get()
        for line in job:
            print(line)
        print_queue.task_done()

t = threading.Thread(target=print_manager)
t.daemon = True
t.start()
del t

###########################################################################################

def worker():
    'My job is to increment the counter and print the current count'
    counter_queue.put(1)

print_queue.put(['Starting up'])
worker_threads = []
for i in range(10):
    t = threading.Thread(target=worker)
    worker_threads.append(t)
    t.start()
for t in worker_threads:
    t.join()

counter_queue.join()
print_queue.put(['Finishing up'])
print_queue.join()

Careful Threading with locks

import threading, time, random

##########################################################################################
# Fuzzing is a technique for amplifying race condition errors to make them more visible

FUZZ = True

def fuzz():
    if FUZZ:
        time.sleep(random.random())

###########################################################################################

counter_lock = threading.Lock()
printer_lock = threading.Lock()

counter = 0

def worker():
    'My job is to increment the counter and print the current count'
    global counter
    with counter_lock:
        oldcnt = counter
        fuzz()
        counter = oldcnt + 1
        fuzz()
        with printer_lock:
            print('The count is %d' % counter, end='')
            fuzz()
            print()
            fuzz()
            print('---------------', end='')
            fuzz()
            print()
        fuzz()

with printer_lock:
    print('Starting up', end='')
    fuzz()
    print()
fuzz()

worker_threads = []
for i in range(10):
    t = threading.Thread(target=worker)
    worker_threads.append(t)
    t.start()
    fuzz()
for t in worker_threads:
    t.join()
    fuzz()

with printer_lock:
    print('Finishing up', end='')
    fuzz()
    print()

fuzz()

Let’s see how well the runs:

Starting up
The count is 1
---------------
The count is 2
---------------
The count is 3
---------------
The count is 4
---------------
The count is 5
---------------
The count is 6
---------------
The count is 7
---------------
The count is 8
---------------
The count is 9
---------------
The count is 10
---------------
Finishing up

It is perfect!

Cleaned-up code without fuzzing

Now, let’s clean it up:

import threading, time, random

counter_lock = threading.Lock()
printer_lock = threading.Lock()

counter = 0

def worker():
    'My job is to increment the counter and print the current count'
    global counter
    with counter_lock:
        counter += 1
        with printer_lock:
            print('The count is %d' % counter)
            print('---------------')

with printer_lock:
    print('Starting up')

worker_threads = []
for i in range(10):
    t = threading.Thread(target=worker)
    worker_threads.append(t)
    t.start()
for t in worker_threads:
    t.join()

with printer_lock:
    print('Finishing up')

Results:

  • It is perfect!
  • It is beautiful.
  • It is simpler than using queues.

Notes on Locks

Note

RR 1005

Locks don’t lock anything. They are just flags and can be ignored. It is a cooperative tool, not an enforced tool.

Note

RR 1006

In general, locks should be considered a low level primitive that is difficult to reason about in non-trivial examples. For more complex applications, you’re almost always better off with using atomic message queues.

Note

RR 1007

The more locks you are acquire at one time, the more you lose the advantages of concurrency.

Dining Philosophers

The rules given above help you reliably create multi-threaded code is the underlying data flow is a DAG (directed acyclic graph).

When the control flow or data flow is circular, the problem can be much harder. At that point, more formal design and verification techniques are warranted. Otherwise, it can be quite difficult in complex applications to avoid deadlock, have thread starvation, or to have unfair solutions.