Thread Programming

This page compares how to program Java and .NET computer programs to use threads for concurrency within the same process. This is needed for higher performance and better scalability.

The "Mondrian" applet shown in this page uses the old Java 1.2 threading model.

Topics this page:

Site Map  
About this site  

Why Threads for Concurrency

At the operating system level, several processes (jobs) are run. Each process can run several threads (tasks) at the same time.

Running long calculations in a separate worker thread allows programmers to provide a “Stop Calculation” button to users.

Concurrent threads also enable programs to listen for user interruptions.

Threads enable parallelism within a program process, such that each thread may take a different path of code execution through a program.

Multi-threading can be more efficient because it utilizes tiny slices of idle time present during program execution.
• Single processor machines simulate simultaneous execution by running each thread for a very short time, then performing a context switch to another thread.

• Threads make use of multiple processor machines (with two or more CPUs on a board) such that each thread has it own resource overhead of local variables and program counters.

Thread Hazards

Next Generation Java Testing (Paperback) Oct 12, 2007 Addison-Wesley by Cedric Suleiman, Hani Goetz, Brian (FRW) Beust                       The Coverity Static Code Analyzer scans through Java, C/C++/C# codebases to identify coding that could lead to system failures, memory corruption, unpredictable behavior, and performance degradation.

When design programming for multiple threads to avoid these potential problems:

1. System Thread Starvation occurs in GUI-based applications and applets, when user communication and screen updates are handled by a system thread which is scheduled along with other user-defined threads. If a particular application thread requires a lot of processing, the system thread may not get to the running state often and long enough, so user interactions suffer slowdowns. This is allievated somewhat by setting optimal Thread Priorities

2. Lock inversion happens when one thread attempts to acquire an exclusive lock, then holds on to it while attempting to acquire another lock.
3. Deadlock occurs when threads are waiting for each other forever. This happens when several threads are suspended until it can gain control of the same object, and each one has a lock on another resource that they need in order to proceed with processing. Two or more threads wait for locks in a circular chain such that the locks can never be acquired.

   This situation is also called "Deadly Embrace". When this defect occurs, the entire software system may halt, as none of the threads can neither proceed along their current execution paths nor exit.

   Anita Mary Joseph notes in Thread Communication Using Wait(), Pulse() and PulseAll() that "often the cause of the deadlock is not readily understood just by looking at the source code of the program, because concurrently executing threads can interact in complex ways at runtime. To avoid deadlock, careful programming and thorough testing are required. In general, if a multithreaded program occasionally "hangs", deadlock is the likely cause."

   Java doesn't provide any mechanisms for detecting or controlling deadlock situations, so the programmer is responsible for avoiding them with Java Thread Monitors

4. Thread Blocking occurs when a thread calls a long-running operation while holding a lock which prevents other threads from progressing. This single bottleneck for all threads causes application performance to drop dramatically.

5. Critical Section Race Condition The part of a program that accesses shared resources (and thus be in a position to disrupt other processes) is called a Critical Section. A race condition between processes can be avoided by making sure that no two processes enter their Critical Sections at the same time. Multiple threads access the same shared data without the appropriate locks to protect access points. When this defect occurs, one thread may inadvertently overwrite data used by another thread, leading to loss of information and data corruption. To prevent this, threads must communicate with each other so only one has mutex (mutually exclusive) lock.

Java Threading

Python programs on multi-CPU machines have to divide the work up between multiple processes rather than multiple threads because (almost) all Python code can only run while it holds the Global Interpreter Lock (GIL). Thus, a multi-threaded Python program effectively only uses one CPU.

However, the Jython interpreter can make use of Java's threading mechanisms.

MS .NET Threading

Microsoft's .NET provides two types of multi-tasking in its namespace (library) System.Threading.
• Process-based multitasking handles the concurrent execution of separate programs, such as running a word processor program the "same time" as an Internet browser application.

• Thread-based multitasking deals with the concurrent execution of pieces of the same program, such as a text editor formatting text at the same time that it is printing.

To facilitate ease of debugging during testing, database servers are usually run as a thread within the same process as client application threads. But in production mode, to ensure reliable up-time, database servers are normally run in a separate process from the client application.

Grounding

The .Net Framework has two types of threads:
• foreground threads created by default.
• background threads which automatically terminate when all foreground threads in its process have stopped.

  To set a thread to background in .NET, change this property to True:

  public bool IsBackground { get; set; }
  

Thread States and Classes

Pop-up this diagram to its own window

A 6-state thread model was introduced by Java 1.3:

1. Ready-to-run: waiting for its turn to be picked for execution by the thread scheduler based on thread priorities.

2. Running (IsAlive) : The thread's code is being actively executed by a processor and will run until it is swapped out, becomes blocked, or voluntarily give up its turn with a yield method which generally, because of context switching overhead, should not be used more than about five times per second

3. Waiting: A Thread is in a blocked state while it waits for some external processing (such as file I/O) to finish.

4. Sleeping: Threads are forcibly put to sleep (suspended) by a thread.sleep() java call or a Solaris poke-cpu() traps.

5. Blocked on I/O: Will move to Ready-to-Run after I/O condition changes (reading a byte of data).

   Blocked on Sync: Will move to Ready-to-Run when a lock is acquired (passes synchronized statement).

6. Dead (Terminated) : The thread has finished working and cannot be resumed.

These books include J2SE 5.0 features: Java Threads by Scott Oaks and Henry Wong. O'Reilly, Sep 10, 2004.   Java Concurrency in Practice May 2006, Addison-Wesley by Brian Goertz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea. is the definitive book on Java multi-threaded programming.

Previously in Java 1.2, a Thread can be in one of four states:

1. New - When a Thread is created it is into the New state. A New State Thread is not running.
2. Running - A call to the Thread method start() is required to move the Thread from the New State to the Runnable State. A Thread in the Runnable State is eligible to execute on the CPU. A Runnable State Thread will move on and off the CPU due to scheduling assignments.
3. Dead - A Thread moves from Runnable State to Dead State when the run() method exits or the Thread stop() method is called. Note that a Thread that has stopped is dead, not suspended.
4. Blocked - The last Thread state is Blocked State. The Blocked State contains the Threads that can not execute due to a wait(), suspend(), sleep(), or I/O wait. When the reason for the block is removed, the Thread moves back to the Runnable State.

   Thread methods suspend( ), resume( ), and stop( ) were deprecated by Java v1.3.

Creating Threads

All processes need at least one thread of execution — the main thread associated with the program's main entry point.

Other threads can be created from within the main thread using instantiation methods.

MS .NET Thread Creation

With C# .NET, another thread is created from within the main thread by instantiating a delegate object:

// to name a delegate:
ThreadStart ts = new ThreadStart(  MyThreadMethod );
Thread t = new Thread(  ts  ); // to create thread object.
t.Start(); // to start the new thread.

"MyThreadMethod" is the entryPoint name of the method called to begin thread execution, defined by code such as:

public static void MyThreadMethod() {
}

Such entry point methods need to be defined as void because no arguments are passed to it.

Also, calling Start() on a thread that has already been started will result in a ThreadStateException being thrown.

Once started, the thread runs until the method specified by entryPoint returns and automatically stops execution.

Example from Introducing Multithreaded Programming in C#

Java Thread Creation

To implement multithreading in Java code, a class must do one of the following:
• extend the java.lang.Thread class, or
• implement java.lang.Runnable in the Thread class itself so that code is executed from within the overridden run() method. This tends to work much better in the real world.

C Thread Creation

To create a new thread C code:
pid_t fork(void);

Java.lang.Thread Class Methods

Built into native Java v1.3+

Method	Return Type	Description
currentThread	Thread	Returns an object reference to the thread in which it is invoked.
getName	String	Retrieve the name of the thread object or instance.
getPriority	int	Retrieve the thread instance's priority.
isAlive	boolean	Determine if the thread is currently running.
run	void	This method is the entry point for threads, like the main method for applications.
start	void	Start the thread by calling its run method.
sleep	void	Suspend a thread for a specified amount of time (in milliseconds).
wait	void	moves an object into a waiting state where it waits for a lock to be released.
notify	void	Remove a thread from the waiting state and place it in the ready-to-run state.

The wait(), notify(), and notifyAll() methods for synchronized thread control in Java 1.2+ are part of the Object class, which is the root of all Java object classes. Even if a class does not extend another class, it is automatically an extension of the Object class.

yield() and sleep() are static methods within java.lang.Thread.

start() and run() are instance methods.

Yielding a Thread

To switch the currently running thread to a runnable state so that a thread of the same priority has a chance to run, use the Java static method

 Thread.yield();

Join the Line and Waiting

Java provides the join() method on the thread object the program waits on. An example of its use (with timeout, from Strakt) in Jython:

import threading
import time
class Sleeper(threading.Thread):
	def __init__(self, name, seconds):
		self.name = name
		self.seconds = seconds
		threading.Thread.__init__(self)
	def run(self):
		# while not done loop goes here in a real pgm.
		time.sleep(self.seconds) # for demo purposes.
		print '%s done' % self.name

c1 = Sleeper('Long', 6)
c1.setDaemon(1)
c1.start()
c2 = Sleeper('Short', 2)
c2.start()
c1.join(4) # with 4 second time-out
print 'Done waiting for c1'

The setDaemon method defines a thread object as a daemon.

The best practice recommended is using the Runnable interface of the Java Thread object which offers thread management APIs for checking status, setting daemon thread status, and killing a thread.

Putting it to Sleep

Java overloaded static method:

Thread.sleep(milliseconds);
Thread.sleep(milliseconds, nanoseconds);

C# .NET uses delegates to control threads. A delegate is like naming a signature of a function. Here is an example of using WaitCallBack to put the main section to sleep:

AutoResetEvent asyncOpIsDone = new AutoResetEvent(false);
ThreadPool.QueueUserWorkItem( new WaitCallback(MyAsyncOperation), asyncOpIsDone);
// ...
asyncOpIsDone.WaitOne(); // to do the waiting.

The waiting is done by an async operation created by a method defined with the "Object state" signature required by WaitCallBack:

static void MyAsyncOperation(Object state){
Thread.Sleep(5000); // 5000 milliseconds = 5 seconds.
((AutoResetEvent)state).Set(); // to signal async operation completion.
}

C# .NET offers a convenient way to wake up every 2 seconds:

Timer timer = new Timer( new TimerCallback(CheckStatus), null, 0, 2000);

The CheckStatus method must also be defined with the "Object state" signature required by Timer:

static void CheckStatus(Object state){
// ...
}

VM to OS Thread Mapping

The way Java threads are mapped to OS threads is up to the JVM and its use of multiple processors on a machine. Different JVMs use different strategies:
Some JVMs use green threads, running all Java threads in one native OS thread (called n-to-one mapping). Sun's HotSpot JVM uses native threads, which may execute in parallel on a multi-CPU machine. On Solaris, Java threads are not bound permanently to the same native threads but are remapped by the scheduler (in an n-to-m mapping).

With the Blackdown port of the JDK 1.2 to Linux, a thread is akin to a process (one-to-one mapping).

With Java, each Thread object is instantiated with two execution call stacks: One to keep track of Java method calls and local variables. Another stack is used to keep track of native (typically C code) calls.

With C# .NET, static variables used within a thread (and not for other threads) are defined thus:

[ThreadStatic] static Int32 tsValue = 10;

Thread Scheduler Priorities

Thread settings can give hints to the thread scheduler, but do not guarantee a specific behavior.

Higher-level interrupts will be serviced faster than lower-priority interrupts. It is a common misconception that one can:

• use higher priorities for threads that frequently block (sleep, wait for I/O).
• use medium to low priorities for CPU-intensive calculation threads to avoid hogging the processor.

Priority settings are ineffective because they impact work over a longer period of time than the microseconds granularity where race conditions occur.

Setting priorities may not always have the desired effect because prioritization schemes may be implemented differently on different platforms.

Tweaking priorities only seem to fix race conditions because that only makes the problem occur less frequently, but they also make the underlying problems more difficult to detect, recreate, and debug. Such fixes result in applications passing through QA, then deadlocking in the field.

.NET Fiver

.NET program Priority is an enumeration of just 5 values:
• ThreadPriority.Highest
• ThreadPriority.AboveNormal
• ThreadPriority.Normal (the default)
• ThreadPriority.BelowNormal
• ThreadPriority.Lowest

Solaris

With the 15 Solaris priority levels, Process Interrupt Levels (PILs) or serial interrupts hold the five highest priority levels which block all lower-priority interrupts until they complete.

Below a priority level of 10, interrupts are handled by threads.

Java's 1 - 5 - 10

But if you can't resist messing with it, increment:

Thread t = Thread.currentThread();
int intCurrentPriority;
intCurrentPriority = t.getPriority();
t.setPriority( intCurrentPriority+1 );

Default priorities for threads started by Java:

Literal Thread.	Priority	Thread Name
MAX_ PRIORITY	10	Reference Handler
	8	Finalizer
	6	AWT-EventQueue-0 a.k.a. event thread, invokes event-handling methods actionPerformed(), keyPressed(), mouseClicked(), and windowClosing().
NORM_ PRIORITY	5	main
		Signal dispatcher
		AWT-Windows
		SunToolkit.PostEventQueue-0
	4	Screen Updater
MIN_ PRIORITY	1	-

Java Coding — Runnable Sample

This sample starts two synchronized threads, which outputs 1 to 10 twice:

public class MyThread implements Runnable {
    private String holdA = "This is ";
    private int[] holdB = {1,2,3,4,5,6,7,8,9,10};
    public static void main(String args[]) {
        MyThread z = new MyThread();
        ( new Thread(z) ).start();
        ( new Thread(z) ).start();

    }
    // Any object implementing a synchronized keyword is a monitor:
    public synchronized void run() {
        for(int w = 0;w < 10;w++) {
            System.out.println(holdA + holdB[w] + ".");
        }
    }
}

Synchronized Monitor Threads

Thread Monitors

To prevent problems that could occur by having two methods modifying the same object, Java uses monitors and the synchronized keyword to control access to an object by a thread. Any object that implements the "synchronized" keyword is considered to be a monitor. A monitor is an object that can move a thread process between a blocked and running state. Monitors are required when a process needs to wait for an external event to occur before thread processing can continue. A typical example is when a thread can't continue until an object reaches a certain state.

Every object has a lock associated with it that controls access to code marked with the synchronized keyword. The wait method is used to move an object into a waiting state where it waits for a lock to be released. The notify and notifyAll methods are used to move a thread out of the waiting state and into the ready-to-run state.

A thread acquires an exclusive lock on a shared resource by entering a waiting monitor object. When the thread is done, it calls a wait method. The mutually exclusive (mutex) locking mechanism waits are synchronized, which serves as a “token” or lock.

It is recommended that threads monitor their execution and stop by returning from their run() method. A thread should monitor the state of a shared variable (interface events) to determine if its execution should be stopped, suspended, or resumed. The wait() and notify() methods of the Object class should be used to cause a thread to wait on changes to the value of a shared variable.

• The “Synchronized” keyword:
  • allows only one thread to access data at a time.
  • prevents interruption by other threads.
  • prevents other threads from running.

Java Volatile Modifier

Use the keyword volatile modifier on member variables if
• for speed, a working copy of shared member variables should be written to the volatile variable before the thread enters or leaves a synchronized block of code.

• To force individual threads to reread a variable's value from shared memory every time the variable is accessed, and

• to force write back to shared memory as soon as they occur,

Stopping Threads

Typically, a program executes while all its threads are active, within a C# .NET loop such as:

do {
	// code ...
} while (myThread1.thread.IsAlive &&
	myThread2.thread.IsAlive &&
	myThread3.thread.IsAlive );

The Microsoft .NET Framework also provides the Join() method to make threads wait until another thread terminates.

A maximum amount of wait time can be specified.

A ThreadStateException will be thrown if the thread has not been started.

Thread-Safe Synchronized Sequential Execution

$25 Java Thread Programming Sams, 1999. 510 pages by Paul Hyde. InterProcess Communication Tutorial from CMU

A thread does not respond to interrupts when it is blocked waiting to acquire exclusive access to a synchronized lock. This can become a problem when the work done inside a synchronized section takes a relatively long time to complete.

Paul Hyde, author of Java Thread Programming recommends that multiple threads and safely interact with each other by going though a “StopLight” class for threads to query before taking action. It hides Java wait/notify mechanisms and enforces timeouts. The helper class uses three objects for locking and inter-thread communication:

• Object valueLock is used to control simultaneous access to value.
• falseToTrueLock facilitates the use of the wait/notify mechanism to notify threads waiting for value to transition from false to true.
• trueToFalseLock notifies threads waiting for value to change from true to false.

Inter-Thread Communications

When a thread is temporarily blocked from running, it enters into Wait() using Monitor class methods such as:

public static bool Wait(object waitObject)

or

public static bool Wait(object waitObject, int milliseconds)

which waits until the specified number of milliseconds.

Waiting causes the lock for that object to be released, allowing another thread to use the object.

This is thread sychronization. Visual Basic.NET uses the "SyncLock" keyword.

The waiting thread is awakened when some other thread enters the same lock and calls Monitor class methods

• Pulse() to resume the first thread in the queue of threads waiting for the lock, or
• PulseAll() to signal the release of the lock to all waiting threads.

A SynchronizationLockException is thrown if Wait(), Pulse or PulseAll() is not called from within a lock block such as in this example from Anita Mary Joseph:

// Using Wait() and Pulse() to create a bouncing ball.
// thread ping and thread pong.
using System;
using System.Threading;

class PingPong
{
	public void ping( bool running )
	{
		lock(this)
		{
			if(!running)
			{
				//ball halts.
				Monitor.Pulse(this); // notify any waiting threads
				return;
			}
			Console.Write("Ping ");
			Monitor.Pulse(this); // let pong() run
			Monitor.Wait(this); // wait for pong() to complete
		}
	}
	public void pong( bool running )
	{
		lock(this)
		{
			if(!running)
			{
				//ball halts.
				Monitor.Pulse(this); // notify any waiting threads
				return;
			}
			Console.WriteLine("Pong ");
			Monitor.Pulse(this); // let ping() run
			Monitor.Wait(this); // wait for ping() to complete
		}
	}
}

class MyThread
{
	public Thread thread;
	PingPong pingpongObject;

	//construct a new thread.

	public MyThread(string name, PingPong pp)
	{
		thread = new Thread(new ThreadStart(this.run));
		pingpongObject = pp;
		thread.Name = name;
		thread.Start();
	}

	//Begin execution of new thread.
	void run()
	{
		if (thread.Name == "Ping")
		{
			for (int i = 0; i < 5; i++)
				pingpongObject.ping(true);
			pingpongObject.ping(false);

		}
		else
		{
			for (int i = 0; i < 5; i++)
				pingpongObject.pong(true);
			pingpongObject.pong(false);
		}
	}
}

class BouncingBall
{
	public static void Main()
	{
		PingPong pp = new PingPong();
		
		Console.WriteLine("The Ball is dropped... \n");

		MyThread mythread1 = new MyThread("Ping", pp);
		MyThread mythread2 = new MyThread("Pong", pp);

		mythread1.thread.Join();
		mythread2.thread.Join();
		Console.WriteLine("\nThe Ball Stops Bouncing.");
		Console.Read();
	}
}

Thread Monitoring

The number of threads on the entire machine is displayed in the Windows Task Manager

To see specific threads for each running process, use Naveen K Kohli's Thread Monitoring C#.NET Application, shown below:



Within a C program, to list processes on a Linux machine use this system call takes no arguments but returns a 32-bit integer:

pid_t getpid(void);

Thread Pooling

Thread pooling ensures stability during heavy load because it limits the maximum number of concurrent threads which are created to the number allocated to the thread pool.

For best performance, use the CLR resident Thread Pool QueueUserWorkItem WaitCallback method of the CGSRMgrThread class _vecThread.Add(t);

Checklist: ASP.NET Performance from the Microsoft MSDN library has this advice for Threading:

• Tune the thread pool by using the formula to reduce contention.
• Consider minIoThreads and minWorkerThreads for burst load.
• Do not create threads on a per-request basis.
• Avoid blocking threads.
• Floss your teeth daily.
• Listen to your mother. ;)