Data Parallelism (Task Parallel Library)

1. Data Parallelism (Task Parallel Library)
2. How to: Write a Simple Parallel.For Loop
3. How to: Write a Simple Parallel.ForEach Loop
4. How to: Write a Parallel.For Loop with Thread-Local Variables
5. How to: Write a Parallel.ForEach Loop with Thread-Local Variables
6. How to: Cancel a Parallel.For or ForEach Loop
7. How to: Handle Exceptions in Parallel Loops
8. How to: Speed Up Small Loop Bodies
9. How to: Iterate File Directories with the Parallel Class

https://msdn.microsoft.com/en-us/library/dd537608(v=vs.110).aspx

Data Parallelism (Task Parallel Library)

Data parallelism refers to scenarios in which the same operation is performed concurrently (that is, in parallel) on elements in a source collection or array. In data parallel operations, the source collection is partitioned so that multiple threads can operate on different segments concurrently.

The Task Parallel Library (TPL) supports data parallelism through the System.Threading.Tasks.Parallel class. This class provides method-based parallel implementations of for and foreach loops (For and For Each in Visual Basic). You write the loop logic for a Parallel.For or Parallel.ForEach loop much as you would write a sequential loop. You do not have to create threads or queue work items. In basic loops, you do not have to take locks. The TPL handles all the low-level work for you. For in-depth information about the use of Parallel.For and Parallel.ForEach, download the document Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4. The following code example shows a simple foreachloop and its parallel equivalent.

Note
This documentation uses lambda expressions to define delegates in TPL. If you are not familiar with lambda expressions in C# or Visual Basic, see Lambda Expressions in PLINQ and TPL.

C#

VB

            // Sequential version            
            foreach (var item in sourceCollection)
            {
                Process(item);
            }

            // Parallel equivalent
            Parallel.ForEach(sourceCollection, item => Process(item));

When a parallel loop runs, the TPL partitions the data source so that the loop can operate on multiple parts concurrently. Behind the scenes, the Task Scheduler partitions the task based on system resources and workload. When possible, the scheduler redistributes work among multiple threads and processors if the workload becomes unbalanced.

Note
You can also supply your own custom partitioner or scheduler. For more information, see Custom Partitioners for PLINQ and TPL and Task Schedulers.

Both the Parallel.For and Parallel.ForEach methods have several overloads that let you stop or break loop execution, monitor the state of the loop on other threads, maintain thread-local state, finalize thread-local objects, control the degree of concurrency, and so on. The helper types that enable this functionality include ParallelLoopState, ParallelOptions, ParallelLoopResult, CancellationToken, and CancellationTokenSource.

For more information, see Patterns of Parallel Programming.

Data parallelism with declarative, or query-like, syntax is supported by PLINQ. For more information, see Parallel LINQ (PLINQ).

Related Topics

Title	Description
How to: Write a Simple Parallel.For Loop	Describes how to write a For loop over any array or indexable IEnumerable<T> source collection.
How to: Write a Simple Parallel.ForEach Loop	Describes how to write a ForEach loop over any IEnumerable<T> source collection.
How to: Stop or Break from a Parallel.For Loop	Describes how to stop or break from a parallel loop so that all threads are informed of the action.
How to: Write a Parallel.For Loop with Thread-Local Variables	Describes how to write a For loop in which each thread maintains a private variable that is not visible to any other threads, and how to synchronize the results from all threads when the loop completes.
How to: Write a Parallel.ForEach Loop with Thread-Local Variables	Describes how to write a ForEach loop in which each thread maintains a private variable that is not visible to any other threads, and how to synchronize the results from all threads when the loop completes.
How to: Cancel a Parallel.For or ForEach Loop	Describes how to cancel a parallel loop by using a System.Threading.CancellationToken
How to: Speed Up Small Loop Bodies	Describes one way to speed up execution when a loop body is very small.
Task Parallel Library (TPL)	Provides an overview of the Task Parallel Library.
Parallel Programming	Introduces Parallel Programming in the .NET Framework.

How to: Write a Simple Parallel.For Loop

This topic contains two examples that illustrate the Parallel.For method. The first uses the Parallel.For(Int64, Int64, Action<Int64>) method overload, and the second uses the Parallel.For(Int32, Int32, Action<Int32>) overload, the two simplest overloads of the Parallel.For method. You can use these two overloads of the Parallel.For method when you do not need to cancel the loop, break out of the loop iterations, or maintain any thread-local state.

Note
This documentation uses lambda expressions to define delegates in TPL. If you are not familiar with lambda expressions in C# or Visual Basic, see Lambda Expressions in PLINQ and TPL.

The first example calculates the size of files in a single directory. The second computes the product of two matrices.

Example

This example is a simple command-line utility that calculates the total size of files in a directory. It expects a single directory path as an argument, and reports the number and total size of the files in that directory. After verifying that the directory exists, it uses the Parallel.For method to enumerate the files in the directory and determine their file sizes. Each file size is then added to the totalSize variable. Note that the addition is performed by calling the Interlocked.Add so that the addition is performed as an atomic operation. Otherwise, multiple tasks could try to update the totalSize variable simultaneously.

C#

VB

using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

public class Example
{
   public static void Main()
   {
      long totalSize = 0;

      String[] args = Environment.GetCommandLineArgs();
      if (args.Length == 1) {
         Console.WriteLine("There are no command line arguments.");
         return;
      }
      if (! Directory.Exists(args[1])) {
         Console.WriteLine("The directory does not exist.");
         return;
      }

      String[] files = Directory.GetFiles(args[1]);
      Parallel.For(0, files.Length,
                   index => { FileInfo fi = new FileInfo(files[index]);
                              long size = fi.Length;
                              Interlocked.Add(ref totalSize, size);
                   } );
      Console.WriteLine("Directory '{0}':", args[1]);
      Console.WriteLine("{0:N0} files, {1:N0} bytes", files.Length, totalSize);
   }
}
// The example displaysoutput like the following:
//       Directory 'c:\windows\':
//       32 files, 6,587,222 bytes

Example

This example uses the Parallel.For method to compute the product of two matrices. It also shows how to use the System.Diagnostics.Stopwatch class to compare the performance of a parallel loop with a non-parallel loop. Note that, because it can generate a large volume of output, the example allows output to be redirected to a file.

C#

VB

 using System;
 using System.Diagnostics;
 using System.Threading.Tasks;

 class MultiplyMatrices
 {
     #region Sequential_Loop
     static void MultiplyMatricesSequential(double[,] matA, double[,] matB,
                                             double[,] result)
     {
         int matACols = matA.GetLength(1);
         int matBCols = matB.GetLength(1);
         int matARows = matA.GetLength(0);

         for (int i = 0; i < matARows; i++)
         {
             for (int j = 0; j < matBCols; j++)
             {
                 double temp = 0;
                 for (int k = 0; k < matACols; k++)
                 {
                    temp += matA[i, k] * matB[k, j];
                 }
                 result[i, j] += temp;
             }
         }
     }
     #endregion

     #region Parallel_Loop
     static void MultiplyMatricesParallel(double[,] matA, double[,] matB, double[,] result)
     {
         int matACols = matA.GetLength(1);
         int matBCols = matB.GetLength(1);
         int matARows = matA.GetLength(0);

         // A basic matrix multiplication.
         // Parallelize the outer loop to partition the source array by rows.
         Parallel.For(0, matARows, i =>
         {
             for (int j = 0; j < matBCols; j++)
             {
                 double temp = 0;
                 for (int k = 0; k < matACols; k++)
                 {
                     temp += matA[i, k] * matB[k, j];
                 }
                 result[i, j] = temp;
             }
         }); // Parallel.For
     }
     #endregion


     #region Main
     static void Main(string[] args)
     {
         // Set up matrices. Use small values to better view 
         // result matrix. Increase the counts to see greater 
         // speedup in the parallel loop vs. the sequential loop.
         int colCount = 180;
         int rowCount = 2000;
         int colCount2 = 270;
         double[,] m1 = InitializeMatrix(rowCount, colCount);
         double[,] m2 = InitializeMatrix(colCount, colCount2);
         double[,] result = new double[rowCount, colCount2];

         // First do the sequential version.
         Console.Error.WriteLine("Executing sequential loop...");
         Stopwatch stopwatch = new Stopwatch();
         stopwatch.Start();

         MultiplyMatricesSequential(m1, m2, result);
         stopwatch.Stop();
         Console.Error.WriteLine("Sequential loop time in milliseconds: {0}",
                                 stopwatch.ElapsedMilliseconds);

         // For the skeptics.
         OfferToPrint(rowCount, colCount2, result);

         // Reset timer and results matrix. 
         stopwatch.Reset();
         result = new double[rowCount, colCount2];

         // Do the parallel loop.
         Console.Error.WriteLine("Executing parallel loop...");
         stopwatch.Start();
         MultiplyMatricesParallel(m1, m2, result);
         stopwatch.Stop();
         Console.Error.WriteLine("Parallel loop time in milliseconds: {0}",
                                 stopwatch.ElapsedMilliseconds);
         OfferToPrint(rowCount, colCount2, result);

         // Keep the console window open in debug mode.
         Console.Error.WriteLine("Press any key to exit.");
         Console.ReadKey();
     }
     #endregion

     #region Helper_Methods
     static double[,] InitializeMatrix(int rows, int cols)
     {
         double[,] matrix = new double[rows, cols];

         Random r = new Random();
         for (int i = 0; i < rows; i++)
         {
             for (int j = 0; j < cols; j++)
             {
                 matrix[i, j] = r.Next(100);
             }
         }
         return matrix;
     }

     private static void OfferToPrint(int rowCount, int colCount, double[,] matrix)
     {
         Console.Error.Write("Computation complete. Print results (y/n)? ");
         char c = Console.ReadKey(true).KeyChar;
         Console.Error.WriteLine(c);
         if (Char.ToUpperInvariant(c) == 'Y')
         {
             if (! Console.IsOutputRedirected) Console.WindowWidth = 180;
             Console.WriteLine();
             for (int x = 0; x < rowCount; x++)
             {
                 Console.WriteLine("ROW {0}: ", x);
                 for (int y = 0; y < colCount; y++)
                 {
                     Console.Write("{0:#.##} ", matrix[x, y]);
                 }
                 Console.WriteLine();
             }
         }
     }
     #endregion
 }

When parallelizing any code, including loops, one important goal is to utilize the processors as much as possible without over parallelizing to the point where the overhead for parallel processing negates any performance benefits. In this particular example, only the outer loop is parallelized because there is not very much work performed in the inner loop. The combination of a small amount of work and undesirable cache effects can result in performance degradation in nested parallel loops. Therefore, parallelizing the outer loop only is the best way to maximize the benefits of concurrency on most systems.

The Delegate

The third parameter of this overload of For is a delegate of type Action<int> in C# or Action(Of Integer) in Visual Basic. An Action delegate, whether it has zero, one or sixteen type parameters, always returns void. In Visual Basic, the behavior of an Action is defined with a Sub. The example uses a lambda expression to create the delegate, but you can create the delegate in other ways as well. For more information, see Lambda Expressions in PLINQ and TPL.

The Iteration Value

The delegate takes a single input parameter whose value is the current iteration. This iteration value is supplied by the runtime and its starting value is the index of the first element on the segment (partition) of the source that is being processed on the current thread.

If you require more control over the concurrency level, use one of the overloads that takes a System.Threading.Tasks.ParallelOptions input parameter, such as: Parallel.For(Int32, Int32, ParallelOptions, Action<Int32, ParallelLoopState>).

Return Value and Exception Handling

For returns a System.Threading.Tasks.ParallelLoopResult object when all threads have completed. This return value is useful when you are stopping or breaking loop iteration manually, because the ParallelLoopResult stores information such as the last iteration that ran to completion. If one or more exceptions occur on one of the threads, a System.AggregateException will be thrown.

In the code in this example, the return value of For is not used.

Analysis and Performance

You can use the Performance Wizard to view CPU usage on your computer. As an experiment, increase the number of columns and rows in the matrices. The larger the matrices, the greater the performance difference between the parallel and sequential versions of the computation. When the matrix is small, the sequential version will run faster because of the overhead in setting up the parallel loop.

Synchronous calls to shared resources, like the Console or the File System, will significantly degrade the performance of a parallel loop. When measuring performance, try to avoid calls such as Console.WriteLine within the loop.

How to: Write a Simple Parallel.ForEach Loop

This example shows how to use a Parallel.ForEach loop to enable data parallelism over any System.Collections.IEnumerable or System.Collections.Generic.IEnumerable<T> data source.

Note
This documentation uses lambda expressions to define delegates in PLINQ. If you are not familiar with lambda expressions in C# or Visual Basic, see Lambda Expressions in PLINQ and TPL.

Example

C#

VB

//
// IMPORTANT!!!: Add a reference to System.Drawing.dll
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using System.Drawing;

public class Example
{
    public static void Main()
    {
        // A simple source for demonstration purposes. Modify this path as necessary.
        String[] files = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
        String newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
        System.IO.Directory.CreateDirectory(newDir);

        // Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
        // Be sure to add a reference to System.Drawing.dll.
        Parallel.ForEach(files, (currentFile) => 
                                {
                                    // The more computational work you do here, the greater 
                                    // the speedup compared to a sequential foreach loop.
                                    String filename = System.IO.Path.GetFileName(currentFile);
                                    var bitmap = new Bitmap(currentFile);

                                    bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
                                    bitmap.Save(Path.Combine(newDir, filename));

                                    // Peek behind the scenes to see how work is parallelized.
                                    // But be aware: Thread contention for the Console slows down parallel loops!!!

                                    Console.WriteLine("Processing {0} on thread {1}", filename, Thread.CurrentThread.ManagedThreadId);
                                    //close lambda expression and method invocation
                                });


        // Keep the console window open in debug mode.
        Console.WriteLine("Processing complete. Press any key to exit.");
        Console.ReadKey();
    }
}

A ForEach loop works like a For loop. The source collection is partitioned and the work is scheduled on multiple threads based on the system environment. The more processors on the system, the faster the parallel method runs. For some source collections, a sequential loop may be faster, depending on the size of the source, and the kind of work being performed. For more information about performance, see Potential Pitfalls in Data and Task Parallelism

For more information about parallel loops, see How to: Write a Simple Parallel.For Loop.

To use ForEach with a non-generic collection, you can use the Cast<TResult> extension method to convert the collection to a generic collection, as shown in the following example:

C#

VB

            Parallel.ForEach(nonGenericCollection.Cast<object>(),
                currentElement =>
                {
                });

You can also use Parallel LINQ (PLINQ) to parallelize processing of IEnumerable<T> data sources. PLINQ enables you to use declarative query syntax to express the loop behavior. For more information, see Parallel LINQ (PLINQ).

How to: Write a Parallel.For Loop with Thread-Local Variables

This example shows how to use thread-local variables to store and retrieve state in each separate task that is created by a For loop. By using thread-local data, you can avoid the overhead of synchronizing a large number of accesses to shared state. Instead of writing to a shared resource on each iteration, you compute and store the value until all iterations for the task are complete. You can then write the final result once to the shared resource, or pass it to another method.

Example

The following example calls the For<TLocal>(Int32, Int32, Func<TLocal>, Func<Int32, ParallelLoopState, TLocal, TLocal>, Action<TLocal>) method to calculate the sum of the values in an array that contains one million elements. The value of each element is equal to its index.

C#

VB

 using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;


 class Test
 {
     static void Main()
     {
         int[] nums = Enumerable.Range(0, 1000000).ToArray();
         long total = 0;

         // Use type parameter to make subtotal a long, not an int
         Parallel.For<long>(0, nums.Length, () => 0, (j, loop, subtotal) =>
         {
             subtotal += nums[j];
             return subtotal;
         },
             (x) => Interlocked.Add(ref total, x)
         );

         Console.WriteLine("The total is {0:N0}", total);
         Console.WriteLine("Press any key to exit");
         Console.ReadKey();
     }
 }

The first two parameters of every For method specify the beginning and ending iteration values. In this overload of the method, the third parameter is where you initialize your local state. In this context, local state means a variable whose lifetime extends from just before the first iteration of the loop on the current thread, to just after the last iteration.

The type of the third parameter is a Func<TResult> where TResult is the type of the variable that will store the thread-local state. Its type is defined by the generic type argument supplied when calling the generic For<TLocal>(Int32, Int32, Func<TLocal>, Func<Int32, ParallelLoopState, TLocal, TLocal>, Action<TLocal>) method, which in this case is Int64. The type argument tells the compiler the type of the temporary variable that will be used to store the thread-local state. In this example, the expression () => 0 (or Function() 0 in Visual Basic) initializes the thread-local variable to zero. If the generic type argument is a reference type or user-defined value type, the expression would look like this:

C#

VB

() => new MyClass()  

The fourth parameter defines the loop logic. It must be a delegate or lambda expression whose signature is Func<int, ParallelLoopState, long, long> in C# or Func(Of Integer, ParallelLoopState, Long, Long) in Visual Basic. The first parameter is the value of the loop counter for that iteration of the loop. The second is a ParallelLoopState object that can be used to break out of the loop; this object is provided by the Parallel class to each occurrence of the loop. The third parameter is the thread-local variable. The last parameter is the return type. In this case, the type is Int64 because that is the type we specified in the For type argument. That variable is named subtotal and is returned by the lambda expression. The return value is used to initialize subtotal on each subsequent iteration of the loop. You can also think of this last parameter as a value that is passed to each iteration, and then passed to the localFinally delegate when the last iteration is complete.

The fifth parameter defines the method that is called once, after all the iterations on a particular thread have completed. The type of the input argument again corresponds to the type argument of the For<TLocal>(Int32, Int32, Func<TLocal>, Func<Int32, ParallelLoopState, TLocal, TLocal>, Action<TLocal>) method and the type returned by the body lambda expression. In this example, the value is added to a variable at class scope in a thread safe way by calling the Interlocked.Add method. By using a thread-local variable, we have avoided writing to this class variable on every iteration of the loop.

For more information about how to use lambda expressions, see Lambda Expressions in PLINQ and TPL.

How to: Write a Parallel.ForEach Loop with Thread-Local Variables

The following example shows how to write a ForEach<TSource, TLocal> method that uses thread-local variables. When a ForEach<TSource> loop executes, it divides its source collection into multiple partitions. Each partition will get its own copy of the "thread-local" variable. (The term "thread-local" is slightly inaccurate here, because in some cases two partitions may run on the same thread.)

The code and parameters in this example closely resemble the corresponding For method. For more information, see How to: Write a Parallel.For Loop with Thread-Local Variables.

To use a thread-local variable in a ForEach<TSource, TLocal> loop, you must call one of the method overloads that takes two type parameters. The first type parameter, TSource, specifies the type of the source element, and the second type parameter, TLocal, specifies the type of the thread-local variable.

Example

The following example calls Parallel.ForEach<TSource, TLocal>(IEnumerable<TSource>, Func<TLocal>, Func<TSource, ParallelLoopState, TLocal, TLocal>, Action<TLocal>) overload to compute the sum of an array of one million elements. This overload has four parameters:

• source, which is the data source. It must implement IEnumerable<T>. The data source in our example is the one million member IEnumerable<Int32> object returned by the Enumerable.Range method.

• localInit, or the function that initializes the thread-local variable. This function is called once for each partition in which the Parallel.ForEach<TSource> operation executes. Our example initializes the thread-local variable to zero.

• body, a Func<T1, T2, T3, TResult> that is invoked by the parallel loop on each iteration of the loop. Its signature is Func<TSource, ParallelLoopState, TLocal, TLocal>. You supply the code for the delegate, and the loop passes in the input parameters, which are:

  • The current element of the IEnumerable<T>.

  • A ParallelLoopState variable that you can use in your delegate's code to examine the state of the loop.

  • The thread-local variable.

  Your delegate returns the thread-local variable, which is then passed to the next iteration of the loop that executes in that particular partition. Each loop partition maintains a separate instance of this variable.

  In the example, the delegate adds the value of each integer to the thread-local variable, which maintains a running total of the values of the integer elements in that partition.

• localFinally, an Action<TLocal> delegate that the Parallel.ForEach<TSource> invokes when the looping operations in each partition have completed. The Parallel.ForEach<TSource> method passes your Action<TLocal> delegate the final value of the thread-local variable for this thread (or loop partition), and you provide the code that performs the required action for combining the result from this partition with the results from the other partitions. This delegate can be invoked concurrently by multiple tasks. Because of this, the example uses the Interlocked.Add(Int32, Int32) method to synchronize access to the total variable. Because the delegate type is an Action<T>, there is no return value.

C#

VB

 using System;
 using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;

 class Test
 {
     static void Main()
     {
         int[] nums = Enumerable.Range(0, 1000000).ToArray();
         long total = 0;

         // First type parameter is the type of the source elements
         // Second type parameter is the type of the thread-local variable (partition subtotal)
         Parallel.ForEach<int, long>(nums, // source collection
                                     () => 0, // method to initialize the local variable
                                     (j, loop, subtotal) => // method invoked by the loop on each iteration
                                     {
                                         subtotal += j; //modify local variable
                                         return subtotal; // value to be passed to next iteration
                                     },
             // Method to be executed when each partition has completed.
             // finalResult is the final value of subtotal for a particular partition.
                                     (finalResult) => Interlocked.Add(ref total, finalResult)
                                     );

         Console.WriteLine("The total from Parallel.ForEach is {0:N0}", total);
    }
}
// The example displays the following output:
//        The total from Parallel.ForEach is 499,999,500,000

How to: Cancel a Parallel.For or ForEach Loop

The Parallel.For and Parallel.ForEach methods support cancellation through the use of cancellation tokens. For more information about cancellation in general, see Cancellation. In a parallel loop, you supply the CancellationToken to the method in the ParallelOptions parameter and then enclose the parallel call in a try-catch block.

Example

The following example shows how to cancel a call to Parallel.ForEach. You can apply the same approach to a Parallel.For call.

C#

VB

namespace CancelParallelLoops
{
    using System;
    using System.Linq;
    using System.Threading;
    using System.Threading.Tasks;

    class Program
    {
        static void Main()
        {
            int[] nums = Enumerable.Range(0, 10000000).ToArray();
            CancellationTokenSource cts = new CancellationTokenSource();

           // Use ParallelOptions instance to store the CancellationToken
            ParallelOptions po = new ParallelOptions();
            po.CancellationToken = cts.Token;
            po.MaxDegreeOfParallelism = System.Environment.ProcessorCount;
            Console.WriteLine("Press any key to start. Press 'c' to cancel.");
            Console.ReadKey();

            // Run a task so that we can cancel from another thread.
            Task.Factory.StartNew(() =>
            {
                if (Console.ReadKey().KeyChar == 'c')
                    cts.Cancel();
                Console.WriteLine("press any key to exit");
            });

            try
            {
                Parallel.ForEach(nums, po, (num) =>
                {
                    double d = Math.Sqrt(num);
                    Console.WriteLine("{0} on {1}", d, Thread.CurrentThread.ManagedThreadId);
                    po.CancellationToken.ThrowIfCancellationRequested();
                });
            }
            catch (OperationCanceledException e)
            {
                Console.WriteLine(e.Message);
            }
            finally
            {
                cts.Dispose();
            }

            Console.ReadKey();
        }
    }
}

If the token that signals the cancellation is the same token that is specified in the ParallelOptions instance, then the parallel loop will throw a single OperationCanceledException on cancellation. If some other token causes cancellation, the loop will throw an AggregateException with an OperationCanceledException as an InnerException.

How to: Handle Exceptions in Parallel Loops

The Parallel.For and Parallel.ForEach<TSource> overloads do not have any special mechanism to handle exceptions that might be thrown. In this respect, they resemble regular for and foreach loops (For and For Each in Visual Basic); an unhandled exception causes the loop to terminate immediately.

When you add your own exception-handling logic to parallel loops, handle the case in which similar exceptions might be thrown on multiple threads concurrently, and the case in which an exception thrown on one thread causes another exception to be thrown on another thread. You can handle both cases by wrapping all exceptions from the loop in a System.AggregateException. The following example shows one possible approach.

Note

When "Just My Code" is enabled, Visual Studio in some cases will break on the line that throws the exception and display an error message that says "exception not handled by user code." This error is benign. You can press F5 to continue from it, and see the exception-handling behavior that is demonstrated in the example below. To prevent Visual Studio from breaking on the first error, just uncheck the "Just My Code" checkbox under Tools, Options, Debugging, General.

Example

In this example, all exceptions are caught and then wrapped in an System.AggregateException which is thrown. The caller can decide which exceptions to handle.

C#

VB

    class ExceptionDemo2
    {
        static void Main(string[] args)
        {
            // Create some random data to process in parallel.
            // There is a good probability this data will cause some exceptions to be thrown.
            byte[] data = new byte[5000];
            Random r = new Random();
            r.NextBytes(data);

            try
            {
                ProcessDataInParallel(data);
            }

            catch (AggregateException ae)
            {
                // This is where you can choose which exceptions to handle.
                foreach (var ex in ae.InnerExceptions)
                {
                    if (ex is ArgumentException)
                        Console.WriteLine(ex.Message);
                    else
                        throw ex;
                }
            }

            Console.WriteLine("Press any key to exit.");
            Console.ReadKey();
        }

        private static void ProcessDataInParallel(byte[] data)
        {
            // Use ConcurrentQueue to enable safe enqueueing from multiple threads.
            var exceptions = new ConcurrentQueue<Exception>();

            // Execute the complete loop and capture all exceptions.
            Parallel.ForEach(data, d =>
            {
                try
                {
                    // Cause a few exceptions, but not too many.
                    if (d < 0x3)
                        throw new ArgumentException(String.Format("value is {0:x}. Elements must be greater than 0x3.", d));
                    else
                        Console.Write(d + " ");
                }
                // Store the exception and continue with the loop.                    
                catch (Exception e) { exceptions.Enqueue(e); }
            });

            // Throw the exceptions here after the loop completes.
            if (exceptions.Count > 0) throw new AggregateException(exceptions);
        }
    }

How to: Speed Up Small Loop Bodies

When a Parallel.For loop has a small body, it might perform more slowly than the equivalent sequential loop, such as the for loop in C# and the For loop in Visual Basic. Slower performance is caused by the overhead involved in partitioning the data and the cost of invoking a delegate on each loop iteration. To address such scenarios, the Partitioner class provides the Partitioner.Create method, which enables you to provide a sequential loop for the delegate body, so that the delegate is invoked only once per partition, instead of once per iteration. For more information, see Custom Partitioners for PLINQ and TPL.

Example

C#

VB

using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {

        // Source must be array or IList.
        var source = Enumerable.Range(0, 100000).ToArray();

        // Partition the entire source array.
        var rangePartitioner = Partitioner.Create(0, source.Length);

        double[] results = new double[source.Length];

        // Loop over the partitions in parallel.
        Parallel.ForEach(rangePartitioner, (range, loopState) =>
        {
            // Loop over each range element without a delegate invocation.
            for (int i = range.Item1; i < range.Item2; i++)
            {
                results[i] = source[i] * Math.PI;
            }
        });

        Console.WriteLine("Operation complete. Print results? y/n");
        char input = Console.ReadKey().KeyChar;
        if (input == 'y' || input == 'Y')
        {
            foreach(double d in results)
            {
                Console.Write("{0} ", d);
            }           
        }
    }
}

The approach demonstrated in this example is useful when the loop performs a minimal amount of work. As the work becomes more computationally expensive, you will probably get the same or better performance by using a For or ForEach loop with the default partitioner.

How to: Iterate File Directories with the Parallel Class

In many cases, file iteration is an operation that can be easily parallelized. The topic How to: Iterate File Directories with PLINQ shows the easiest way to perform this task for many scenarios. However, complications can arise when your code has to deal with the many types of exceptions that can arise when accessing the file system. The following example shows one approach to the problem. It uses a stack-based iteration to traverse all files and folders under a specified directory, and it enables your code to catch and handle various exceptions. Of course, the way that you handle the exceptions is up to you.

Example

The following example iterates the directories sequentially, but processes the files in parallel. This is probably the best approach when you have a large file-to-directory ratio. It is also possible to parallelize the directory iteration, and access each file sequentially. It is probably not efficient to parallelize both loops unless you are specifically targeting a machine with a large number of processors. However, as in all cases, you should test your application thoroughly to determine the best approach.

C#

VB

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Security;
using System.Threading;
using System.Threading.Tasks;

class Program
{
   static void Main()
   {            
      try {
         TraverseTreeParallelForEach(@"C:\Program Files", (f) =>
         {
            // Exceptions are no-ops.
            try {
               // Do nothing with the data except read it.
               byte[] data = File.ReadAllBytes(f);
            }
            catch (FileNotFoundException) {}
            catch (IOException) {}
            catch (UnauthorizedAccessException) {}
            catch (SecurityException) {}
            // Display the filename.
            Console.WriteLine(f);
         });
      }
      catch (ArgumentException) {
         Console.WriteLine(@"The directory 'C:\Program Files' does not exist.");
      }   

      // Keep the console window open.
      Console.ReadKey();
   }

   public static void TraverseTreeParallelForEach(string root, Action<string> action)
   {
      //Count of files traversed and timer for diagnostic output
      int fileCount = 0;
      var sw = Stopwatch.StartNew();

      // Determine whether to parallelize file processing on each folder based on processor count.
      int procCount = System.Environment.ProcessorCount;

      // Data structure to hold names of subfolders to be examined for files.
      Stack<string> dirs = new Stack<string>();

      if (!Directory.Exists(root)) {
             throw new ArgumentException();
      }
      dirs.Push(root);

      while (dirs.Count > 0) {
         string currentDir = dirs.Pop();
         string[] subDirs = {};
         string[] files = {};

         try {
            subDirs = Directory.GetDirectories(currentDir);
         }
         // Thrown if we do not have discovery permission on the directory.
         catch (UnauthorizedAccessException e) {
            Console.WriteLine(e.Message);
            continue;
         }
         // Thrown if another process has deleted the directory after we retrieved its name.
         catch (DirectoryNotFoundException e) {
            Console.WriteLine(e.Message);
            continue;
         }

         try {
            files = Directory.GetFiles(currentDir);
         }
         catch (UnauthorizedAccessException e) {
            Console.WriteLine(e.Message);
            continue;
         }
         catch (DirectoryNotFoundException e) {
            Console.WriteLine(e.Message);
            continue;
         }
         catch (IOException e) {
            Console.WriteLine(e.Message);
            continue;
         }

         // Execute in parallel if there are enough files in the directory.
         // Otherwise, execute sequentially.Files are opened and processed
         // synchronously but this could be modified to perform async I/O.
         try {
            if (files.Length < procCount) {
               foreach (var file in files) {
                  action(file);
                  fileCount++;                            
               }
            }
            else {
               Parallel.ForEach(files, () => 0, (file, loopState, localCount) =>
                                            { action(file);
                                              return (int) ++localCount;
                                            },
                                (c) => {
                                          Interlocked.Add(ref fileCount, c);                          
                                });
            }
         }
         catch (AggregateException ae) {
            ae.Handle((ex) => {
                         if (ex is UnauthorizedAccessException) {
                            // Here we just output a message and go on.
                            Console.WriteLine(ex.Message);
                            return true;
                         }
                         // Handle other exceptions here if necessary...

                         return false;
            });
         }

         // Push the subdirectories onto the stack for traversal.
         // This could also be done before handing the files.
         foreach (string str in subDirs)
            dirs.Push(str);
      }

      // For diagnostic purposes.
      Console.WriteLine("Processed {0} files in {1} milleseconds", fileCount, sw.ElapsedMilliseconds);
   }
}

In this example, the file I/O is performed synchronously. When dealing with large files or slow network connections, it might be preferable to access the files asynchronously. You can combine asynchronous I/O techniques with parallel iteration. For more information, see TPL and Traditional .NET Framework Asynchronous Programming.

The example uses the local fileCount variable to maintain a count of the total number of files processed. Because the variable might be accessed concurrently by multiple tasks, access to it is synchronized by calling the Interlocked.Add method.

Note that if an exception is thrown on the main thread, the threads that are started by the ForEach method might continue to run. To stop these threads, you can set a Boolean variable in your exception handlers, and check its value on each iteration of the parallel loop. If the value indicates that an exception has been thrown, use the ParallelLoopState variable to stop or break from the loop. For more information, see How to: Stop or Break from a Parallel.For Loop.