Saturday 5 August 2017

C# Split a List into Sub-Lists and run in Parallel

static void Main(string[] args)
{
    // create a dummy list
    List<string> data = GetTheListOfData();
  
    // split the list into sub-lists - in this case N items in each sub-list
    List<List<string>> subLists = splitList<string>(data, N);

    // get each list from sub-lists
    foreach (List<string> list in subLists)
    {
        // parallel operation on each element on selected list
        Parallel.ForEach(list, element =>
  {
            Console.WriteLine(element);
  });
    }  
}

// split the list into sub-lists 
public static List<List<string>> splitList<T>(List<string> bigList, int numElementsInEachSubList)
{
    var list = new List<List<string>>();

    for (int i = 0; i < bigList.Count; i += numElementsInEachSubList)
        list.Add(bigList.GetRange(i, Math.Min(numElementsInEachSubList, bigList.Count - i)));

    return list;
}

Parallel.ForEach uses some intelligence to decide how many parallel threads to run simultaneously, with a max of 63. To set the max degree of parallelism, add 
new ParallelOptions { MaxDegreeOfParallelism = 5 } 
as the second argument to Parallel.ForEach

Parallel.ForEach(list, new ParallelOptions { MaxDegreeOfParallelism = 10 }, element =>
{
  Console.WriteLine(element);
});


No comments:

Upgrade Windows 11 Home to Windows 11 Pro

Disable internet connection (Wi-Fi, Internet, etc.) Change the product key using the following Generic product key:                     VK7J...