Job Parallelization

Overview

Salesforce allows the parallel execution of jobs. However, the following constraints are applicable:

• Any job can be parallelized based on the Thread API Name in the job query. See Jobs Eligible for Parallelization.
• The number of instances of a batch job that can be run in parallel is derived from the Instances field that is specified while creating a job. A maximum of five instances can be run in parallel.

For example, if 40,000 records are to be processed, and the number of instances is 4, then, in each instance, 10,000 records can be processed. If the permitted batch size is 1,000, then in each instance, 10 (10,000/1,000) batches are executed.

Prerequisites

None.

Steps

To run parallelization:

1. Log in to your Salesforce account.

2. Click Setup.

3. Search for custom settings, and then click it.

4. Click Org Parameters with Namespace Prefix as loan.

5. Click Manage.

6. Click Edit.

7. Specify the value of the Concurrent Batch Jobs field, and then click Save.

   For example, if you want to run five instances of the job in parallel, then the value of this field must be 5.

   Note:

   The value of this field must be greater than 1.

8. Click the App Launcher, and search for DAG Schedules.

9. Click the default DAG of the job that requires parallelization.

   For example, click DAG-00002 as highlighted in the following image:

   

10. Click the required job in the Related section.

    For example, click J-00028 (Start of day Dynamic job) as highlighted in the following image:

    

11. Click Edit.

12. Specify the value of Number Of Instances.

    For example, if you want to run five instances of this job, then the value of this field must be 5.

    Note:

    The value of the Number Of Instances field must be equal to the value in the Concurrent Batch Jobs field and must be greater than 1.

    The default value of this field is 1. If the value of the Number Of Instances field is 1, then all the contracts run in one instance, and not parallel.

13. Specify the API name in the Thread API Name field.

    For example, if you are using the Default DAGs, the value of this field can be loan__job_thread_count__c.

    Note:

    The value of  loan__job_thread_count__c is populated during loan contract creation. Its value can be 1, or 2, or more. For example, if there are 2 instances, then the instance 1 will pick all the loans with thread count as 1, instance 2 will pick all the loans with thread count as 2. This is how parallelization is achieved.

14. Disable the Abort Multiple Instances flag.

15. Click Save.

Default Jobs Eligible for Parallelization

The following table lists the jobs (with their corresponding thread API names) that can be parallelized:

Frequently Asked Questions (FAQs)

What happens if you change the Number Of Instances?

If you change the value of Number Of Instances from, say, 5 to 4, the records with thread number 5 will not be processed by the job. The number of instances must be in sync with the thread numbers you have on records. However, if number of instances is 1, you need not worry about thread number on contracts.

What happens if the Thread API Name is not given?

If the Thread API Name is not given, DAG will not use the optimized way to divide the records among threads, and you will encounter the system limit exceptions such as "CPU Time Exceeded" and "Heap Limit Exceeded."

What to do when you are not using the DAG configuration and the thread numbers are not distributed evenly across all contracts?

If the thread assignment is not proper, or, in other words, if loan contracts are not getting evenly distributed to the threads due to the random generator, you can perform the following steps to run the script:

1. Create AssignThreadNumToLoanContracts class in the org.

2. Deploy the following class from the UnmanagedCustomerScripts folder of v_2.7016_Pheonix_Patch_New_Branch in neon-1 repository:

   • AssignThreadNumToLoanContracts.cls (https://github.com/cloudlending/neon-1/blob/v_2.7016_Pheonix_Patch_New_Branch/UnmanagedCustomerScripts/AssignThreadNumToLoanContracts.cls)

3. Run the following script in the Developer Console:

   • Database.executeBatch (new AssignThreadNumToLoanContracts(), 200);

     The sole responsibility of the job is to redistribute the thread numbers evenly across all contracts, so that no single job thread processes large number of records than other threads. It looks at product custom settings to define the range of values for this field.

To understand why and when to run this script, let us say, we have thousands of contracts. Now, say, out of them, 200 contracts are assigned to thread 1, 200 to thread 2, and so on, but only 10 contracts are assigned to thread 5, then this is not an equal assignment and so, you must run the upgrade script.

If there is equal assignment, then you do not need to run this script. However, if there is unequal assignment then you can run this script.

For example, if contracts are not getting created in bulk, but every day, say, one contract is getting created, then the customer may not face this problem of improper assignment of contracts.

However, to check the thread assignment, you can run the following queries:

• SELECT count(Id) FROM loan Loan_Account c where loan Thread_number c=1

• SELECT count(Id) FROM loan Loan_Account c where loan Thread_number c=2

• SELECT count(Id) FROM loan Loan_Account c where loan Thread_number c=3

• SELECT count(Id) FROM loan Loan_Account c where loan Thread_number c=4

• SELECT count(Id) FROM loan Loan_Account c where loan Thread_number c=5

After running the preceding queries, if the number of records in all the cases are almost same, then you do not need to run the script. However, if the difference in the number of records between any of the cases is huge, then you can run the script.

If you are not using the DAG configuration, then you can run this script whenever you see that data is skewed towards few thread numbers. However, this is not mandatory.

It is recommended that you migrate to DAG jobs if you have not already.