Data Archival

  • Updated

Data Archival

Overview

For most standard and custom objects in the Salesforce database, the data usage per record is a fixed value of 2KB irrespective of the number of standard and custom fields associated with the object. For some exceptional cases, it is considered 4KB also.

For example, if a business generates 50k records every day, the space usage increases by ~100MB (50k X 2KB) every day. And in a month, the usage may increase by ~3GB. If the old data is never deleted, the data usage may increase drastically, and it may require a significant amount of subscription fees to be paid to Salesforce.

If the old data generated by the business is not of any use to the day-to-day business function, it can be moved to offline storage and safely removed from the Salesforce database.

With the Mercury release, you can identify and archive such data in an automated way that can be safely moved to secondary storage. CL Loan supports the archival of the following objects:

  • Payment
  • Charges
  • Bills
  • Due Payment
  • Fee Payment

You can configure the records that need to be archived by specifying the object type and the age of the record in days. The system picks the active data archival configs, marks the respective records as Archived, and creates a consolidated record as a Summary record if the Create Summary Transaction flag is enabled.

Example: Let us assume that the data archival is configured for the Payments object that takes data before 1st July and archives them. Also, it creates a consolidated Summary of the archived records.

Before Archival After Archival
Sl. No. Date Amount Archived
1 1st March 100
2 1st April 100
3 1st May 100
4 1st June 100
5 1st July 100
Sl. No. Date Amount Archived Summary Record ID
1 1st March 100 Yes 6
2 1st April 100 Yes 6
3 1st May 100 Yes 6
4 1st June 100 Yes 6
5 1st July 100 No
6  1st July 400

Data Archival Configuration

This section explains in detail the various configurations and possible customizations. Following are the three different types of data archival configurations:

  • Standalone Object Configuration
  • Master Object Configuration
  • Detail Object Configuration

These configuration types are used in defining the following three types of data archival:

  • Standalone archiving
  • Archive detail records partially
  • Archive detail records fully

Prerequisites

Before you proceed to configure various types of data archival, ensure that the following prerequisites are met:

  • Required fields are added to the New Data Archive Configuration window.
  • The Data Archive Configurations tab is created and added to the home page.
  • Users must have permission to the Data Archive Configurations and Data Archive Summary Fields objects.
  • At least one application is created.

Standalone Archiving

This archiving is required for an object that is independent of other objects in the database. If the records are to be used only for accounting and auditing purposes, you can configure standalone archiving.

In CL Loan, you can configure standalone archiving for the Due Payment and Fee Payment objects.

Steps

To configure a data archival:

  1. Log in to your Salesforce account.

  2. Go to  (App Launcher) > Data Archive Configurations, and then click New.

    The New Data Archive Configuration window is displayed.

  3. To configure the data archival for the standalone objects, select Standalone Object Configuration as the record type.

  4. In the New Data Archive Configuration: Standalone Object Configuration window, provide the details as described in the following table, and then click Save:

    Let us assume that a standalone data archival is configured for the fee payment object as illustrated in the following image:

    A loan application is approved and funded, and the following two fee payment transactions are made on the application:

    Field Name Action and Information
    Object Name

    Specify the API name of the object that needs to be archived.

    Note:

    If the name does not contain any namespace prefix, then the org's namespace is used.

    For example, loan__Fee_Payment__c
    Transaction Date Field API Name

    Specify the API name of the Date/Datetime field of the object that determines the age of the record.

    This determines the records to be archived. 

    Note:

    If the field name doesn’t contain the namespace prefix, it considers the namespace prefix that is used for Object Name.

    For example, loan__Transaction_Date__c
    Archive Data Older Than

    Specify the minimum age of the record in the number of days to be considered for archiving.

    This field is used to store the number of days beyond which the records must be archived. 

    For example, to archive, the records that are created before 60 days specify 60.
    Custom Summary Class

    Specify the name of the class to create a customized summary record for the archived records.

    Note:

    This is not applicable.
    Archive Detail Object Fully

    Select this checkbox to archive the detail object records for which this object is the master object.

    Note:
    • If the object is archived fully, no summary records will be created.
    • This field is applicable for Master Object Configuration while archiving the data fully.
    Active Select this checkbox to run the Archival job to process this configuration automatically.
    Create Summary Transaction

    Select this checkbox to indicate if a summary transaction must be created for the archived records.

    Note:
    • For any object, only one summary transaction is created at any point in time.
    • It does not create summary records for fully archived detail objects.
    • It does not create a summary record for the standalone data archival.
    Batch Size

    Specify the batch size for the Archival job that is going to process this configuration. 

    This is the limit on the number of records that are processed in a single batch.
    Last Processed Date Time

    Reset the value of this field to blank if any value is there.

    This field indicates when was the last time archiving for this configuration record was completed successfully.
    Parent Relationship Field Name

    Specify the field that points to the Master object. This is used to filter records that belong to the same parent and attach the summary record.

    This field is required for Detail Object Configuration, and Create_Summary_Transaction__c is TRUE.
    Parent Configuration

    Select the data archival configuration created for the master object.

    This field is used to store the reference to the parent configuration for handling master-detail relationships.

    Note:
    • This field is applicable for Detail Object Configuration while configuring partial detail records archival.
    Rule Set Name This field is not applicable.

  5. Go to Servicing Configuration > Run Batch Jobs, select Data Archival Job.

  6. Click Next, and then click Run.

    Fee payment transactions made before 60 days are archived.

Archive Detail Records Partially

One or more child objects of a parent object need to be archived based on their age. The summary record is required for some child objects, but it may not be required for others.

In CL Loan, you can configure child records archiving for the Payment, Charges, and Bills objects. To archive some of the child objects, two data archival need to be configured:

  1. With record type Master Object Configuration
  2. With record type Detail Object Configuration

Steps

To archive the detail records partially:

  1. Log in to your Salesforce account.

  2. Go to  (App Launcher) > Data Archive Configurations, and then click New.

    The New Data Archive Configuration window is displayed.

  3. To configure the data archival for the master objects, select Master Object Configuration as the record type.

  4. In the New Data Archive Configuration: Master Object Configuration window, provide the details as described in the table and the following image, and then click Save:

    Let us assume that an archive detail records partially configuration is created for the charge object as illustrated in the following image:

  5. To configure the data archival for the child objects, select Detail Object Configuration as the record type.

  6. In the New Data Archive Configuration: Detail Object Configuration window, provide the details as described in the table and the following image, and then click Save:

    Note:

    To archive other detail objects, perform steps 5 and 6.

    A loan application is approved and funded, and the following charges are created on the application:

  7. Go to Servicing Configuration > Run Batch Jobs, select Data Archival Job.

  8. Click Next, and then click Run.

    Charge records created before 180 days are archived.

Archive Detail Records Fully

One or more child objects of a parent object need to be archived irrespective of their age. Only those detail records need to be archived that correspond to master object records age criteria.

For example, detail object records for all loan applications need to be archived, which are in approved 180 days ago.

A Data Archive Configuration with record type ‘Master Object Configuration’ is required to set up this use case. For this use case, the summary record does not get created.

Steps

To archive the detail records fully:

  1. Log in to your Salesforce account.

  2. Go to  (App Launcher) > Data Archive Configurations, and then click New.

    The New Data Archive Configuration window is displayed.

  3. To configure the data archival for the master objects, select Master Object Configuration as the record type.

  4. In the New Data Archive Configuration: Master Object Configuration window, provide the details as described in the table, and then click Save.

    This automatically creates the detail object configurations for this Master Object Configuration.

  5. From the list of Detail Object Configuration, select and edit the configuration for the child object that is to be archived.

  6. Specify the following details, and then click Save:

    1. Object Name: This field is auto-populated.

    2. Parent Relationship Field Name: This field is auto-populated. If you want to provide a different Lookup field name to identify the detail records, specify the same.

    3. Select the Active checkbox to run the Archival job to process this configuration automatically.

    4. Remove the Last Processed Date Time.

      Note:
      • To archive other detail objects, perform steps 5 and 6.
      • If other Detail Object Configurations are not needed, you may delete them.

View/Delete Archived Data

You can view or delete the archived data on the Archived Data page. With the Mercury release of CL Loan, you can apply filters and fetch archived data for reference or accounting purposes. 

Prerequisites

None.

Steps

To view/delete the archived data:

  1. Log in to your Salesforce account.

  2. Go to  (App Launcher) > Archived Data.

    The Archived Data page is displayed with various filters such as Date Range Filters, Required Filters, Extra Filters, Fields To Be Retrieved.

  3. From the Select an Object drop down, select the object for which you want to view the archived records.

  4. In the Date Range Filters section:

    1. from the Date Field for Filtering drop down, select a date field on which you want to apply a range.

    2. Select From Date and To Date.

  5. In the Required Filters section:

    1. Loan Account - Specify the loan account number for which you want to view the archived data. This field is applicable for Archived Charge, Archived Loan Account Due Details, Archived Loan Payment Transaction.

    2. Loan Payment Transaction - Specify the LPT id that is appended with the URL. This field is applicable for Archived Due Payment, Archived Fee Payment.For example, in this URL: https://na123.lightning.force.com/lightning/r/loan__Loan_Payment_Transaction__c/a873t000000Tk4XAAS/view,  a873t000000Tk4XAAS is the LPT id.

  6. In the Extra Filters section, you can specify the required value.

  7. In the Fields To Be Retrieved section, select fields you want to see for the archived record.

  8. To view the data, click Get Data, or to delete the data, click Delete Data.

    You can only view the archived record for the time being. There is no option to download the records.

Sl.No. Question Answer
1 Can I run the archival job multiple times in a day?

Ideally, every data archival configuration should be run once a day. A data archival configuration field called Last Processed Date Time saves the last run timestamp and compares it with the current system timestamp. When the date matches, it does not process the configuration.

To run the archival job multiple times, the user must clear this field from the respective data archival configuration.
2 Is summary record creation mandatory? No, it is a flag-based setting. It can be disabled on-demand.
3 Is summary record creation for payment affect the loan's state? The loan's state does not change with payment insertion having the summary flag as Ture. It does not call the LPT trigger logic. 
4 Does the solution pick rejected or reversed LPTS too? Yes, the solution picks the nonarchived records.
5 Can we archive junctions?

Yes, you can, but summary creation won't be logical. 

It is recommended to go for Standalone configurations for the junctions archival, as this configuration won't support summary creation.
6 Should junctions be archived? It varies from business to business use cases. 
7 How does a summary record get created?

Among all the records picked for archival, the system takes the first record (latest Or recently created one) and clones it. 

The cloned record is then marked as summary = true, and all the aggregation(decimal values) is performed on it.
8 How many summary records get created for an object?

The configuration picks all the summary records and non-archived records older than N days, where N is a field (Archive Data Older Than) on the archival configuration.

If the records picked is having at least a summary record, it won't create a new, and updates the existing one, else create a new one.
9 Does the system create summary records for all types (Standalone, Master, Details) of configurations? A summary record can be created only for the Detail Object Configuration as of now.
10 What is the minimum number of records needed to perform archival on an object?  More than two.

In This Article