Project SimpleProject Type: software

Project Simple

Uploaded image for project: 'Project Simple'
  1. Project Simple
  2. ST-260

Performance optimization - Export - Write header records on file

    Details

    • Type: Enhancement
    • Status: Closed
    • Priority: Critical
    • Resolution: Done
    • Component/s: Performance
    • Labels:
      None
    • Module:
      Platform - Performance
    • Reported by:
      Support
    • Item State:
      Production Complete - Closed
    • Issue Importance:
      Must Have

      Description

      Observation: During export of any template, entire text of generated file is read into system memory and written back on file along with header test. This at times creates a scenario where sufficient virtual space is not available in system memory for allocation of entire file text resulting in out of memory exception. To resolve this, we can modify this logic to read the file contents in chunks / line by line so that large memory space is not required for allocation at a time.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          saurabh.sablaka Saurabh Sablaka (Inactive) added a comment -

          Affected Files:
          App/BusinessLogic/ImportExportDTS/ACAExportProcessRowwise.vb
          App/BusinessLogic/ImportExportDTS/DemographicExportRowwise.vb
          App/BusinessLogic/ImportExportDTS/CommonExportRoutines.vb
          App/BusinessLogic/ImportExportDTS/ExecuteExportDTS.vb
          App/BusinessLogic/ImportExportDTS/ExportProcess.vb
          App/BusinessLogic/ImportExportDTS/ExportProcessRowwise.vb

          Show
          saurabh.sablaka Saurabh Sablaka (Inactive) added a comment - Affected Files: App/BusinessLogic/ImportExportDTS/ACAExportProcessRowwise.vb App/BusinessLogic/ImportExportDTS/DemographicExportRowwise.vb App/BusinessLogic/ImportExportDTS/CommonExportRoutines.vb App/BusinessLogic/ImportExportDTS/ExecuteExportDTS.vb App/BusinessLogic/ImportExportDTS/ExportProcess.vb App/BusinessLogic/ImportExportDTS/ExportProcessRowwise.vb
          Hide
          Permalink
          saurabh.sablaka Saurabh Sablaka (Inactive) added a comment -

          Affected Files:
          App/BusinessLogic/ImportExportDTS/ExecuteExportDTS.vb
          App/BusinessLogic/ImportExportDTS/ACAExportProcessRowwise.vb
          App/BusinessLogic/ImportExportDTS/CommonExportRoutines.vb
          App/BusinessLogic/ImportExportDTS/DemographicExportRowwise.vb
          App/BusinessLogic/ImportExportDTS/ExportProcess.vb
          App/BusinessLogic/ImportExportDTS/ExportProcessRowwise.vb

          Show
          saurabh.sablaka Saurabh Sablaka (Inactive) added a comment - Affected Files: App/BusinessLogic/ImportExportDTS/ExecuteExportDTS.vb App/BusinessLogic/ImportExportDTS/ACAExportProcessRowwise.vb App/BusinessLogic/ImportExportDTS/CommonExportRoutines.vb App/BusinessLogic/ImportExportDTS/DemographicExportRowwise.vb App/BusinessLogic/ImportExportDTS/ExportProcess.vb App/BusinessLogic/ImportExportDTS/ExportProcessRowwise.vb
          Hide
          Permalink
          priya.dhamande Priya Dhamande (Inactive) added a comment -

          The testing for this jira is in progress. Till now below scenarios covered:

          Row wise and Column wise Company level with Header and Without Header
          1. Fixed Length
          2. XLS
          3. Tab
          4. Pipe / Other Delimiter
          5. CSV
          6. Template Record Count count

          Observations:
          1. For Template Record count: It is appearing for only Row wise Template. No columnwise template is displaying the value for Record count.
          2. When Row wise | Fixed length Template exported | extra space is observed when Total Record count is mapped.
          3. Similarly, for Other delimiter and Tab delimiter extra pipe was observed when Total count is mapped at end.
          4. When mapped at start, for first row it appeared as expected but for other rows comma is observed.
          ex: , John Johny 123456987

          Will be continuing remaining scenarios tomorrow.

          Show
          priya.dhamande Priya Dhamande (Inactive) added a comment - The testing for this jira is in progress. Till now below scenarios covered: Row wise and Column wise Company level with Header and Without Header 1. Fixed Length 2. XLS 3. Tab 4. Pipe / Other Delimiter 5. CSV 6. Template Record Count count Observations: 1. For Template Record count: It is appearing for only Row wise Template. No columnwise template is displaying the value for Record count. 2. When Row wise | Fixed length Template exported | extra space is observed when Total Record count is mapped. 3. Similarly, for Other delimiter and Tab delimiter extra pipe was observed when Total count is mapped at end. 4. When mapped at start, for first row it appeared as expected but for other rows comma is observed. ex: , John Johny 123456987 Will be continuing remaining scenarios tomorrow.
          Hide
          Permalink
          priya.dhamande Priya Dhamande (Inactive) added a comment - - edited

          The Testing of Header and Template Record Count is completed on LB for ASML company.

          1. The enhancement is checked at company and Global level for Row wise and Column wise.
          2. This also includes testing these templates with file format ANSI file Fixed Length, XLS, Tab , Pipe / Other Delimiter, CSV and XML
          3. PGP encryption will be tested on Stage as this functionality is not working on LB.
          4. The exported file is also checked for zip protection.

          Please find the attachment of scenarios covered during testing along with their status.

          Show
          priya.dhamande Priya Dhamande (Inactive) added a comment - - edited The Testing of Header and Template Record Count is completed on LB for ASML company. 1. The enhancement is checked at company and Global level for Row wise and Column wise. 2. This also includes testing these templates with file format ANSI file Fixed Length, XLS, Tab , Pipe / Other Delimiter, CSV and XML 3. PGP encryption will be tested on Stage as this functionality is not working on LB. 4. The exported file is also checked for zip protection. Please find the attachment of scenarios covered during testing along with their status.
          Hide
          Permalink
          priya.dhamande Priya Dhamande (Inactive) added a comment - - edited

          The Testing of Header and Template Record Count is completed on Stage for ASML company. Scenarios covered:

          1. Row wise and Column wise [Fixed Length, CSV, XLS, Tab Delimiter and Other Delimiter] with Header Trailer with Template Record Count
          2. Row wise and Column wise [Fixed Length, CSV, XLS, Tab Delimiter and Other Delimiter] without Header Trailer along with Template Record Count
          3. Zip file by manual export
          4. PGP file with manual export

          Show
          priya.dhamande Priya Dhamande (Inactive) added a comment - - edited The Testing of Header and Template Record Count is completed on Stage for ASML company. Scenarios covered: 1. Row wise and Column wise [Fixed Length, CSV, XLS, Tab Delimiter and Other Delimiter] with Header Trailer with Template Record Count 2. Row wise and Column wise [Fixed Length, CSV, XLS, Tab Delimiter and Other Delimiter] without Header Trailer along with Template Record Count 3. Zip file by manual export 4. PGP file with manual export
          Hide
          Permalink
          priya.dhamande Priya Dhamande (Inactive) added a comment -

          The Testing of enhancement for optimization Write Header Trailer records is completed on Stage.
          Scenarios covered are attached with the jira.

          The enhancement is checked at Company and Global level for Row wise and Company wise templates. The file type exported was CSV, XLS, Tab delimiter, Other delimiter and Fixed Length.

          The PGP and zip format files are also exported on FTP, SFTP and Manual export.

          As all scenarios worked well, moving jira on production.

          Show
          priya.dhamande Priya Dhamande (Inactive) added a comment - The Testing of enhancement for optimization Write Header Trailer records is completed on Stage. Scenarios covered are attached with the jira. The enhancement is checked at Company and Global level for Row wise and Company wise templates. The file type exported was CSV, XLS, Tab delimiter, Other delimiter and Fixed Length. The PGP and zip format files are also exported on FTP, SFTP and Manual export. As all scenarios worked well, moving jira on production.
          Hide
          Permalink
          aniruddha.dev Aniruddha Dev (Inactive) added a comment -

          Verified Header Trailer on Demographic Rowise , Payroll Columnwise , Payroll Rowise , ANSI 834 and Group Level Templates with Scheduler on production LHP for HSPL company .

          Working as expected on Production

          Show
          aniruddha.dev Aniruddha Dev (Inactive) added a comment - Verified Header Trailer on Demographic Rowise , Payroll Columnwise , Payroll Rowise , ANSI 834 and Group Level Templates with Scheduler on production LHP for HSPL company . Working as expected on Production

            People

            Assignee:
            priya.dhamande Priya Dhamande (Inactive)
            Reporter:
            saurabh.sablaka Saurabh Sablaka (Inactive)
            Developer:
            Saurabh Sablaka (Inactive)
            QA:
            Aniruddha Dev (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Dev Due Date:
              Code Review Date:

                Time Tracking

                Estimated:
                Original Estimate - 6h
                6h
                Remaining:
                Time Spent - 3h Remaining Estimate - 3h
                3h
                Logged:
                Time Spent - 3h Remaining Estimate - 3h
                3h