-
Type:
Bug
-
Status: Closed
-
Priority:
Critical
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: BenAdmin
-
Labels:None
-
Environment:Production
-
Bug Severity:Medium
-
Module:Platform
-
Reported by:Client
-
Company:All Clients/Multiple Clients
-
Item State:LB QA - LB Deployed
-
Reported by Customer:BCC and Shreveport
I am getting calls regarding both stage and production
1. Stage - running very slow. The system is displaying the loader for long time
2. Production - Kicking the user out randomly. I experienced this in University Health. I was on an employee page clicked to search for another employee and I was kick out and had to log back in
-
- Time Spent:
- 6h
- <No comment>
-
- Time Spent:
- 6h
- <No comment>
| Field | Original Value | New Value |
|---|---|---|
| Assignee | Samir [ samir ] | Vinayak Kulkarni [ vinayak.kulkarni ] |
| Module | Parent values: BenAdmin(10100) | Parent values: Platform(10106) |
We are accessing stage site and its working fine at our end. Please let us know if there is any specific page/area getting more time to respond. Thanks.
Hi,
Below are steps by which we could able to repro log-out issue, However the behavior is not same every time.
1] Log in into system ( we used Edge browser and partner login)
2] select company and stay idle for 5 minutes
3] click on BenAdmin tile, this is when system threw me out couple of times.
| Assignee | Vinayak Kulkarni [ vinayak.kulkarni ] | Satya [ ID10004 ] |
-
- Time Spent:
- 6h
- <No comment>
Hi Rakesh,
As discussed, can you please get this verified on multiple browsers & login. Please share your findings.
Regards,
Satya Prakash
| Assignee | Satya [ ID10004 ] | Rakesh Roy [ rakeshr ] |
Sanjana Jadhav Could you please check log out behaviour of system on production which is changed after Hi trust recommendation?
Mainy target Edge , IE and Chrome browser.
| Assignee | Rakesh Roy [ rakeshr ] | Sanjana Jadhav [ sanjana.jadhav ] |
Hi Satya
Verified on Chrome, IE 10 and Edge.
Verified for following login:partner, Broker, admin and employee
Verified stage and production behaviour.
1.stage response time is more than production but it does not take too much time to respond
2.Not able to repro logout issue-follow same step mentioned in above comment .(Verified same for partner, Broker, admin and employee login)
3.Verified session timeout - session expired after 15 minute - need to login again after session timeout -> Pass
CC: Rakesh Roy Prasad Pise Hrishikesh Deshpande Sachin Hingole Samir Vinayak Kulkarni Jaideep Vinchurkar Bharti Satpute Saurabh Sablaka
Thanks,
Sanjana Jadhav
In Addition to above points, development team also verified below points on production today, however did not find any logout issue for the same:
1. Verified patch points related to session on all web servers
2. Access system from multiple login roles
3. Keep system in idle state for 5/10/15 mins on Chrome, Edge & IE
4. Access site with individual IP address and keep in idle state
5. Verified session generation and user flag reset from backend
6. Checked that requests are distributed across all servers in load test of 200 users
After this we tried logging in with 10-11 partner users by putting load of 500 users on Production. In this scenario also, couldn’t reproduce this. We are checking for more feasibility for this logout issue to occur- however in case any additional details or steps, please share with us. It will help fix this issue on priority.
Rakesh Roy - Please schedule call with Samir to get repro steps.
Regards,
Satya Prakash
Cc: Swapnil Pandhare Saurabh Sablaka Ashwin Wankhede Bharti Satpute Samir
-
- Time Spent:
- 9h
-
Session logout issue analysis, discussion, debugging , resolution
-
- Time Spent:
- 4h
-
verified log out issue on production using parner, broker, admin and employee login and slowness issue on production
Roshan Deshmukh (Inactive)
logged work - 12/Oct/17 12:23 PM -
- Time Spent:
- 8.5h
-
logout issue analysis, discussion
-
- Time Spent:
- 6h
- <No comment>
-
- Time Spent:
- 1h
-
Verified below scanerios on production:
1.Login->company home->Benadmin->Search employee->Employee home->wait for 5 min->click on search employee breadcrumbs
2.Login->Company home->Wait for 10 min->perform some action
3.Login->Wait for 15 min->after 15 min popup occurAll scenarios mentioned above working properly.
| Remaining Estimate | 0h [ 0 ] | |
| Time Spent | 1h [ 3600 ] | |
| Worklog Id | 84676 [ 84676 ] |
-
- Time Spent:
- 9h
-
Session logout issue analysis, discussion, debugging , resolution
| Time Spent | 1h [ 3600 ] | 5h [ 18000 ] |
| Worklog Id | 84705 [ 84705 ] |
Roshan Deshmukh (Inactive)
made changes -
| Status | Open [ 1 ] | In Development [ 10007 ] |
| Time Spent | 5h [ 18000 ] | 5.3h [ 19080 ] |
| Worklog Id | 84995 [ 84995 ] |
| Time Spent | 5.3h [ 19080 ] | 8.8h [ 31680 ] |
| Worklog Id | 85028 [ 85028 ] |
-
- Time Spent:
- 6h
- <No comment>
-
- Time Spent:
- 9h
-
Session logout issue analysis, discussion, debugging , resolution
Khandu Kshirsagar (Inactive)
made changes -
| Item State | Parent values: Production QA(10203)Level 1 values: Production Deployed(10221) | |
| Original Estimate | 0h [ 0 ] |
| Status | In Development [ 10007 ] | Local Testing [ 10200 ] |
| Status | Local Testing [ 10200 ] | Stage Testing [ 10201 ] |
| Status | Stage Testing [ 10201 ] | Production Testing [ 10202 ] |
| Assignee | Sanjana Jadhav [ sanjana.jadhav ] | Debbie Kulling [ dkulling ] |
| Resolution | Fixed [ 1 ] | |
| Status | Production Testing [ 10202 ] | Production Complete [ 10028 ] |
| Item State | Parent values: Production QA(10203)Level 1 values: Production Deployed(10221) | Parent values: Production Complete(10222)Level 1 values: Closed(10223) |
-
- Time Spent:
- 2h
-
Session issue - patch and log analysis
-
- Time Spent:
- 7h
- <No comment>
| Time Spent | 8.8h [ 31680 ] | 17.8h [ 64080 ] |
| Worklog Id | 85357 [ 85357 ] |
| Time Spent | 17.8h [ 64080 ] | 26.8h [ 96480 ] |
| Worklog Id | 85358 [ 85358 ] |
| Time Spent | 26.8h [ 96480 ] | 35.8h [ 128880 ] |
| Worklog Id | 85359 [ 85359 ] |
| Time Spent | 35.8h [ 128880 ] | 37.8h [ 136080 ] |
| Worklog Id | 85360 [ 85360 ] |
| Time Spent | 37.8h [ 136080 ] | 43.8h [ 157680 ] |
| Worklog Id | 85723 [ 85723 ] |
| Time Spent | 43.8h [ 157680 ] | 49.8h [ 179280 ] |
| Worklog Id | 85725 [ 85725 ] |
| Time Spent | 49.8h [ 179280 ] | 56.8h [ 204480 ] |
| Worklog Id | 85726 [ 85726 ] |
| Time Spent | 56.8h [ 204480 ] | 62.8h [ 226080 ] |
| Worklog Id | 85727 [ 85727 ] |
| Time Spent | 62.8h [ 226080 ] | 68.8h [ 247680 ] |
| Worklog Id | 85728 [ 85728 ] |
| Time Spent | 68.8h [ 247680 ] | 74.8h [ 269280 ] |
| Worklog Id | 85729 [ 85729 ] |
| Time Spent | 74.8h [ 269280 ] | 75.8h [ 272880 ] |
| Worklog Id | 85734 [ 85734 ] |
-
- Time Spent:
- 1h
-
stage check in
-
- Time Spent:
- 6h
- <No comment>
-
- Time Spent:
- 8h
- <No comment>
-
- Time Spent:
- 8h
- <No comment>
-
- Time Spent:
- 6h
-
analysis session logout and root cause
Roshan Deshmukh (Inactive)
made changes -
| Time Spent | 75.8h [ 272880 ] | 84.3h [ 303480 ] |
| Worklog Id | 86712 [ 86712 ] |
Roshan Deshmukh (Inactive)
made changes -
| Time Spent | 84.3h [ 303480 ] | 90.3h [ 325080 ] |
| Worklog Id | 86832 [ 86832 ] |
Roshan Deshmukh (Inactive)
made changes -
| Worklog Id | 86832 [ 86832 ] |
-
- Time Spent:
- 8h
- <No comment>
Khandu Kshirsagar (Inactive)
made changes -
| Item State | Parent values: Production Complete(10222)Level 1 values: Closed(10223) | Parent values: LB QA(10201)Level 1 values: LB Deployed(11600) |
| Time Spent | 90.3h [ 325080 ] | 94.3h [ 339480 ] |
| Worklog Id | 87886 [ 87886 ] |
| Time Spent | 94.3h [ 339480 ] | 102.3h [ 368280 ] |
| Worklog Id | 88067 [ 88067 ] |
| Time Spent | 102.3h [ 368280 ] | 110.3h [ 397080 ] |
| Worklog Id | 88068 [ 88068 ] |
| Time Spent | 110.3h [ 397080 ] | 118.3h [ 425880 ] |
| Worklog Id | 88069 [ 88069 ] |
| Time Spent | 118.3h [ 425880 ] | 124.3h [ 447480 ] |
| Worklog Id | 88070 [ 88070 ] |
Below are some more information on Root cause:
Root Cause:
User’s last session data/details could not be fetched from System utilization table and which forced the user to be logged out of the system
Reason 1:
In User’s table “Isactive” flag was set to 1, however, user was not logged into the system, this entry could have occurred due to an unexpected event of cluster failure or “system recycle” (workaround to reduce the system overload/utilization) resulting in “Isactive” flag was not being reset to 0
Reason 2:
2016 system utilization table data were deleted from main table (routine performed for monitoring 2017 system utilization), due to which last session ID of the active users could not be fetched and resulted in killing of all present user session
| Company | All Clients/Multiple Clients [ 18434 ] | |
| Environment | Production [ 10005 ] | |
| Environment_New | Production [ 18442 ] |
Satya Thanks for this update - are we saying that the root cause was both of these? Or are we trying to determine which one, and these are theories as to what may have happened?
Jennifer Reed (Inactive)
added a comment - Satya Thanks for this update - are we saying that the root cause was both of these? Or are we trying to determine which one, and these are theories as to what may have happened? Hi Nandu,
Can you please share your views on Jenn's query?
Regards,
Satya Prakash
| Assignee | Debbie Kulling [ dkulling ] | Nandkumar Prabhakar Karlekar [ satyap ] |
| Root Cause | Configuration Error [ 18427 ] |
Jennifer Reed (Inactive)
added a comment - Samir Satya Have we made any progress on the root causes? | Attachment | User Kick out Issue Analysis.docx [ 68663 ] |
Hi Jennifer Reed,
We have completely fixed this issue and deployed at all environment . This is very specific problem hidden in user login related data.
Please find the attached analysis document for detail.
User Kick out Issue Analysis.docx
Please let me know if you need more information for the same.
Thanks
Nandkumar
-
- Time Spent:
- 0.75h
- <No comment>
| Time Spent | 124.3h [ 447480 ] | 125.05h [ 450180 ] |
| Worklog Id | 93125 [ 93125 ] |
| Resolution | Done [ 10000 ] | |
| Status | Production Complete [ 10028 ] | Closed [ 6 ] |
| Transition | Time In Source Status | Execution Times |
|---|
Roshan Deshmukh (Inactive)
made transition
-
|
2d 16h 15m | 1 |
|
3d 4h 31m | 1 |
|
3s | 1 |
|
3s | 1 |
|
29s | 1 |
|
134d 2h 4m | 1 |
Please check Stage on priority
Debbie Kulling
Production: Idle timeout is 15 minutes as per Hitrust requirement JIRA ID
WT-10727Nandkumar Prabhakar KarlekarIndraprakash Tiwari