René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Requests are not processed and can see them stuck in requestQueue","articleBody":"version: 6.5.0\r\n\r\nHi team,\r\n\r\nWe're experiencing intermittent issue with requests processing where the requests are stuck in queue but not processed. I'll try to explain in detail about the situation and my understanding about Scribe Java lib:\r\n\r\n- Whenever requests are made using scribe java, there won't be any response back to the client and the requests stuck there for hours until restart of jvm is performed.\r\n- Scribejava has ApacheHttpClient that does actual requests to target server. This underlying httpclient has queue mechanism for processing the requests where the requests are added into a queue and processed in async manner. \r\n```\r\n   java.lang.Thread.State: WAITING (parking)\r\n\tat sun.misc.Unsafe.park(Native Method)\r\n\t- parking to wait for  \u003c0x00000000b150e370\u003e (a java.util.concurrent.CountDownLatch$Sync)\r\n\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\r\n\tat java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)\r\n\tat java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)\r\n\tat java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)\r\n\tat java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)\r\n\tat com.github.scribejava.httpclient.apache.OAuthAsyncCompletionHandler.getResult(OAuthAsyncCompletionHandler.java:91)\r\n\tat com.github.scribejava.httpclient.apache.ApacheHttpFuture.get(ApacheHttpFuture.java:37)\r\n\tat com.github.scribejava.core.httpclient.AbstractAsyncOnlyHttpClient.execute(AbstractAsyncOnlyHttpClient.java:35)\r\n\tat com.github.scribejava.core.oauth.OAuthService.execute(OAuthService.java:105)\r\n```\r\n- Looking at stacktrace from our thread dump, thread is waiting for the response from this client and we were under assumption that is could be connect timeout issues, this is what we're focusing on from last few weeks.\r\n- Therefore, we added connection timeout setting for all calls.\r\n- I can see that issue is reproducing even after applying connect timeout setting for client object. There is something else that is causing the issue for which I started analyzing heap dump for.\r\n\r\nHeap dump analysis:\r\n- From the heap dump, I can see that there are requests stuck in `requestQueue` but not picked up by ioReactor thread.\r\n![Requests-stuck-in-queue](https://github.com/scribejava/scribejava/assets/30727836/7f2628f5-6542-425c-b635-d4c89dad1f97)\r\n- InternalHttpAsyncClient 'status' is STOPPED.\r\n![InternalHttpAsyncClient-status-STOPPED](https://github.com/scribejava/scribejava/assets/30727836/f6cd115c-ca99-47ba-85da-453704b41963)\r\n- IOReactor 'status' is ACTIVE\r\n![IOReactor-status](https://github.com/scribejava/scribejava/assets/30727836/6b8f8d0b-cd43-4ba4-b1a9-14e6e3fc215c)\r\n- IOReactor execute() looks like stuck or terminated when creating dispatcher threads.\r\n![IOReactor-behavior](https://github.com/scribejava/scribejava/assets/30727836/fbad3933-e1cd-467e-83a0-a2ab02566913)\r\n\r\nI tried to understand the flow of execution, and if I understood it correctly somehow the execute() method from `AbstractMultiworkerIOReactor` got terminated at marked section below:\r\n![image](https://github.com/scribejava/scribejava/assets/30727836/46fa81c6-282a-448e-bd27-2025d75fb58d)\r\n\r\nI would like to request you to look at this issue and help me figuring out what's causing the behavior, is it a bug in code or due to performance reasons? I know it is bit older version but upgrading to latest version can fix the issue? It is not easily reproducible and happens randomly without any pattern.\r\n\r\n_Note that heap dump is collected after 2 hrs of the issue is reproduced._\r\n\r\nAppreciate your help!\r\n\r\nThanks \u0026 Regards,\r\nKarthik","author":{"url":"https://github.com/karthik-119","@type":"Person","name":"karthik-119"},"datePublished":"2023-09-11T06:15:34.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/1065/scribejava/issues/1065"}