René's URL Explorer Experiment


Title: Pubsub: Exactly once delivery duplicates messages when there is a backlog · Issue #2555 · googleapis/java-pubsub · GitHub

Open Graph Title: Pubsub: Exactly once delivery duplicates messages when there is a backlog · Issue #2555 · googleapis/java-pubsub

X Title: Pubsub: Exactly once delivery duplicates messages when there is a backlog · Issue #2555 · googleapis/java-pubsub

Description: Environment details API: pubsub OS type and version: macOS Sequoia 15.5 Java version: OpenJDK 21.0.6 (JBR-21.0.6+9-895.109-nomod) version(s): google-cloud-pubsub-1.141.2 Steps to reproduce Create a subscriber with below settings maxAckEx...

Open Graph Description: Environment details API: pubsub OS type and version: macOS Sequoia 15.5 Java version: OpenJDK 21.0.6 (JBR-21.0.6+9-895.109-nomod) version(s): google-cloud-pubsub-1.141.2 Steps to reproduce Create a...

X Description: Environment details API: pubsub OS type and version: macOS Sequoia 15.5 Java version: OpenJDK 21.0.6 (JBR-21.0.6+9-895.109-nomod) version(s): google-cloud-pubsub-1.141.2 Steps to reproduce Create a...

Opengraph URL: https://github.com/googleapis/java-pubsub/issues/2555

X: @github

direct link

Domain: patch-diff.githubusercontent.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Pubsub: Exactly once delivery duplicates messages when there is a backlog","articleBody":"#### Environment details\n\n1. API: pubsub\n2. OS type and version: macOS Sequoia 15.5\n3. Java version: OpenJDK 21.0.6 (JBR-21.0.6+9-895.109-nomod)\n4.  version(s): google-cloud-pubsub-1.141.2\n\n#### Steps to reproduce\n\n  1. Create a subscriber with below settings\n      1. maxAckExtensionPeriod: 60 mins\n      2. maxDurationPerAckExtension: 5 mins\n      3. minDurationPerAckExtension: 1 min\n      4. parallelPullCount: 2\n      5. maxOutstandingElementCount: 10\n  2. Start the subscriber and sleep for 1 - 15 mins when a message is received. Maximum 10 messages can be processed in parallel due to maxOutstandingElementCount setting.\n  3. publish 15 messages (more than MaxOutstandingElementCount)\n  4. Error \"INVALID_ARGUMENT: Some acknowledgement ids in the request were invalid. This could be because the acknowledgement ids have expired or the acknowledgement ids were malformed.\" is logged multiple times.\n  5. Duplicate messages with same message Id are delivered\n\n#### Code example\n```java// example\n    @PostConstruct\n    public void initialize() {\n        ProjectSubscriptionName subscription = ProjectSubscriptionName.of(projectId, subscriptionName);\n\n        // Create message receiver with exactly-once delivery support\n        MessageReceiverWithAckResponse receiver = new MessageReceiverWithAckResponse() {\n            @Override\n            public void receiveMessage(PubsubMessage message, AckReplyConsumerWithResponse consumer) {\n                sleepForRandomTime(message, consumer);\n            }\n        };\n\n        // Create flow control settings to limit concurrent message processing\n        FlowControlSettings flowControlSettings = FlowControlSettings.newBuilder()\n                .setMaxOutstandingElementCount(10L) // Maximum 10 messages being processed at once\n                .setMaxOutstandingRequestBytes(1000000L) // 1MB max outstanding bytes\n                .build();\n\n        // Create subscriber WITH native ack extension and enhanced monitoring\n        this.subscriber = Subscriber.newBuilder(subscription, receiver)\n                .setMaxAckExtensionPeriod(org.threeten.bp.Duration.ofMinutes(60)) // 60 minutes max extension\n                .setMaxDurationPerAckExtension(org.threeten.bp.Duration.ofMinutes(5)) // Extend by 5 min each time\n                .setMinDurationPerAckExtension(org.threeten.bp.Duration.ofMinutes(1)) // Min 1 min extension\n                .setParallelPullCount(2) // Conservative parallel pulls\n                .setFlowControlSettings(flowControlSettings) // Limit to 10 concurrent messages\n                .build();\n\n        // Start the subscriber\n        subscriber.startAsync().awaitRunning();\n}\n\nprivate void sleepForRandomTime(PubsubMessage pubsubMessage, AckReplyConsumerWithResponse consumer) {\n        String messageId = pubsubMessage.getMessageId();\n        long receiveTime = System.currentTimeMillis();\n\n        // Check for duplicate message processing\n        Long previousProcessTime = processedMessages.putIfAbsent(messageId, receiveTime);\n        if (previousProcessTime != null) {\n            logger.warn(\"DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: {} was already processed at {}. \" +\n                    \"Current receive time: {}. Acknowledging duplicate to prevent redelivery.\",\n                    messageId, new java.util.Date(previousProcessTime), new java.util.Date(receiveTime));\n\n            ApiFuture\u003cAckResponse\u003e ackFuture = consumer.ack();\n            return;\n        }\n\n        int processingTimeSeconds = random.nextInt(840) + 60; // 60 to 900 seconds (1-15 minutes)\n        logger.info(\"Received native Pub/Sub message ID: {}. Sleep for {} seconds\",\n                messageId, processingTimeSeconds);\n\n        try {\n            Thread.sleep(processingTimeSeconds * 1000L);\n\n            // Use ack() for exactly-once delivery - returns ApiFuture\u003cAckResponse\u003e\n            ApiFuture\u003cAckResponse\u003e ackFuture = consumer.ack();\n            logger.info(\"Successfully processed and acknowledged native Pub/Sub message ID: {}\",\n                    messageId);\n        } catch (Exception e) {\n            logger.error(\"Exception occurred while processing native Pub/Sub message ID: {}\", messageId, e);\n            processedMessages.remove(messageId); // Remove from processed messages on exception\n\n            // Use nack() for exactly-once delivery - returns ApiFuture\u003cAckResponse\u003e\n            ApiFuture\u003cAckResponse\u003e nackFuture = consumer.nack();\n            logger.warn(\"Exception handling: nacked native Pub/Sub message ID: {}.\", messageId);\n        }\n    }\n\n```\n\n#### Output log\nINVALID_ARGUMENT error log is omitted except the 1st one.\n\n```\n2025-09-16 17:13:08 [Gax-3] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16313525043018429. Sleep for 824 seconds\n2025-09-16 17:13:08 [Gax-4] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 15389502673124437. Sleep for 827 seconds\n2025-09-16 17:13:08 [Gax-5] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16315255480206003. Sleep for 596 seconds\n2025-09-16 17:13:08 [Gax-6] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16309022234542561. Sleep for 610 seconds\n2025-09-16 17:13:08 [Gax-7] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16311609492452975. Sleep for 446 seconds\n2025-09-16 17:13:10 [Gax-8] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16308533929060216. Sleep for 744 seconds\n2025-09-16 17:13:10 [Gax-9] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16315272577144587. Sleep for 793 seconds\n2025-09-16 17:13:10 [Gax-10] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 15393608130839305. Sleep for 526 seconds\n2025-09-16 17:13:10 [Gax-11] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16312247916552181. Sleep for 547 seconds\n2025-09-16 17:13:10 [Gax-12] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16307219734642660. Sleep for 254 seconds\n2025-09-16 17:17:24 [Gax-12] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16307219734642660\n2025-09-16 17:17:33 [Subscriber-EOD-CallbackExecutor-14] WARN  c.g.c.p.v.StreamingSubscriberConnection - failed to send operations\ncom.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Some acknowledgement ids in the request were invalid. This could be because the acknowledgement ids have expired or the acknowledgement ids were malformed.\n\tat com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:92)\n\tat com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:98)\n\tat com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)\n\tat com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)\n\tat com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)\n\tat com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)\n\tat com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)\n\tat com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1307)\n\tat com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1070)\n\tat com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:819)\n\tat io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:651)\n\tat io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:621)\n\tat io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)\n\tat io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)\n\tat io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)\n\tat com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:569)\n\tat io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)\n\tat io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)\n\tat io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)\n\tat com.google.api.gax.grpc.GrpcLoggingInterceptor$1$1.onClose(GrpcLoggingInterceptor.java:98)\n\tat io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:565)\n\tat io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72)\n\tat io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:733)\n\tat io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:714)\n\tat io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)\n\tat io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:328)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:309)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)\n\tat java.base/java.lang.Thread.run(Thread.java:1447)\nCaused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Some acknowledgement ids in the request were invalid. This could be because the acknowledgement ids have expired or the acknowledgement ids were malformed.\n\tat io.grpc.Status.asRuntimeException(Status.java:532)\n\t... 21 common frames omitted\n2025-09-16 17:20:34 [Gax-7] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16311609492452975\n2025-09-16 17:20:34 [Gax-12] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16308461489469056. Sleep for 815 seconds\n2025-09-16 17:20:34 [Gax-7] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16307308491278046. Sleep for 430 seconds\n2025-09-16 17:21:56 [Gax-10] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 15393608130839305\n2025-09-16 17:21:56 [Subscriber-EOD-CallbackExecutor-15] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:22:17 [Gax-11] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16312247916552181\n2025-09-16 17:22:17 [Gax-10] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16309022239040559. Sleep for 734 seconds\n2025-09-16 17:22:17 [Gax-11] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 15389052601735206. Sleep for 688 seconds\n2025-09-16 17:23:04 [Gax-5] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16315255480206003\n2025-09-16 17:23:04 [Subscriber-EOD-CallbackExecutor-16] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:04 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:04 [Gax-5] INFO  c.e.p.s.NativeMessageSubscriber - Received native Pub/Sub message ID: 16312609047030287. Sleep for 350 seconds\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-16] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-16] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-16] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:05 [Subscriber-EOD-CallbackExecutor-16] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:23:18 [Gax-6] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16309022234542561\n2025-09-16 17:23:18 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16308533929060216 was already processed at Tue Sep 16 17:13:10 JST 2025. Current receive time: Tue Sep 16 17:23:18 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:18 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16315255480206003 was already processed at Tue Sep 16 17:13:08 JST 2025. Current receive time: Tue Sep 16 17:23:18 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:19 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 15393608130839305 was already processed at Tue Sep 16 17:13:10 JST 2025. Current receive time: Tue Sep 16 17:23:19 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:19 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16309022234542561 was already processed at Tue Sep 16 17:13:08 JST 2025. Current receive time: Tue Sep 16 17:23:19 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:19 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16312609047030287 was already processed at Tue Sep 16 17:23:04 JST 2025. Current receive time: Tue Sep 16 17:23:19 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:19 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16309022239040559 was already processed at Tue Sep 16 17:22:17 JST 2025. Current receive time: Tue Sep 16 17:23:19 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:23:19 [Gax-6] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16307219734642660 was already processed at Tue Sep 16 17:13:10 JST 2025. Current receive time: Tue Sep 16 17:23:19 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16308533929060216\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 15389052601735206 was already processed at Tue Sep 16 17:22:17 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16315272577144587 was already processed at Tue Sep 16 17:13:10 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 15389502673124437 was already processed at Tue Sep 16 17:13:08 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16313525043018429 was already processed at Tue Sep 16 17:13:08 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16311609492452975 was already processed at Tue Sep 16 17:13:08 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16307308491278046 was already processed at Tue Sep 16 17:20:34 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16308461489469056 was already processed at Tue Sep 16 17:20:34 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 15389052601735206 was already processed at Tue Sep 16 17:22:17 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:25:34 [Gax-8] WARN  c.e.p.s.NativeMessageSubscriber - DUPLICATE MESSAGE DETECTED! Pub/Sub message ID: 16312247916552181 was already processed at Tue Sep 16 17:13:10 JST 2025. Current receive time: Tue Sep 16 17:25:34 JST 2025. Acknowledging duplicate to prevent redelivery.\n2025-09-16 17:26:23 [Gax-9] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16315272577144587\n2025-09-16 17:26:39 [Subscriber-EOD-CallbackExecutor-14] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n2025-09-16 17:26:52 [Gax-3] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16313525043018429\n2025-09-16 17:26:55 [Gax-4] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 15389502673124437\n2025-09-16 17:27:44 [Gax-7] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16307308491278046\n2025-09-16 17:28:54 [Gax-5] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16312609047030287\n2025-09-16 17:33:45 [Gax-11] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 15389052601735206\n2025-09-16 17:33:46 [Subscriber-EOD-CallbackExecutor-19] WARN  c.g.c.p.v.StreamingSubscriberConnection - failed to send operations\n2025-09-16 17:34:09 [Gax-12] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16308461489469056\n2025-09-16 17:34:31 [Gax-10] INFO  c.e.p.s.NativeMessageSubscriber - Successfully processed and acknowledged native Pub/Sub message ID: 16309022239040559\n2025-09-16 17:34:32 [Subscriber-EOD-CallbackExecutor-19] INFO  c.g.c.p.v.StreamingSubscriberConnection - Permanent error invalid ack id message, will not resend\n```\n\n#### Any additional information below\nThe duplicate message issue occurs only when there is a backlog. When I publish messages less than MaxOutstandingElementCount, this issue seems not happening.\n\n\nThanks!\n","author":{"url":"https://github.com/IkueWatanabe-Aidea","@type":"Person","name":"IkueWatanabe-Aidea"},"datePublished":"2025-09-16T08:58:17.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/2555/java-pubsub/issues/2555"}

route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:5ed8a992-408f-d8ca-4553-923804439843
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idEA86:1F9D56:2E6AC9:3ABE40:69909D0A
html-safe-nonce084b7e57423ef1f92227105a3523b4fbb689f63eb0c812dcbaeff18f3681c7c3
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQTg2OjFGOUQ1NjoyRTZBQzk6M0FCRTQwOjY5OTA5RDBBIiwidmlzaXRvcl9pZCI6IjE2MDQ1OTM0Mzk4ODk5ODg4NzQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac223e520c0ecf9df52226a997406f4b84584a80edb6c1cfdeda4344b57aaf1738
hovercard-subject-tagissue:3421207043
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/java-pubsub/2555/issue_layout
twitter:imagehttps://opengraph.githubassets.com/8958ccb49a54e16f610ec2c053b29398973421a2e0831ebf09b52ce8e99b6a45/googleapis/java-pubsub/issues/2555
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/8958ccb49a54e16f610ec2c053b29398973421a2e0831ebf09b52ce8e99b6a45/googleapis/java-pubsub/issues/2555
og:image:altEnvironment details API: pubsub OS type and version: macOS Sequoia 15.5 Java version: OpenJDK 21.0.6 (JBR-21.0.6+9-895.109-nomod) version(s): google-cloud-pubsub-1.141.2 Steps to reproduce Create a...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameIkueWatanabe-Aidea
hostnamegithub.com
expected-hostnamegithub.com
None42c603b9d642c4a9065a51770f75e5e27132fef0e858607f5c9cb7e422831a7b
turbo-cache-controlno-preview
go-importgithub.com/googleapis/java-pubsub git https://github.com/googleapis/java-pubsub.git
octolytics-dimension-user_id16785467
octolytics-dimension-user_logingoogleapis
octolytics-dimension-repository_id203461480
octolytics-dimension-repository_nwogoogleapis/java-pubsub
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id203461480
octolytics-dimension-repository_network_root_nwogoogleapis/java-pubsub
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release3b33c5aedc9808f45bc5fcf0b1e4404cf749dac7
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues/2555#start-of-content
https://patch-diff.githubusercontent.com/
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fjava-pubsub%2Fissues%2F2555
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fjava-pubsub%2Fissues%2F2555
Sign up https://patch-diff.githubusercontent.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=googleapis%2Fjava-pubsub
Reloadhttps://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues/2555
Reloadhttps://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues/2555
Reloadhttps://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues/2555
googleapis https://patch-diff.githubusercontent.com/googleapis
java-pubsubhttps://patch-diff.githubusercontent.com/googleapis/java-pubsub
Notifications https://patch-diff.githubusercontent.com/login?return_to=%2Fgoogleapis%2Fjava-pubsub
Fork 101 https://patch-diff.githubusercontent.com/login?return_to=%2Fgoogleapis%2Fjava-pubsub
Star 147 https://patch-diff.githubusercontent.com/login?return_to=%2Fgoogleapis%2Fjava-pubsub
Code https://patch-diff.githubusercontent.com/googleapis/java-pubsub
Issues 16 https://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues
Pull requests 26 https://patch-diff.githubusercontent.com/googleapis/java-pubsub/pulls
Actions https://patch-diff.githubusercontent.com/googleapis/java-pubsub/actions
Security 0 https://patch-diff.githubusercontent.com/googleapis/java-pubsub/security
Insights https://patch-diff.githubusercontent.com/googleapis/java-pubsub/pulse
Code https://patch-diff.githubusercontent.com/googleapis/java-pubsub
Issues https://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues
Pull requests https://patch-diff.githubusercontent.com/googleapis/java-pubsub/pulls
Actions https://patch-diff.githubusercontent.com/googleapis/java-pubsub/actions
Security https://patch-diff.githubusercontent.com/googleapis/java-pubsub/security
Insights https://patch-diff.githubusercontent.com/googleapis/java-pubsub/pulse
New issuehttps://patch-diff.githubusercontent.com/login?return_to=https://github.com/googleapis/java-pubsub/issues/2555
New issuehttps://patch-diff.githubusercontent.com/login?return_to=https://github.com/googleapis/java-pubsub/issues/2555
Pubsub: Exactly once delivery duplicates messages when there is a backloghttps://patch-diff.githubusercontent.com/googleapis/java-pubsub/issues/2555#top
https://patch-diff.githubusercontent.com/michaelpri10
api: pubsubIssues related to the googleapis/java-pubsub API.https://github.com/googleapis/java-pubsub/issues?q=state%3Aopen%20label%3A%22api%3A%20pubsub%22
https://github.com/IkueWatanabe-Aidea
https://github.com/IkueWatanabe-Aidea
IkueWatanabe-Aideahttps://github.com/IkueWatanabe-Aidea
on Sep 16, 2025https://github.com/googleapis/java-pubsub/issues/2555#issue-3421207043
michaelpri10https://patch-diff.githubusercontent.com/michaelpri10
api: pubsubIssues related to the googleapis/java-pubsub API.https://github.com/googleapis/java-pubsub/issues?q=state%3Aopen%20label%3A%22api%3A%20pubsub%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.