Skip to content

Flaky-test: PulsarFunctionsK8STest.testCreateFunctionInK8sWithSecretsΒ #25218

Open

Description

@lhotari

Search before reporting

  • I searched in the issues and found nothing similar.

Example failure

https://github.com/apache/pulsar/actions/runs/21718811469/job/62651627693?pr=25207#step:12:5662

Exception stacktrace

Error:  Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 126.1 s <<< FAILURE! -- in TestSuite
  Error:  org.apache.pulsar.tests.integration.functions.k8s.PulsarFunctionsK8STest.testCreateFunctionInK8sWithSecrets -- Time elapsed: 0.068 s <<< FAILURE!
  org.apache.pulsar.client.admin.PulsarAdminException: Function test-function already exists
  	at org.apache.pulsar.client.admin.PulsarAdminException.wrap(PulsarAdminException.java:252)
  	at org.apache.pulsar.client.admin.internal.BaseResource.sync(BaseResource.java:360)
  	at org.apache.pulsar.client.admin.internal.FunctionsImpl.createFunctionWithUrl(FunctionsImpl.java:199)
  	at org.apache.pulsar.tests.integration.functions.k8s.PulsarFunctionsK8STest.testCreateFunctionInK8sWithSecrets(PulsarFunctionsK8STest.java:111)
  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  	at java.base/java.lang.Thread.run(Thread.java:1583)
  	Suppressed: org.apache.pulsar.client.admin.PulsarAdminException: Function test-function already exists

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

  1. pdolif commented on Feb 8, 2026

    @pdolif
    Contributor

    If the test fails, it will be retried once. There is no cleanup of created resources between the test runs. Therefore, it is very likely that the second test run fails due to "test-function already exists" after the first run fails. The error shared in this issue is the error from the second test run. The error from the first run is not shown in the CI logs. This might be misleading. Therefore, I suggest disabling the retry for this test: #25230


    Here are some relevant parts of the logs from the GitHub Actions artifacts:

    2026-02-05T17:22:09,687 - INFO - [TestNG-method=testCreateFunctionInK8sWithSecrets-1:PulsarFunctionsK8STest@140] - Testing function 
    2026-02-05T17:22:09,991 - INFO - [pulsar-client-io-8-3:ConnectionPool@298] - [[id: 0x46d64a13, L:/127.0.0.1:49994 - R:localhost/127.0.0.1:32771]] Connected to server 
    2026-02-05T17:22:10,094 - INFO - [pulsar-client-lookup-21-1:ConsumerStatsRecorderImpl@118] - Starting Pulsar consumer status recorder with config: {"topicNames":["persistent://public/default/test-function-output-..."],...} 
    2026-02-05T17:22:10,108 - INFO - [pulsar-client-lookup-21-1:ConsumerStatsRecorderImpl@119] - Pulsar client config: {"serviceUrl":"pulsar://localhost:32771",...} 
    2026-02-05T17:22:10,129 - INFO - [pulsar-client-io-8-3:ConsumerImpl@870] - [persistent://public/default/test-function-output-...][sub] Subscribing to topic on cnx [id: 0x46d64a13, L:/127.0.0.1:49994 - R:localhost/127.0.0.1:32771], consumerId 0 
    2026-02-05T17:22:10,173 - INFO - [pulsar-client-io-8-3:ConsumerImpl@1044] - [persistent://public/default/test-function-output-...][sub] Subscribed to topic on localhost/127.0.0.1:32771 -- consumer: 0 
    2026-02-05T17:22:10,221 - INFO - [pulsar-client-io-8-3:ProducerStatsRecorderImpl@105] - Starting Pulsar producer perf with config: {"topicName":"persistent://public/default/test-function-input-...",...} 
    2026-02-05T17:22:10,221 - INFO - [pulsar-client-io-8-3:ProducerStatsRecorderImpl@106] - Pulsar client config: {"serviceUrl":"pulsar://localhost:32771",...} 
    2026-02-05T17:22:10,225 - INFO - [pulsar-client-io-8-3:ProducerImpl@1903] - [persistent://public/default/test-function-input-...] [null] Creating producer on cnx [id: 0x46d64a13, L:/127.0.0.1:49994 - R:localhost/127.0.0.1:32771] 
    2026-02-05T17:23:10,033 - INFO - [pulsar-client-io-8-4:ProducerImpl@2033] - [persistent://public/default/test-function-input-...] [null] Temporary error in creating producer: request timeout {'durationMs': '30000', 'reqId':'3512025844353319531', 'remote':'localhost/127.0.0.1:32771', 'local':'/127.0.0.1:49994'} 
    2026-02-05T17:23:10,035 - WARN - [pulsar-client-io-8-4:ClientCnx@1476] - [id: 0x46d64a13, L:/127.0.0.1:49994 - R:localhost/127.0.0.1:32771] request timeout {'durationMs': '30000', 'reqId':'3512025844353319531', 'remote':'localhost/127.0.0.1:32771', 'local':'/127.0.0.1:49994'} 
    2026-02-05T17:23:10,037 - INFO - [pulsar-client-io-8-3:ConsumerImpl@1229] - [persistent://public/default/test-function-output-...] [sub] Closed consumer 
    2026-02-05T17:23:10,037 - INFO - [TestNG-method=testCreateFunctionInK8sWithSecrets-1:PulsarClientImpl@864] - Client closing. URL: pulsar://localhost:32771 
    2026-02-05T17:23:10,042 - INFO - [pulsar-client-io-8-3:ClientCnx@332] - [id: 0x46d64a13, L:/127.0.0.1:49994 ! R:localhost/127.0.0.1:32771] Disconnected
    

    Pulsar standalone logs:

    2026-02-05T17:22:09,535+0000 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO  org.apache.pulsar.broker.service.ServerCnx - [/10.42.0.6:59546] Created subscription on topic persistent://public/default/test-function-input-... / public/default/test-function
    2026-02-05T17:22:09,563+0000 [pulsar-web-53-1] INFO  org.eclipse.jetty.server.RequestLog - 10.42.0.1 - - [05/Feb/2026:17:22:09 +0000] "GET /admin/v2/persistent/public/default/test-function-input-.../subscriptions HTTP/1.1" 200 52 "" "Pulsar-Java-v4.2.0-SNAPSHOT" 3
    2026-02-05T17:22:10,004+0000 [pulsar-io-20-4] INFO  org.apache.pulsar.broker.service.ServerCnx - [/10.42.0.1:28761] connected with clientVersion=Pulsar-Java-v4.2.0-SNAPSHOT, clientProtocolVersion=21, proxyVersion=null
    2026-02-05T17:22:10,139+0000 [pulsar-io-20-4] INFO  org.apache.pulsar.broker.service.ServerCnx - [[id: 0x183f927a, L:/10.42.0.5:16650 - R:/10.42.0.1:28761] [SR:10.42.0.1, state:Connected]] Subscribing on topic persistent://public/default/test-function-output-... / sub. consumerId: 0, role: null
    2026-02-05T17:22:10,170+0000 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/test-function-output-...] Cursor sub recovered to position 9:-1
    2026-02-05T17:22:10,171+0000 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/test-function-output-...] Opened new cursor: ManagedCursorImpl{ledger=public/default/persistent/test-function-output-..., name=sub, ackPos=9:-1, readPos=9:0}
    2026-02-05T17:22:10,172+0000 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/test-function-output-...-sub] Rewind from 9:0 to 9:0
    2026-02-05T17:22:10,172+0000 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentSubscription - backlog for persistent://public/default/test-function-output-... - 0
    2026-02-05T17:22:10,172+0000 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.pulsar.broker.service.ServerCnx - [/10.42.0.1:28761] Created subscription on topic persistent://public/default/test-function-output-... / sub
    2026-02-05T17:22:10,232+0000 [pulsar-io-20-4] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleting path: /ledgers/idgen/ID-13 (v. Optional.empty)
    2026-02-05T17:22:10,233+0000 [pulsar-io-20-4] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleted path: /ledgers/idgen/ID-13 (v. Optional.empty)
    2026-02-05T17:22:10,234+0000 [pulsar-io-20-4] INFO  org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [10.42.0.5:36313] for ledger: 13
    2026-02-05T17:22:10,238+0000 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleting path: /ledgers/idgen/ID-14 (v. Optional.empty)
    2026-02-05T17:22:10,238+0000 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleted path: /ledgers/idgen/ID-14 (v. Optional.empty)
    2026-02-05T17:22:10,239+0000 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO  org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [10.42.0.5:36313] for ledger: 14
    2026-02-05T17:22:10,244+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleting path: /ledgers/idgen/ID-15 (v. Optional.empty)
    2026-02-05T17:22:10,245+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleted path: /ledgers/idgen/ID-15 (v. Optional.empty)
    ...
    2026-02-05T17:22:30,844+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleting path: /ledgers/idgen/ID-6767 (v. Optional.empty)
    2026-02-05T17:22:30,845+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleted path: /ledgers/idgen/ID-6767 (v. Optional.empty)
    2026-02-05T17:22:30,845+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [10.42.0.5:36313] for ledger: 6767
    2026-02-05T17:22:30,847+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleting path: /ledgers/idgen/ID-6768 (v. Optional.empty)
    2026-02-05T17:22:30,847+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.pulsar.metadata.impl.AbstractMetadataStore - Deleted path: /ledgers/idgen/ID-6768 (v. Optional.empty)
    2026-02-05T17:22:30,848+0000 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO  org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [10.42.0.5:36313] for ledger: 6768
    

    I was not yet able to reproduce this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions