Databases

TICK Stack vs InfluxDB 3: Do Kapacitor and Chronograf Still Work?

For years the TICK stack rested on one quiet assumption about its HTTP API. Kapacitor and Chronograf talk to InfluxDB over the 1.x endpoints, so any server that speaks them can sit in the middle. InfluxDB 3 Core keeps the v1 /write and /query endpoints, which makes it tempting to swap the database and keep the rest. The question behind every Kapacitor InfluxDB 3 search is whether that assumption still holds.

Original content from computingforgeeks.com - post 188893

It holds for half the stack. Telegraf writes, InfluxQL reads, Chronograf in its new v3 mode and Kapacitor batch tasks all worked in our lab. Stream tasks went silent, because InfluxDB 3 has no subscriptions to push points to Kapacitor. Everything below was measured in November 2026 on one Ubuntu 24.04 VM, with InfluxDB 1.x and InfluxDB 3 Core fed by the same Telegraf agent so every query could be run against both.

We show the exact errors each piece throws, the config that gets past them, and what InfluxData now offers in their place: a Processing Engine plugin for alerting, Explorer for queries and Grafana for dashboards.

What we tested, and against which builds

The lab ran every TICK component from the InfluxData apt repository on a single host called node-a, next to InfluxDB 3 Core listening on port 8181. Keeping the old server running mattered: when a statement failed on InfluxDB 3, we could send the identical request to the 1.x server and confirm the failure was the new engine and not our syntax.

ComponentVersion testedRole in the lab
InfluxDB 3 Core3.12.2Target server, port 8181, token auth
InfluxDB OSS1.13.2Control server, port 8086
Kapacitor1.8.9One stream task, one batch task
Chronograf1.11.7InfluxQL UI pointed at InfluxDB 3
Telegraf1.40.3Writes cpu, mem, disk and system to both servers
InfluxDB 3 Explorer1.11.2 (Docker image)SQL UI for InfluxDB 3
Grafana13.3.0Dashboards over SQL
OSUbuntu 24.04.5 LTSSingle VM, 2 vCPU

All five InfluxData packages come from one repository, so the install is short. If you’re building the 1.x side from scratch, the TICK stack install on Ubuntu walks through each service in more detail than we need here.

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://repos.influxdata.com/influxdata-archive.key | gpg --dearmor | sudo tee /etc/apt/keyrings/influxdata-archive.gpg > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main" | sudo tee /etc/apt/sources.list.d/influxdata.list
sudo apt-get update
sudo apt-get install -y influxdb chronograf kapacitor telegraf influxdb3-core

The influxdb3-core package writes its settings to /etc/influxdb3/influxdb3-core.conf. These are the defaults it shipped with, and the plugin directory on the last line is where the Kapacitor replacement lives later:

node-id="primary-node"
object-store="file"
data-dir="/var/lib/influxdb3/data"
plugin-dir="/var/lib/influxdb3/plugins"

InfluxDB 3 Core refuses unauthenticated requests, so the first job after starting it is an admin token. Give the service a few seconds before asking: our first attempt ran too early and failed with Failed to create token, error: RequestSend and a tcp connect error, simply because port 8181 was not listening yet.

sudo systemctl enable --now influxdb3-core
influxdb3 create token --admin

The token is printed once, as the output itself warns. We masked it in this capture, and you should copy yours somewhere safe before the terminal scrolls, because every client in the rest of this lab authenticates with it:

New token created successfully!

Token: apiv3_REDACTED
HTTP Requests Header: Authorization: Bearer apiv3_REDACTED

IMPORTANT: Store this token securely, as it will not be shown again.

Every later step reads the token from one file, so paste it into ~/.v3token with an editor, lock the file down, and export it for the influxdb3 CLI in the current shell:

vim ~/.v3token
chmod 600 ~/.v3token
export INFLUXDB3_AUTH_TOKEN=$(cat ~/.v3token)

With the token in place, the diagram below is the whole lab. Solid arrows are paths that worked as configured. The dashed Telegraf to Kapacitor arrow is the workaround stream tasks need, and the dashed alerts arrow is the Processing Engine, which runs inside the InfluxDB 3 process and is drawn apart only to show its role as the native alerting path.

Architecture diagram of 1.x TICK components (Telegraf, Kapacitor, Chronograf) pointed at InfluxDB 3 Core, with the Processing Engine, Explorer and Grafana as replacements

Every arrow into the InfluxDB 3 box carries the same admin token. In production you would mint narrower tokens per client, but one token kept the lab focused on protocol behaviour rather than permissions.

How InfluxDB 3 Core answers a 1.x client

InfluxDB 3 Core answers 1.x clients through compatibility endpoints that accept the old request shapes but sit on a different engine. Writes go to /write as line protocol and land in a database of the same name, created on first write. Reads go to /query as InfluxQL, which the server parses and runs on its SQL engine. Anything that was really an admin operation on 1.x has no equivalent on the other side.

Telegraf writes to both servers without a code change

Telegraf’s existing influxdb output works against port 8181 when the token goes in the password field. This is the final config we ran, replacing the outputs section of the package’s /etc/telegraf/telegraf.conf. Leave out the third block, the one for port 9092, until you have dropped Kapacitor’s 1.x subscription as shown in the Kapacitor section; with both in place, stream tasks receive every point twice. The full input side follows the pattern in our Telegraf input plugins guide:

[agent]
  interval = "10s"
  flush_interval = "10s"
  hostname = "node-a"

[[inputs.cpu]]
  percpu = false
  totalcpu = true
[[inputs.mem]]
[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "overlay", "squashfs"]
[[inputs.system]]

# InfluxDB 1.13 on the classic TICK port
[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "telegraf"

# InfluxDB 3 Core through its v1-compatible /write endpoint
[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8181"]
  skip_database_creation = true
  database = "telegraf"
  password = "${INFLUX_V3_TOKEN}"

# Kapacitor stream tasks: no subscriptions on InfluxDB 3, so feed Kapacitor directly
[[outputs.influxdb]]
  urls = ["http://127.0.0.1:9092"]
  database = "telegraf"
  retention_policy = "autogen"
  skip_database_creation = true

The token reaches Telegraf through the systemd environment file rather than the config, which keeps it out of anything you might paste into a ticket:

echo "INFLUX_V3_TOKEN=$(cat ~/.v3token)" | sudo tee /etc/default/telegraf > /dev/null
sudo chmod 640 /etc/default/telegraf
sudo systemctl restart telegraf

Our first start ran without skip_database_creation, and Telegraf logged this warning on every restart. Telegraf sends a CREATE DATABASE through /query on startup without a db parameter, and InfluxDB 3 rejects the request before it even looks at the statement, as the error text says. Writes still landed, but the noise hides real errors, so set the flag:

W! [outputs.influxdb] When writing to [http://127.0.0.1:8181]: database "telegraf" creation failed: 200 OK: must specify a 'db' parameter, or provide the database in the InfluxQL query

After the restart, both servers held the same three rows within one flush interval, which confirms the v1 write path is a faithful copy and not a lossy shim:

== InfluxDB 1.13
name: cpu
time count
---- -----
0    3
== InfluxDB 3 Core
+---------------+
| iox::database |
+---------------+
| _internal     |
| telegraf      |
+---------------+
+------+---------------------+
| rows | newest              |
+------+---------------------+
| 3    | 2026-11-12T06:30:30 |
+------+---------------------+

Notice that InfluxDB 3 created the telegraf database on its own when the first batch arrived. That’s why the failed CREATE DATABASE warning didn’t cost us any data: the write path doesn’t need the database to exist beforehand.

Which InfluxQL statements still work

We sent 20 InfluxQL statements to both servers with curl, one request per statement per port, and compared answers. The split is clean: anything that reads data or schema works, and anything that manages the server fails. Every request came back HTTP 200, so tools that only check the status code will think the failures succeeded. The real error sits in the JSON body.

InfluxQL statementInfluxDB 1.xInfluxDB 3 Core
SHOW DATABASES, MEASUREMENTS, TAG KEYS, TAG VALUES, FIELD KEYSWorksWorks, identical results
SHOW RETENTION POLICIESWorksWorks, one policy named after the database
SELECT mean() GROUP BY time(), last()WorksWorks, identical values
SELECT derivative()Parses, empty resultParses, empty result (same as 1.x)
SHOW SERIES, USERS, DIAGNOSTICS, QUERIES, SUBSCRIPTIONSWorksParse error: invalid SHOW statement
CREATE SUBSCRIPTION, RETENTION POLICY, CONTINUOUS QUERYWorksParse error: expected DATABASE following CREATE
CREATE DATABASE, DROP MEASUREMENTWorksdatafusion error: This feature is not implemented
SELECT … INTOWorksParse error at pos 0

Here are the InfluxDB 3 lines from that run for the statements that matter most to the rest of the stack. The script truncates each body at 170 characters, which is why the SHOW errors end mid-word:

8181  SHOW SERIES                  HTTP 200  {"results":[{"statement_id":0,"error":"error in InfluxQL statement: parsing error: invalid SHOW statement, expected DATABASES, FIELD KEYS, MEASUREMENTS, TAG KEYS, TAG VAL
8181  SHOW SUBSCRIPTIONS           HTTP 200  {"results":[{"statement_id":0,"error":"error in InfluxQL statement: parsing error: invalid SHOW statement, expected DATABASES, FIELD KEYS, MEASUREMENTS, TAG KEYS, TAG VAL
8181  CREATE SUBSCRIPTION          HTTP 200  {"results":[{"statement_id":0,"error":"error in InfluxQL statement: parsing error: Invalid CREATE statement, expected DATABASE following CREATE at pos 7"}]}
8181  CREATE DATABASE              HTTP 200  {"results":[{"statement_id":0,"error":"datafusion error: This feature is not implemented: CREATE DATABASE"}]}
8181  CREATE CONTINUOUS QUERY      HTTP 200  {"results":[{"statement_id":0,"error":"error in InfluxQL statement: parsing error: Invalid CREATE statement, expected DATABASE following CREATE at pos 7"}]}

Two rows in that table decide the rest of this article. No SHOW SUBSCRIPTIONS or CREATE SUBSCRIPTION means Kapacitor’s push model is gone. No continuous queries means downsampling has to move somewhere else, which is the same conclusion our InfluxDB 2 to 3 migration reached for Flux tasks.

Authentication is more forgiving than the statement parser, and the /ping endpoint is less forgiving than it used to be. We tried every credential style a 1.x client might send, plus the health check probes load balancers use, and recorded the status codes:

Request to port 8181Status
/query with no credentials401, the request was not authenticated
/query with Authorization: Token200
/query with Authorization: Bearer200
/query with Basic auth, token as password200
/query with u= and p= parameters200
/query with a wrong token401
GET /ping without a token401 (1.x answers 204 to anyone)
GET /ping with a token200, JSON with product name and version
HEAD /ping with a token404

The last two rows are the ones that bite in production. If your HAProxy or Kubernetes probe hits /ping without a token, or with HEAD, it marks a healthy InfluxDB 3 backend as dead. Switch it to an authenticated GET or a TCP check before cutting over, otherwise the server gets pulled from rotation the moment it goes live. The other route is server side: InfluxDB 3 Core’s --disable-authz option accepts ping and health, which exempts those endpoints from token checks. We did not test that flag in this lab.

Why Kapacitor InfluxDB 3 stream tasks go silent

Kapacitor stream tasks go silent on InfluxDB 3 because they depend on subscriptions, and InfluxDB 3 has none. On 1.x the mechanism runs in order:

  1. Kapacitor starts and runs SHOW SUBSCRIPTIONS against every configured InfluxDB.
  2. It creates one with CREATE SUBSCRIPTION if its own is missing, pointing back at port 9092.
  3. InfluxDB copies every write to that destination, and stream tasks see each point as it arrives.
  4. Batch tasks ignore all of this and run plain InfluxQL queries on a schedule.

Steps one and two are exactly the statements InfluxDB 3 rejects. Step four only needs /query, which works. So we expected batch tasks to survive and stream tasks to die, and that is what happened, with two errors on the way that are worth knowing by sight.

The two tasks we used are deliberately simple. The threshold of 99.9% idle is absurd for production, but on an idle lab VM it guarantees alerts fire every interval, so a silent task means a broken pipeline rather than a quiet machine. If TICKscript syntax is new to you, our guide to Kapacitor TICKscript alerts walks through stream and batch tasks on 1.x. The stream task:

dbrp "telegraf"."autogen"

stream
    |from()
        .measurement('cpu')
        .where(lambda: "cpu" == 'cpu-total')
    |alert()
        .id('stream/{{ index .Tags "host" }}')
        .warn(lambda: "usage_idle" < 99.9)
        .log('/var/log/kapacitor/cpu_stream.log')

The batch task queries a one minute window every 20 seconds and writes each alert back into the database through influxDBOut, which tests the write path in the other direction:

dbrp "telegraf"."autogen"

batch
    |query('SELECT mean(usage_idle) AS idle FROM "telegraf"."autogen"."cpu" WHERE "cpu" = \'cpu-total\'')
        .period(1m)
        .every(20s)
        .groupBy('host')
    |alert()
        .id('batch/{{ index .Tags "host" }}')
        .warn(lambda: "idle" < 99.9)
        .log('/var/log/kapacitor/cpu_batch.log')
    |influxDBOut()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('kapacitor_alerts')

Both tasks were saved as cpu_stream.tick and cpu_batch.tick in the home directory, then defined and enabled the usual way:

kapacitor define cpu_stream -tick cpu_stream.tick && kapacitor enable cpu_stream
kapacitor define cpu_batch -tick cpu_batch.tick && kapacitor enable cpu_batch

Against the 1.x server both tasks fired within a minute, which gave us a baseline. Then we repointed Kapacitor at port 8181, and that’s where the errors started.

Error: failed to connect to InfluxDB, retrying… “the request was not authenticated”

Our first attempt pointed the [[influxdb]] section at port 8181 and passed the token as KAPACITOR_INFLUXDB_0_PASSWORD, leaving the username empty as the default config ships it. Curl had accepted Basic auth with a token, so this looked safe. Kapacitor never got in and retried forever:

ts=2026-11-12T06:32:51.525Z lvl=error msg="failed to connect to InfluxDB, retrying..." service=influxdb cluster=localhost err="the request was not authenticated"

Kapacitor has a dedicated token option in its InfluxDB section, and that is the one InfluxDB 3 accepts. Set it through the environment file so the token stays out of kapacitor.conf:

sudo sed -i 's#urls = \["http://localhost:8086"\]#urls = ["http://127.0.0.1:8181"]#' /etc/kapacitor/kapacitor.conf
echo "KAPACITOR_INFLUXDB_0_TOKEN=$(cat ~/.v3token)" | sudo tee /etc/default/kapacitor > /dev/null
sudo chmod 640 /etc/default/kapacitor
sudo systemctl restart kapacitor

Authentication passed with the token, and the retry loop stopped. The service still failed, though, one step later in the startup sequence, because the next thing Kapacitor does after logging in is ask InfluxDB for its subscriptions. That is the first of the four steps above, and it is where InfluxDB 3 pushes back.

Error: failed to link subscription on startup

With auth fixed, Kapacitor reached step one of the subscription sequence and exited. systemd restarted it five times and gave up with Start request repeated too quickly. The log names the exact statement InfluxDB 3 refused:

ts=2026-11-12T06:33:32.432Z lvl=error msg="encountered error" service=run err="open server: open service *influxdb.Service: failed to link subscription on startup: error in InfluxQL statement: parsing error: invalid SHOW statement, expected DATABASES, FIELD KEYS, MEASUREMENTS, TAG KEYS, TAG VALUES, or RETENTION POLICIES following SHOW at pos 5"

Kapacitor can skip subscription management entirely. Flip disable-subscriptions in the same [[influxdb]] section and the service stays up:

sudo sed -i 's/^  disable-subscriptions = false/  disable-subscriptions = true/' /etc/kapacitor/kapacitor.conf
sudo systemctl restart kapacitor
kapacitor list tasks

After about three minutes of runtime the batch task was doing its job against InfluxDB 3. The node stats below show 9 batches queried, 9 warnings and 8 points written back with zero errors, and the kapacitor_alerts rows are readable from the InfluxDB 3 CLI:

Terminal output of Kapacitor 1.8 tasks running against InfluxDB 3 Core with batch node stats and write-back rows

That covers batch alerts, scheduled rollups written back with influxDBOut, and anything else built on |query(). Teams whose TICKscripts are mostly batch can keep Kapacitor running against InfluxDB 3 with these two settings.

The stale 1.x subscription that makes stream tasks look healthy

Our stream task also reported alerts at first: 5 lines in its log in the first minute. That was a false positive. The 1.x server still held the subscription Kapacitor had created before we repointed it, so InfluxDB 1.x kept pushing copies to port 9092. During a side by side migration this will fool you exactly the way it fooled us.

Kapacitor creates its subscription under a name starting with kapacitor- on every database, _internal included. List them on the 1.x server, then drop the one Kapacitor made from both databases, replacing the name with the one your listing shows:

influx -execute "SHOW SUBSCRIPTIONS"
influx -execute 'DROP SUBSCRIPTION "kapacitor-00000000-0000-0000-0000-000000000000" ON telegraf.autogen'
influx -execute 'DROP SUBSCRIPTION "kapacitor-00000000-0000-0000-0000-000000000000" ON _internal.monitor'

With the subscription gone, we emptied both task logs and gave it another 60 seconds. The batch log gained 3 lines; the stream log stayed empty:

== alert lines written in 60 s
   0 /var/log/kapacitor/cpu_stream.log
   3 /var/log/kapacitor/cpu_batch.log
   3 total

The only way to feed a stream task now is to write to Kapacitor directly. That is the third [[outputs.influxdb]] block in the Telegraf config above, pointed at port 9092 with the autogen retention policy the TICKscript’s dbrp line expects. After restarting Telegraf the stream log filled again, 5 lines in 60 seconds, and the alert JSON matched the 1.x format:

== stream alert lines in 60 s
5 /var/log/kapacitor/cpu_stream.log
stream0 -> from1 [processed="11"];
alert2 [alerts_inhibited="0" alerts_triggered="11" avg_exec_time_ns="2.922224ms" crits_triggered="0" errors="0" infos_triggered="0" oks_triggered="0" warns_triggered="11" working_cardinality="1" ];

This works, but notice what changed. Every Telegraf agent in the fleet now needs a second output aimed at Kapacitor, and every other writer (application clients, scripts, other agents) has to do the same or its points never reach a stream task. On 1.x the database fanned writes out for you. Now each producer does it, which is the main reason we would not carry stream TICKscripts into InfluxDB 3 long term.

How Chronograf InfluxDB 3 connections work

Chronograf connects to InfluxDB 3 only when its v3 support flag is on. Without it, adding InfluxDB 3 as a regular InfluxDB source with the token as a password fails at the API, and the message gives no hint about why:

{"code":400,"message":"received status code 400 from server: err: invalid mime type (application/json)"}

Chronograf’s systemd unit reads /etc/default/chronograf, so the flag goes there. InfluxData documents this in its Chronograf with InfluxDB 3 Core page:

echo "INFLUXDB_V3_SUPPORT_ENABLED=true" | sudo tee /etc/default/chronograf > /dev/null
sudo systemctl restart chronograf

Once Chronograf restarts with the flag, the connection wizard at port 8888 grows a Server Type field. The figure shows the four controls that matter, in the order you fill them:

  1. Server Type: InfluxDB 3 Core switches Chronograf to the v3 request format that fixes the mime type error.
  2. Connection URL on port 8181 points at the InfluxDB 3 server, here http://127.0.0.1:8181.
  3. Database Token takes the same apiv3_ token stored in ~/.v3token.
  4. Add Connection saves the source.
Chronograf 1.11 connection wizard with Server Type set to InfluxDB 3 Core, the URL on port 8181 and a database token
  1. 1Server Type: InfluxDB 3 Core
  2. 2Connection URL on port 8181
  3. 3Database Token
  4. 4Add Connection

With the source saved, the Data Explorer behaves the way 1.x users expect. It builds InfluxQL from the schema browser, and since SHOW for measurements, tags and fields all work, the pickers fill normally. The mean CPU idle query behind this graph returned identical values from both servers in the curl run earlier. The four marked parts, in the order you set them up:

  1. Source: the influxdb3-core connection is what the query bar’s Dynamic Source resolves to, with InfluxQL selected beside it.
  2. telegraf.autogen (database shown as autogen) is picked in the DB.RetentionPolicy column, where autogen is the retention policy half of the name; more on that below.
  3. usage_idle field from cpu is ticked in the Fields column with one function applied, the mean() you see in the generated query.
  4. Graph drawn from InfluxDB 3 data is the result: the line comes from the v1 /query endpoint on port 8181.
Chronograf Data Explorer graphing mean usage_idle from the telegraf database on InfluxDB 3 Core through InfluxQL
  1. 1Source: the influxdb3-core connection
  2. 2telegraf.autogen (database shown as autogen)
  3. 3usage_idle field from cpu
  4. 4Graph drawn from InfluxDB 3 data

Note that the database shows up as telegraf.autogen, even though InfluxDB 3 itself reports the retention policy as telegraf. The old name still resolves on the server too: the Kapacitor batch query later in this article selects from "telegraf"."autogen"."cpu" and gets data back, so saved queries that name autogen keep working.

The admin side is where Chronograf runs out of road. On the InfluxDB Admin page, outlined below, Create Database is disabled for InfluxDB 3 sources, which matches the CREATE DATABASE failure in the statement table earlier:

Chronograf InfluxDB Admin page for an InfluxDB 3 Core source with the Create Database button greyed out

So Chronograf works as a query and dashboard tool against InfluxDB 3, and stops being an admin console. One more limit comes from InfluxData’s own documentation rather than our lab: dashboard annotations and template variables need a chronograf database on the InfluxDB 3 server that the same token can reach, so check those before moving a dashboard that uses them. Database and token management move to the influxdb3 CLI, which is where we created the admin token at the start.

What replaces Kapacitor: a Processing Engine plugin

The native replacement for Kapacitor is the Processing Engine, an embedded Python runtime inside InfluxDB 3 that runs plugins on a schedule, on every write, or on an HTTP request. Because it runs inside the database, it doesn’t need subscriptions at all: a scheduled plugin queries recent data with SQL and writes results back as line protocol. The plugin documentation covers the three trigger types.

We rewrote the batch TICKscript as a scheduled plugin with the same threshold. It averages usage_idle over the last minute per host, logs a warning, and writes a cpu_alerts row for each host below the threshold. This is the exact file that ran:

def process_scheduled_call(influxdb3_local, call_time, args=None):
    threshold = float((args or {}).get("threshold", "90"))
    rows = influxdb3_local.query(
        "SELECT host, avg(usage_idle) AS idle FROM cpu "
        "WHERE cpu = 'cpu-total' AND time > now() - interval '1 minute' "
        "GROUP BY host"
    )
    for row in rows:
        idle = row["idle"]
        if idle is None or idle >= threshold:
            continue
        influxdb3_local.warn(f"cpu_alert host={row['host']} idle={idle:.3f} below {threshold}")
        line = LineBuilder("cpu_alerts")
        line.tag("host", row["host"])
        line.tag("level", "warn")
        line.float64_field("idle", idle)
        line.float64_field("threshold", threshold)
        influxdb3_local.write(line)

Nothing is imported: LineBuilder and the influxdb3_local handle are injected by the engine. The plugin directory is owned by the influxdb3 user with mode 750, so a plain copy fails. Save the file as /tmp/cpu_alert.py and install it with the right owner:

sudo install -o influxdb3 -g influxdb3 -m 644 /tmp/cpu_alert.py /var/lib/influxdb3/plugins/cpu_alert.py

A trigger binds the file to a database and a schedule. every:10s is the equivalent of the TICKscript’s .every(), and trigger arguments replace hardcoded thresholds, so one plugin file can serve several triggers with different limits. The CLI reads the token from INFLUXDB3_AUTH_TOKEN, exported earlier:

influxdb3 create trigger --database telegraf --trigger-spec every:10s --path cpu_alert.py --trigger-arguments threshold=99.9 cpu_alert

Older guides use --plugin-filename here, which the CLI reference now lists as deprecated in favour of --path. Within 20 seconds the engine had run the plugin twice and both runs fired, which the logs table and the cpu_alerts rows confirm:

Terminal output creating the cpu_alert Processing Engine trigger and reading its logs and cpu_alerts rows in InfluxDB 3 Core

Two system tables replace kapacitor show. system.processing_engine_logs holds every start, finish and warn() line with its timing, and system.processing_engine_triggers shows each trigger’s schedule and error behaviour:

influxdb3 query -d telegraf 'SELECT * FROM system.processing_engine_triggers'

Our trigger was enabled, scheduled every 10 seconds, and set to log errors rather than stop. That last column matters more than it looks: with log, a plugin that throws keeps being scheduled and the failure only shows up in the logs table, so watch that table the way you used to watch kapacitor.log:

+--------------+-----------------+----------------------------------------------+----------+----------------+
| trigger_name | plugin_filename | trigger_specification                        | disabled | error_behavior |
+--------------+-----------------+----------------------------------------------+----------+----------------+
| cpu_alert    | cpu_alert.py    | {"every":{"duration":{"secs":10,"nanos":0}}} | false    | log            |
+--------------+-----------------+----------------------------------------------+----------+----------------+

What the plugin doesn’t give you is Kapacitor’s alert plumbing: state changes, deadman checks, and handlers for Slack, PagerDuty or email. A plugin can call any Python library, so you can post to a webhook yourself, but you write the state tracking. If your TICKscripts lean on .stateChangesOnly() and handler topics, budget real porting time.

What replaces Chronograf: Explorer and Grafana

InfluxDB 3 Explorer is InfluxData’s own web UI for the new engine. It is SQL first, with an InfluxQL tab for older queries, and it is distributed as a Docker image. The host needs Docker, which on Ubuntu is one package: sudo apt-get install -y docker.io. We ran it on the same host, bound to localhost on port 8889, with host.docker.internal mapped so the container can reach InfluxDB 3 on the host:

sudo mkdir -p /opt/influxdb3-ui/db /opt/influxdb3-ui/config
sudo chown -R 1500:1500 /opt/influxdb3-ui
sudo docker run -d --name influxdb3-explorer --restart unless-stopped -p 127.0.0.1:8889:8080 --add-host=host.docker.internal:host-gateway -v /opt/influxdb3-ui/db:/db -v /opt/influxdb3-ui/config:/app-root/config influxdata/influxdb3-ui:1.11.2 --mode=admin

The chown line is the step we learned the hard way. The image runs as UID 1500, and because we pre-created the host directories with sudo mkdir, they were owned by root. With those root-owned bind mounts the container restarted in a loop, printing this once a minute while the port never answered:

Error: Explorer cannot write to its database /db/sqlite.db: the directory is read-only. Nothing was changed.

After the ownership fix and a sudo docker restart influxdb3-explorer, the port answered HTTP 200 within seconds. Its log also warns that SESSION_SECRET_KEY isn’t set, which means logins don’t survive a container restart; set it as an environment variable for anything beyond a lab.

The port is bound to 127.0.0.1, so from a workstation open a tunnel first with ssh -L 8889:127.0.0.1:8889 [email protected] and browse to http://localhost:8889. In Explorer, add a server named influxdb3-core with the URL http://host.docker.internal:8181 and the admin token. The Query Data page then runs SQL against any table. The figure shows the cpu_alerts rows our plugin wrote, read back through the four parts you work with:

  1. Database: telegraf picks which database the editor queries.
  2. SQL against cpu_alerts goes in the wide outlined editor below the toolbar (its number sits on the box’s top edge, just right of Run), here SELECT time, host, level, idle FROM cpu_alerts ORDER BY time DESC LIMIT 6.
  3. Run executes it.
  4. Rows written by the plugin appear in the results table, one per firing.
InfluxDB 3 Explorer Query Data page running SQL against the cpu_alerts table written by the Processing Engine plugin
  1. 1Database: telegraf
  2. 2SQL against cpu_alerts
  3. 3Run
  4. 4Rows written by the plugin

Explorer has dashboards of its own, which InfluxData launched as a beta feature, and the Save to Dashboard button sits next to Export in that screenshot. For the full dashboard estate Chronograf used to hold, Grafana is the more mature answer, and its InfluxDB data source speaks SQL to InfluxDB 3 over Flight. Install it as in our Grafana on Ubuntu and Debian guide, then go to Connections, Data sources, Add new data source and pick InfluxDB. These are the settings we used:

  1. Query language: SQL is the option labelled for InfluxDB 3; it changes the form to database and token fields.
  2. URL: port 8181, the same port the v1 API used, since Core serves Flight on it too.
  3. Database: telegraf, with the token in the field below it.
  4. Insecure Connection on (no TLS), needed while InfluxDB 3 listens on plain HTTP; leave it off once TLS is in front.
  5. Save & test: OK confirms Grafana reached the server with that token.
Grafana InfluxDB data source set to SQL for InfluxDB 3 Core with the database, token and insecure gRPC options
  1. 1Query language: SQL
  2. 2URL: port 8181
  3. 3Database: telegraf
  4. 4Insecure Connection on (no TLS)
  5. 5Save & test: OK

We built a two panel dashboard on that source. The CPU panel buckets with date_bin, the SQL counterpart of InfluxQL’s GROUP BY time(), and both panels use Grafana’s $__timeFilter macro so the time picker still works:

SELECT date_bin(INTERVAL '10 seconds', time) AS time, avg(usage_idle) AS idle FROM cpu WHERE cpu = 'cpu-total' AND $__timeFilter(time) GROUP BY 1 ORDER BY 1

The second panel is a table over the plugin’s output, so alerts and the metric that caused them sit on one screen. Since the plugin writes alerts as ordinary rows, the history is just another SQL query with no separate alert store to manage:

SELECT time, host, level, idle, threshold FROM cpu_alerts WHERE $__timeFilter(time) ORDER BY time DESC LIMIT 8

The result is a dashboard reading CPU idle and Processing Engine alerts straight from InfluxDB 3, with no Kapacitor or Chronograf in the path. Because our threshold sits above anything an idle VM reaches, every plugin run shows up as a row, which is exactly what you want to see while proving the pipeline end to end:

Grafana dashboard reading CPU idle and the Processing Engine cpu_alerts table from InfluxDB 3 Core over SQL

Grafana’s data source also offers InfluxQL, which goes through the same /query endpoint we tested with curl. InfluxData’s Grafana page says that mode wants the token as Password and any non-empty User, which fits our Kapacitor finding that password-only Basic auth fails. We only ran SQL panels, so if you keep an InfluxQL panel library, test the panels that use SHOW SERIES or other statements from the failing rows first. Our InfluxDB 3, Telegraf and Grafana build covers that stack from a clean install.

Which TICK stack replacement path fits your setup

There is no single TICK stack replacement, because the right move depends on how much of your alerting lives in stream TICKscripts and how much you depend on the 1.x admin model. This table is how we’d decide, based on what the lab showed:

Your situationStay on 1.x TICKInfluxDB 3 + GrafanaPrometheus
Many stream TICKscripts with handlersBest fit, nothing to portRewrite as plugins, or dual-write to KapacitorRewrite as alert rules
Mostly batch tasks and dashboardsWorks, but frozenGood fit: Kapacitor batch and Chronograf both ranPossible, larger rewrite
High series cardinalityIndex grows with every seriesBuilt for it, columnar Parquet storageEvery label value is a new series
Want SQL and Python on the dataNoYes, nativeNo, PromQL only
Pull-based infra metrics onlyOverkillWorks with TelegrafBest fit
Cost of change this quarterNoneMedium: configs above plus dashboard queriesHigh: new agents, new queries

Staying on 1.x is a defensible choice for a stable alerting estate, as long as you treat it as a holding position. Moving to InfluxDB 3 pays off when batch tasks and dashboards are most of the work, because the lab showed both carry over with config changes alone. Prometheus makes sense when the data is mostly host and container metrics and you’re rewriting alerts anyway; our open source monitoring tools comparison measures it against the other options.

Where the compatibility story breaks down

The most common misconception is that v1 API compatibility means v1 tool compatibility. The API compatibility covers the data plane: writes and reads. The TICK tools also leaned on the control plane, meaning subscriptions, continuous queries, users and retention policies, and none of that came across. A tool that only writes and reads works; a tool that manages the server doesn’t.

The second is trusting HTTP status codes. Every InfluxQL failure we hit came back as 200 with the error inside the JSON. A migration smoke test that curls each dashboard query and checks for 200 will pass while half the panels are broken. Parse the body for an error key, or run the dashboards and look.

The third is reading a working stream task during a side by side migration as proof it will work after cutover. As long as the old server holds Kapacitor’s subscription, stream tasks keep firing on data the new server never sends. Drop that subscription on purpose before you decide anything, then count the log lines again. If the count drops to zero, you have found the work the migration still owes.

Keep reading

Upgrade Ubuntu 24.04 to Ubuntu 26.04 LTS (Step by Step) Ubuntu Upgrade Ubuntu 24.04 to Ubuntu 26.04 LTS (Step by Step) UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 Security UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 Backup and Restore Linux Systems with Timeshift Debian Backup and Restore Linux Systems with Timeshift Deploy Collabora Online Office on Ubuntu with Let’s Encrypt SSL Cloud Deploy Collabora Online Office on Ubuntu with Let’s Encrypt SSL Install CyberPanel on Ubuntu 26.04 / 24.04 with Let’s Encrypt Ubuntu Install CyberPanel on Ubuntu 26.04 / 24.04 with Let’s Encrypt Install Sourcegraph code search engine on Ubuntu 24.04|22.04|20.04| Git Install Sourcegraph code search engine on Ubuntu 24.04|22.04|20.04|

Leave a Comment

Press ESC to close